Method for detecting rail transit line abnormity, electronic equipment, storage medium and rail vehicle

Abstract

The invention discloses a method for detecting rail transit line abnormity, an electronic device, a storage medium and a rail vehicle, the method for detecting the rail transit line abnormity comprises the following steps: obtaining rail defect information of a running line of the rail vehicle according to a first image, and obtaining obstacle information on the running line of the rail vehicle according to a second image, wherein the first image is acquired by a bottom camera at the bottom of the railway vehicle, and the second image is acquired by a top camera at the top of the railway vehicle; and sending track defect information and obstacle information. By adopting the method, different targets can be respectively detected through the bottom camera and the top camera, so that the error-tolerant rate of the system is higher, the requirements on hardware and algorithms are lower, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit, and in particular to a method for detecting abnormality of a rail transit line, as well as electronic equipment, a storage medium and a rail vehicle. Background Art

[0002] In the related technologies, there are two methods for detecting anomalies in rail transit: one is based on video sequences and uses a timing network or a converter model for recognition; the other is based on a convolutional neural network method and detects single-frame images; the hardware equipment basically uses ordinary color cameras, and there is no granularity division for the objects to be detected, which greatly reduces the detection rate. At the same time, the detection method based on video sequences requires higher computing power and places higher requirements on hardware costs. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one purpose of the present invention is to provide a method for detecting anomalies in a rail transit line, which can detect different targets respectively through a first camera and a second camera, so that the system has a higher fault tolerance rate, lower requirements on hardware and algorithms, and improved detection efficiency.

[0004] A second objective of the present invention is to provide an electronic device.

[0005] The third object of the present invention is to provide a non-volatile readable storage medium.

[0006] A fourth objective of the present invention is to provide a rail vehicle.

[0007] In order to solve the above problems, an embodiment of the first aspect of the present invention provides a method for detecting abnormalities of a rail transit line, comprising: obtaining defect information of a track itself on the line on which a rail vehicle is traveling based on a first image, and obtaining obstacle information on the line on which the rail vehicle is traveling based on a second image, wherein the first image is acquired by a first camera and the second image is acquired by a second camera; and sending the defect information of the track itself and the obstacle information.

[0008] According to the method for detecting abnormalities of rail transit lines in an embodiment of the present invention, the method can detect different targets respectively through the first camera and the second camera, so that the fault tolerance rate of the system is higher, and the detection information is sent to the monitoring center at the same time, the defect information of the track itself and the obstacle information are analyzed and judged, the potential risks of the line are predicted and reminded in time, and obstacle avoidance is achieved to prevent accidents before they happen.

[0009] In some embodiments, the defect information of the track itself is obtained by performing image recognition on the first image using a lightweight neural network.

[0010] In some embodiments, the track defect information includes the type of track defect, the line location where the track defect exists, and the accuracy of identifying the track defect.

[0011] In some embodiments, the types of defects in the rails themselves include rail defects and rail cracks.

[0012] In some embodiments, the obstacle information includes whether there is an obstacle on the track and the location information of the obstacle on the track line.

[0013] In some embodiments, whether there is an obstacle on the track is determined based on image intersection information; the image intersection information is the image intersection of the track area and the obstacle area identified based on the second image.

[0014] In some embodiments, the track area is obtained by segmenting the second image using a deep learning segmentation method.

[0015] In some embodiments, the obstacle area is obtained based on a target binary image; the target binary image is obtained by comparing a first sliding window processed image and a second sliding window processed image, wherein the first sliding window processed image is obtained by performing sliding window processing on the second image, and the second sliding window processed image is obtained by performing sliding window processing on a homography transformation image; the homography transformation image is obtained by performing a homography transformation on the second image.

[0016] In some embodiments, performing a homography transformation on the second image includes: obtaining a plurality of base maps corresponding to route locations of the second images; obtaining a target base map, the target base map being a base map in which the number of matches between feature points in the plurality of base maps and target feature points of the second image is greater than a quantity threshold; and obtaining the homography matrix diagram, the homography matrix diagram being obtained by performing a homography transformation on the target base map based on the target feature points of the second image.

[0017] In some embodiments, sliding window processing is performed on the second image, including: obtaining a gradient map of the second image; traversing the gradient map of the second image with a first preset sliding window, wherein a first pixel mean within the first preset sliding window is obtained, and pixel points scanned by the first preset sliding window are assigned to the corresponding first pixel mean; the first sliding window processed image is a set of all pixel points in the gradient map of the second image that are assigned to the corresponding first pixel mean.

[0018] In some embodiments, sliding window processing is performed on the homography transformation image, including: obtaining a gradient map of the homography transformation image; traversing the gradient map of the homography transformation image with a second preset sliding window, wherein a second pixel mean within the second preset sliding window is obtained, and pixel points scanned by the second preset sliding window are assigned to the corresponding second pixel mean; the second sliding window processed image is a set of all pixel points in the gradient map of the homography transformation image that are assigned to the corresponding second pixel mean.

[0019] In some embodiments, the target binary image is an image marked as 255 when the pixel difference is greater than or equal to the pixel threshold and marked as 0 when the pixel difference is less than the pixel threshold; the pixel difference is the absolute value of the pixel difference between the first sliding window processed image and the second sliding window processed image.

[0020] In some embodiments, the obstacle area is a contour area in the total binary image whose contour area is greater than an area threshold; the total binary image is obtained by performing an AND operation on each of the target binary images, and the contour is obtained by dilating the total binary image and performing contour extraction.

[0021] In some embodiments, the first camera is a bottom camera located at the bottom of the rail vehicle; and the second camera is a top camera located at the top of the rail vehicle.

[0022] A second aspect of the present invention provides an electronic device, comprising: at least one processor; a memory communicatively connected to the at least one processor; a computer program executable by the at least one processor is stored in the memory, and when the at least one processor executes the computer program, the method for detecting abnormalities in a rail transit line described in the above embodiment is implemented.

[0023] According to the electronic device of the embodiment of the present invention, the corresponding program for detecting abnormalities of the rail transit line can be stored in the memory. When implementing the method for detecting abnormalities of the rail transit line, the processor runs the program in the memory to enable the bottom camera and the top camera to detect different targets respectively, and at the same time send the detection information to the monitoring center to analyze and judge the defect information and obstacle information of the track itself.

[0024] A third aspect of the present invention provides a non-volatile readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the method for detecting anomalies in a rail transit line as described in the above embodiment.

[0025] A fourth aspect of the present invention provides a rail vehicle, comprising: a first camera, which is a bottom camera located at the bottom of the rail vehicle; a second camera, which is a top camera located at the top of the rail vehicle; and a controller, which is respectively connected to the bottom camera and the top camera, and is used to execute the method for detecting abnormalities in rail transit routes described in the above embodiment.

[0026] According to the rail vehicle of the embodiment of the present invention, during driving, different targets are detected respectively by the bottom camera and the top camera, and full-scene, all-weather rail transit route abnormality detection is realized by combining positioning and detection. The detection information is sent to the monitoring center, and the defect information and obstacle information of the track itself are analyzed and judged. The potential risks of the line are predicted and reminded in time, so as to avoid obstacles and prevent them before they happen.

[0027] In some embodiments, the optical axis of the bottom camera is perpendicular to the track of the rail vehicle travel route.

[0028] In some embodiments, the bottom camera is a thermal imaging camera.

[0029] In some embodiments, the resolution of the bottom camera's lens is higher than the resolution of the standard lens.

[0030] In some embodiments, the optical axis of the top camera is parallel to the track of the rail vehicle travel route.

[0031] In some embodiments, the top camera is a thermal imaging camera.

[0032] In some embodiments, the focal length of the lens of the top camera is greater than the focal length of the standard lens.

[0033] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0035] Figure 1 is a flow chart of a method for detecting anomalies in a rail transit line according to an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of a track defect according to an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of a track crack according to an embodiment of the present invention;

[0038] Figure 4is a schematic diagram of a track obstacle according to an embodiment of the present invention;

[0039] Figure 5 is a flow chart of a bottom camera algorithm according to one embodiment of the present invention;

[0040] Figure 6 is a flow chart of a top camera according to one embodiment of the present invention;

[0041] Figure 7 is a structural block diagram of an electronic device according to an embodiment of the present invention;

[0042] Figure 8 is a block diagram of a rail vehicle according to an embodiment of the present invention;

[0043] Fig. 9 is a schematic diagram of a rail vehicle according to an embodiment of the present invention.

[0044] Reference numerals:

[0045] Electronic device 100; rail vehicle 200;

[0046] Processor 101 ; memory 102 ; bottom camera 201 ; top camera 202 ; controller 203 . DETAILED DESCRIPTION

[0047] Embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. Embodiments of the present invention are described in detail below.

[0048] Reference below Figure 1 A method for detecting an abnormality of a rail transit line according to an embodiment of the first aspect of the present invention is described as follows: Figure 1 As shown, the method at least includes step S1 to step S2.

[0049] Step S1, obtaining defect information of the track itself on the route on which the rail vehicle travels according to the first image, and obtaining obstacle information on the route on which the rail vehicle travels according to the second image.

[0050] The first image is acquired by a bottom camera at the bottom of the rail vehicle, and the second image is acquired by a top camera at the top of the rail vehicle.

[0051] Specifically, the first image can be understood as being acquired by the bottom camera installed at the bottom of the rail vehicle. When the rail vehicle is running, the bottom camera installed at the bottom of the rail vehicle collects the rail image in real time, and analyzes the collected rail image, i.e., the first image, to obtain the defect information of the rail itself on the route where the rail vehicle is running; the second image can be understood as being acquired by the top camera installed at the top of the rail vehicle. When the rail vehicle is running, the top camera installed at the top of the rail vehicle collects the obstacle image on the rail line in real time, and analyzes the collected obstacle image, i.e., the second image, to obtain the obstacle information on the route where the rail vehicle is running. The two cameras are combined and installed at different positions to detect different anomalies and jointly maintain the normal operation of the train.

[0052] Step S2, sending the track defect information and obstacle information.

[0053] Specifically, the defect information and obstacle information of the track itself are sent to the monitoring and operation and maintenance center. The abnormality detection of the track line is carried out according to the defect information and obstacle information of the track itself and the position information of the rail vehicle. The position information is provided by the positioning module of the rail vehicle, and the type, location and level of the abnormality of the track line are finally obtained, which is convenient for the operation and maintenance personnel to analyze, so that the abnormal location and level can be quickly and accurately located, and maintenance can be carried out in advance to avoid rail traffic interruption as much as possible. According to the defect information of the track itself, it is judged whether the track needs to be maintained. When the track needs maintenance, the rail vehicle is prohibited from passing. After the defect information is removed, the rail vehicle can pass to ensure the safety of rail vehicle driving; according to the obstacle information, it is judged whether the rail vehicle needs to stop in an emergency. If an obstacle is detected on the track line, an early alarm is required to notify the rail vehicle to stop, and the traffic can be opened after the obstacle is removed.

[0054] According to the method for detecting abnormalities of rail transit lines in an embodiment of the present invention, the method can detect different targets through the bottom camera and the top camera respectively, so that the fault tolerance rate of the system is higher. At the same time, the detection information is sent to the monitoring center, and the defect information and obstacle information of the track itself are analyzed and judged, and the potential risks of the line are predicted and reminded in time, so as to avoid obstacles and prevent them before they happen.

[0055] In some embodiments, the defect information of the track itself is obtained by performing image recognition on the first image using a lightweight neural network.

[0056] Specifically, the characteristics of lightweight neural networks mainly include small model size and small number of parameters, while maintaining performance similar to that of heavier models, thereby realizing hardware-friendly neural networks. The design of lightweight neural networks aims to reduce the complexity and computational overhead of the model so that it can run efficiently on hardware with limited resources.

[0057] The first image is acquired by the bottom camera at the bottom of the vehicle. Since the bottom camera is perpendicular to the track surface and has a fixed installation position, the recognition area of ​​the image can be reduced, which can minimize misidentification. The defect information of the track itself is obtained by using a lightweight neural network to perform image recognition on the first image, such as ShuffleNet, MobileNet, HRNet and other series. Therefore, data collection and sample annotation are required in advance, and the result score of the recognition network is recognized. After the recognition is completed, the system will push the original image with positioning information to the monitoring center.

[0058] In some embodiments, the track defect information includes the type of track defect, the line location where the track defect exists, and the accuracy of identifying the track defect.

[0059] Specifically, the track defect information includes multiple types. After the track defect information is sent to the monitoring and operation center, the track defect information is analyzed; the track defect type in the track defect information is used to determine the damage type of the track, so that maintenance personnel can perform corresponding rapid repairs according to the damage type; the line position information in the track defect information is used to determine the specific location of the track defect. After determining the specific location of the track defect, the driving section where the track defect is located is determined to prohibit the operation of track trains, and the track defects at the corresponding location are maintained; the track defect accuracy information in the track defect information is used to score the track defects. The track defect information is scored through a lightweight neural network to determine the degree of harm of the track defects. If the track defect score is high, it means that the track defect is highly harmful and the track should be maintained immediately.

[0060] In some embodiments, the types of defects in the rails themselves include rail defects and rail cracks.

[0061] Specifically, the track defect type of the present invention only identifies track defects and track cracks, and of course it can be expanded to other defects as needed. Track defects and track cracks are as follows: Figure 2 and Figure 3 shown.

[0062] Track defects should be repaired in a timely manner. The repair methods are generally divided into two types: manual repair and mechanized repair. Manual repair refers to pouring welding materials on the track defects and then grinding them flat. Mechanized repair often uses methods such as welding or hot pressing. The repaired track must meet the requirements of the railway industry and be able to maintain a certain flatness and track geometry to ensure the normal operation of the train.

[0063] Broken rails can be repaired using aluminothermic welding technology. This method uses the heat generated by thermite reaction to melt molten iron and fill the broken gap. First, a mold is fixed at the fracture, and then a mixture of aluminum powder and magnesium oxide is ignited. The two substances undergo a replacement reaction, generating a large amount of heat, causing the iron to be burned into liquid form. The flowing molten iron enters the broken rail and forms a solid connection after cooling.

[0064] In some embodiments, the obstacle information includes whether there is an obstacle on the track and the location information of the obstacle on the track line.

[0065] Specifically, track obstacles such as Figure 4 As shown, the obstacle information is used to determine whether there are obstacles on the track, which include people, stones, branches, etc.; based on the obstacle information, it is determined whether the rail vehicle needs to make an emergency stop. If an obstacle is detected on the track line, an early warning is required to notify the rail vehicle to stop, and the traffic can be resumed after the obstacle is removed; the obstacle information also includes the location information of the obstacle on the track line. After it is detected that there is an obstacle on the track line and the rail vehicle cannot pass, the specific location of the obstacle is determined based on the location information of the obstacle on the track line, and the obstacle is removed.

[0066] In some embodiments, whether there is an obstacle on the track is determined based on image intersection information; the image intersection information is the image intersection of the track area and the obstacle area identified based on the second image.

[0067] Specifically, when the rail vehicle is running, the top camera located on the top of the rail vehicle obtains a second image in front of the rail vehicle in real time, divides the obtained second image into a track area and an obstacle area, analyzes and calculates the track area and the obstacle area respectively to determine the obstacle information, takes the intersection of the images of the track area and the obstacle area to obtain image intersection information, and determines whether there is an obstacle on the track based on the image intersection information; the image intersection information can be understood as the image information obtained by taking the intersection of the track area and the obstacle area.

[0068] In some embodiments, the track area is obtained by segmenting the second image using a deep learning segmentation method.

[0069] Specifically, the second image is segmented to divide the track area. The purpose of track area segmentation is to determine whether the obstacle is in the track area. The segmentation method adopts a deep learning segmentation method, such as RebiSeNet, deeplabv3 and other algorithms. The data annotation range includes all areas between the two tracks. If the obstacle is between the two tracks, the data is annotated to confirm that there is an obstacle on the track.

[0070] In some embodiments, the obstacle area is obtained based on the target binary image; the target binary image is obtained by comparing the first sliding window processed image with the second sliding window processed image.

[0071] The first sliding window processed image is obtained by performing sliding window processing on the second image, the second sliding window processed image is obtained by performing sliding window processing on the homography transformed image; and the homography transformed image is obtained by performing homography transformation on the second image.

[0072] Specifically, homography transformation can be simply understood as a method used to describe the position mapping relationship between an object in the world coordinate system and the pixel coordinate system. The corresponding transformation matrix is ​​called the homography matrix. Homography transformation plays a very important role in image correction, image stitching, camera pose estimation, vision and other fields.

[0073] The sliding window algorithm is an algorithm that operates on a string or array of a specific size. It reduces the complexity of the problem by maintaining a window and performing operations on it. The core idea of ​​this algorithm is to use two pointers (left pointer and right pointer) to maintain a window, expand the window by moving the right pointer, and reduce the window by adjusting the left pointer according to the requirements of the problem. In the process of expanding or reducing the window, some intermediate results, such as the maximum value, minimum value, substring length, etc., can be recorded to solve the final answer to the problem.

[0074] After the top camera on the top of the rail vehicle acquires the second image, the second image is subjected to sliding window processing to obtain the first sliding window processed image, so the first sliding window processed image can be understood as the image directly obtained by performing sliding window processing on the second image; after the top camera on the top of the rail vehicle acquires the second image, the second image is subjected to sliding window processing and homography transformation is performed on the second image to obtain the homography transformation image, so the homography transformation image can be understood as the image directly obtained by performing homography transformation processing on the second image, the homography transformation image obtained by performing homography transformation on the second image is subjected to sliding window processing to obtain the second sliding window processed image, so the second sliding window processed image can be understood as the image obtained by performing homography transformation on the second image and then performing sliding window processing. The first sliding window processed image and the second sliding window processed image are compared to obtain a target binary image, and the obstacle area is obtained according to the target binary image.

[0075] In some embodiments, performing a homography transformation on the second image includes: obtaining a base map of multiple second images corresponding to route locations; obtaining a target base map, the target base map being a base map in which the number of matches between feature points in the multiple base maps and target feature points of the second image is greater than a quantity threshold; and obtaining a homography matrix diagram, the homography matrix diagram being obtained by performing a homography transformation on the target base map based on the target feature points of the second image.

[0076] Specifically, in order to obtain the base map of the line positions corresponding to multiple second images, before the system is run, it is necessary to establish a BOW (Bag of Words) and a video base library, that is, the train runs along the track once and collects a video stream, and each frame of the video carries positioning information. Since the track line is long, the video will be sampled to reduce the storage volume. BOW is a library composed of feature points of each frame of the video, that is, the feature points of each frame will be stored as a matrix. BOW is the most commonly used method in the field of image retrieval and the most basic algorithm in content-based image retrieval.

[0077] After the video base library is established, there are multiple base maps of the line position corresponding to the second image in the video base library. SuperPoint is used to extract the feature points of the current frame. The SuperGlue feature matching method is used to select the M-frame base map closest to the current position in the BOW for matching, so as to determine the obstacle information, where M can be 10 or 20; after obtaining the M-frame base map, it is matched with the number of feature points, and the base map that can distinguish the obstacle information is retained, that is, the base map whose matching number of feature points in multiple base maps and the target feature points of the second image is greater than the number threshold. The number threshold can be understood as the minimum value of the number of target feature points matched between the base map and the second image. For example, the number threshold can be set to W. If the matching number of target feature points is less than W, it is discarded, where the number of feature points W can be 8 or 10. The target base map is obtained by comparing multiple base maps with the target feature points of the second image; the target base map is homographed to obtain the homograph matrix diagram, which is obtained by homographing the target base map according to the target feature points of the second image. The homograph matrix is ​​calculated by matching feature points. The calculation method of the homograph matrix H is as follows:

[0078]

[0079] Among them, p2 represents the feature point of the current frame, p1 represents the feature point in BOW, and the selected M frame base map is I i (i=1…N), the corresponding homography transformation matrix is ​​H i (i=1…N), perform homography transformation on the N-frame base image to obtain I hi (i=1…N).

[0080] Since the M-frame base map closest to the current position is selected for matching in BOW, the base map whose number of matches between the feature points of the M-frame base map and the target feature points of the second image is greater than the quantity threshold is the target base map, and the target base map is subjected to a homography transformation to obtain a homography matrix diagram, so N is less than or equal to M.

[0081] SuperPoint is a feature point extraction algorithm based on self-supervised learning. It learns to extract stable and repeatable feature points from input images by using the correspondence between input images during the training process. The SuperPoint algorithm has high robustness and computational efficiency, and can be used to extract feature points in various scenarios. It has a wide range of applications, including but not limited to computer vision, 3D reconstruction, object recognition and other fields.

[0082] SuperGlue is a model for key point matching. Its basic idea is to simulate the process of human eyes finding matching relationships, that is, browsing two images back and forth to enhance the specificity of feature points according to contextual relationships. The SuperGlue framework is divided into an attention graph neural network module and an optimal matching module. The implementation of SuperGlue introduces the attention mechanism into the network to learn the relationship between key points within and between images, thereby constructing a distribution matrix (which can be understood as the similarity between key points of two images), and finally solving the distribution result.

[0083] In some embodiments, sliding window processing is performed on the second image, including: obtaining a gradient map of the second image; traversing the gradient map of the second image with a first preset sliding window, wherein a first pixel mean within the first preset sliding window is obtained, and pixel points scanned by the first preset sliding window are assigned corresponding first pixel means; the first sliding window processed image is a set of all pixel points in the gradient map of the second image that are assigned corresponding first pixel means.

[0084] Specifically, the sliding window algorithm is an algorithm that operates on a string or array of a specific size. It reduces the complexity of the problem by maintaining a window and performing operations on it. The core idea of ​​this algorithm is to use two pointers (left pointer and right pointer) to maintain a window, expand the window by moving the right pointer, and reduce the window by adjusting the left pointer according to the requirements of the problem. In the process of expanding or reducing the window, some intermediate results can be recorded, such as the maximum value, minimum value, substring length, etc., to solve the final answer to the problem.

[0085] In order to remove the influence of the surface temperature and light of the object, the original picture of the second image is transformed into a gradient map, and the gradient map of the second image is traversed through the first preset sliding window. The first preset sliding window can be understood as a sliding window for calculating the first pixel mean; after the first preset sliding window traverses the gradient map of the second image, the pixel points of each gradient map scanned by the first preset sliding window are recorded, and the pixel mean is calculated, and the calculated pixel mean is assigned to the corresponding first pixel mean. The first pixel mean can be understood as the pixel mean calculated after the first preset sliding window traverses the gradient map of the second image; when the image is processed with the first sliding window, the first sliding window processed image is a set of all pixel points in the gradient map of the second image that are assigned to the corresponding first pixel mean.

[0086] In some embodiments, sliding window processing is performed on the homography transformation image, including: obtaining a gradient map of the homography transformation image; traversing the gradient map of the homography transformation image with a second preset sliding window, wherein a second pixel mean within the second preset sliding window is obtained, and the pixel points scanned by the second preset sliding window are assigned to the corresponding second pixel mean; the second sliding window processed image is a set of all pixel points in the gradient map of the homography transformation image that are assigned to the corresponding second pixel mean.

[0087] Specifically, the homography transformation image is first transformed into a gradient map, and the gradient map of the homography transformation image is traversed through a second preset sliding window, and the second preset sliding window can be understood as a sliding window for calculating a second pixel mean; after the second preset sliding window traverses the gradient map of the homography transformation image, the pixel points of each gradient map scanned by the second preset sliding window are recorded, and the pixel mean is calculated, and the calculated pixel mean is assigned to the corresponding second pixel mean, and the second pixel mean can be understood as the pixel mean calculated after the second preset sliding window traverses the gradient map of the homography transformation image; when processing the gradient map of the homography transformation image with two sliding windows, the second sliding window processed image is a set of all pixel points in the gradient map of the homography transformation image that are assigned to the corresponding second pixel mean.

[0088] For example, suppose the gradient map of the current frame image is G c (The gradient map can be calculated using the Sobel operator), traverse G c Medium pixel p ij (i=1…R,j=1…C,R is the number of image rows, C is the number of image columns), use a sliding window of size (h,w) (h is height, w is width) to scan each pixel, calculate the mean value m within the sliding window, and set the currently scanned pixel point to be p rc , r is the row coordinate, c is the column coordinate, then Then at this time p rc =m. Similarly, for I hi (i=1…N), adopt the same strategy, assuming that the final processed image of the current frame is I pc ,I hi (i=1…N) The corresponding processed image is I hpi (i=1...N), find I di =abs(I pc -I hpi ), i=1...N, for I di Perform threshold processing.

[0089] In some embodiments, the target binary image is an image marked as 255 when the pixel difference is greater than or equal to the pixel threshold and marked as 0 when the pixel difference is less than the pixel threshold; the pixel difference is the absolute value of the pixel difference between the first sliding window processed image and the second sliding window processed image.

[0090] Specifically, after obtaining the pixel value of the first sliding window processed image and the pixel value of the second sliding window processed image, the pixel value of the first sliding window processed image is subtracted from the pixel value of the second sliding window processed image and then the absolute value is taken to obtain the pixel difference; let the pixel threshold be T, the pixel difference calculated from the first sliding window processed image and the second sliding window processed image is compared with the pixel threshold T, when it is greater than or equal to the pixel threshold, it is marked as 255, and when it is less than the pixel threshold, it is marked as 0, and finally the target binary image is obtained as B i (i=1…N), for B i Execute AND operation, that is, B=B1&B2…&B N , perform dilation on B.

[0091] In some embodiments, the obstacle area is a contour area in the total binary image whose contour area is greater than an area threshold; the total binary image is obtained by performing an AND operation on each target binary image, and the contour is obtained by dilating the total binary image and performing contour extraction.

[0092] Specifically, after obtaining multiple target binary images that are closest to the current obstacle area, each target binary image is subjected to an AND operation to obtain a total binary image, the total binary image is dilated and contour extraction is performed to obtain the obstacle contour, and the contour area in the total binary image whose area is greater than the area threshold is set as the obstacle area. The area threshold can be understood as the maximum value of the obstacle area that affects the normal passage of rail vehicles. The contour in the total binary image is extracted, and the area of ​​the contour is determined. By comparing the contour surface with the area threshold, it is determined whether it is an obstacle, thereby obtaining the obstacle area.

[0093] Reference below Figure 5 The bottom camera algorithm steps of the embodiment of the present invention are illustrated by way of example, and the specific contents are as follows.

[0094] Step S3: The bottom camera acquires an image.

[0095] Step S4, the track area is determined.

[0096] Step S5, lightweight neural network.

[0097] Step S6, recognition result (category, accuracy).

[0098] Step S7: the original image with the identification category and accuracy is sent to the monitoring center.

[0099] Reference below Figure 6The top camera algorithm steps of the embodiment of the present invention are illustrated by way of example, and the specific contents are as follows.

[0100] Step S8: The top camera acquires an image.

[0101] Step S9: feature point extraction.

[0102] Step S10, establishing a BOW model and a video base.

[0103] Step S11, position.

[0104] Step S12, track area segmentation.

[0105] Step S13, homography change.

[0106] Step S14: image comparison.

[0107] Step S15, abnormal area.

[0108] Step S16, detection results.

[0109] A second aspect of the present invention provides an electronic device, such as Figure 7 As shown, the electronic device 100 includes: at least one processor 101 and a memory 102 .

[0110] The memory 102 stores a computer program that can be executed by at least one processor 101 , and the at least one processor 101 implements a method for detecting abnormalities in a rail transit line when executing the computer program.

[0111] According to the electronic device of the embodiment of the present invention, the corresponding program for detecting abnormalities of the rail transit line can be stored in the memory. When implementing the method for detecting abnormalities of the rail transit line, the processor runs the program in the memory to enable the bottom camera and the top camera to detect different targets respectively, and at the same time send the detection information to the monitoring center to analyze and judge the defect information and obstacle information of the track itself.

[0112] A third aspect of the present invention provides a non-volatile readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the method for detecting anomalies in a rail transit line of the above embodiment.

[0113] A fourth aspect of the present invention provides a rail vehicle, such as Figure 8 As shown, the rail vehicle 200 includes:

[0114] Bottom camera 201 , top camera 202 and controller 203 .

[0115] The bottom camera 201 is located at the bottom of the rail vehicle 200, the top camera 202 is located at the top of the rail vehicle 200, and the controller 203 is connected to the bottom camera 201 and the top camera 202 respectively, for executing the method for detecting abnormalities in the rail transit route.

[0116] Specifically, when the rail vehicle 200 travels along the track, the controller 203 controls the bottom camera 201 and the top camera 202 to obtain the track defect information and obstacle information in real time, and sends the obtained track defect information and obstacle information to the operation and maintenance center, and performs abnormal detection of the track line based on the track defect information and obstacle information and the rail vehicle position information.

[0117] According to the rail vehicle of the embodiment of the present invention, during driving, different targets are detected respectively by the bottom camera and the top camera, and full-scene, all-weather rail transit route abnormality detection is realized by combining positioning and detection. The detection information is sent to the monitoring center, and the defect information and obstacle information of the track itself are analyzed and judged. The potential risks of the line are predicted and reminded in time, so as to avoid obstacles and prevent them before they happen.

[0118] In some embodiments, the optical axis of the bottom camera 201 is perpendicular to the track of the rail vehicle 200 travel route.

[0119] Specifically, Fig. 9 As shown, the bottom camera 201 is located at the bottom of the rail vehicle 200, the optical axis of the bottom camera 201 is perpendicular to the track of the rail vehicle 200, and the camera field of view of the bottom camera 201 is used to determine the defects of the track itself.

[0120] In some embodiments, bottom camera 201 is a thermal imaging camera.

[0121] Specifically, thermal imaging cameras collect medium and long infrared light with a wavelength range of 7.5μm to 13.5μm, while the atmosphere, fog, smoke and dust absorb visible light and near infrared, but are "transparent" to medium and long infrared light in the above wavelength range. Therefore, it is difficult for traditional visible light cameras to capture clear images in foggy environments, while thermal imaging cameras can effectively penetrate haze, smoke, dust and other harsh environments to capture clear images, reduce light interference, and achieve full working conditions.

[0122] In some embodiments, the resolution of the lens of bottom camera 201 is higher than the resolution of a standard lens.

[0123] Specifically, lens resolution, also known as resolution or clarity, is one of the quality indicators for evaluating the optical system of a camera lens. The higher the lens resolution, the more it can show the most subtle differences in the image. Because the bottom camera 201 needs to detect subtle defects in the track, a high-resolution lens can capture more subtle track defects.

[0124] In some embodiments, the optical axis of the top camera 202 is parallel to the track of the rail vehicle 200 travel route.

[0125] Specifically, Fig. 9 As shown, the top camera 202 is located on the top of the rail vehicle 200 , the optical axis of the top camera 202 is parallel to the track of the rail vehicle 200 , and the camera field of view of the top camera 202 is used to determine the obstacle information on the route of the rail vehicle 200 .

[0126] In some embodiments, top camera 202 is a thermal imaging camera.

[0127] Specifically, thermal imaging cameras can effectively penetrate harsh environments such as haze, smoke, and dust to capture clear images. At the same time, the distance at which they can detect targets is also farther than that of visible light monitoring, reaching more than 10 km. Therefore, the top camera 202 can accurately obtain obstacle information more than 10 km in front of the rail vehicle 200, thereby improving the early warning capability of the rail vehicle 200.

[0128] In some embodiments, the focal length of the lens of the top camera 202 is greater than the focal length of a standard lens.

[0129] Specifically, focal length is a measure of the convergence or divergence of light in an optical system, and refers to the distance from the optical center of the lens to the focus of light convergence when parallel light is incident; the size of the focal length is proportional to the size of the image, the larger the focal length, the larger the image, and the smaller the focal length, the smaller the image. In order to facilitate the judgment of obstacle information, the focal length of the lens of the top camera 202 is greater than the focal length of the standard lens, so that the image is larger, longer distance detection can be achieved, and the judgment of obstacles is convenient.

[0130] In the description of this specification, any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code including one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0131] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0132] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0133] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0135] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0136] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, substrates, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example.

[0137] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for detecting abnormalities in a rail transit line, characterized in that: include: Obtaining defect information of a track on a route traveled by a rail vehicle according to a first image, and obtaining obstacle information on the route traveled by the rail vehicle according to a second image, wherein the first image is acquired by a first camera and the second image is acquired by a second camera; The track defect information and the obstacle information are sent.

2. The method according to claim 1, characterized in that The defect information of the track itself is obtained by performing image recognition on the first image using a lightweight neural network.

3. The method according to claim 2, characterized in that The track defect information includes the type of track defect, the line location where the track defect exists, and the accuracy of identifying the track defect.

4. The method according to claim 2, characterized in that: The types of track defects include track defects and track cracks.

5. The method according to any one of claims 1 to 4, characterized in that: The obstacle information includes whether there is an obstacle on the track and the position information of the obstacle on the track line.

6. The method according to claim 5, characterized in that Whether there is an obstacle on the track is determined based on image intersection information; The image intersection information is the image intersection of the track area and the obstacle area identified according to the second image.

7. The method according to claim 6, characterized in that The track area is obtained by segmenting the second image using a deep learning segmentation method.

8. The method according to claim 6, characterized in that The obstacle area is obtained according to the target binary image; The target binary image is obtained by performing image comparison between a first sliding window processed image and a second sliding window processed image, wherein the first sliding window processed image is obtained by performing sliding window processing on the second image, and the second sliding window processed image is obtained by performing sliding window processing on the homography transformation image; The homography transformed image is obtained by performing homography transformation on the second image.

9. The method according to claim 8, characterized in that Performing homography transformation on the second image includes: Obtaining a plurality of base maps of route locations corresponding to the second images; Obtaining a target base map, wherein the target base map is a base map in which the number of matching feature points in the base maps and the target feature points of the second image is greater than a number threshold; The homography matrix image is obtained by performing a homography transformation on the target base image according to the target feature points of the second image.

10. The method according to claim 8, characterized in that Performing sliding window processing on the second image includes: Obtaining a gradient map of the second image; Traversing the gradient map of the second image with a first preset sliding window, wherein a first pixel mean value within the first preset sliding window is obtained, and pixel points scanned by the first preset sliding window are assigned corresponding first pixel mean values; The first sliding window processed image is a set of all pixel points in the gradient map of the second image that are assigned the corresponding first pixel mean value.

11. The method according to claim 8, characterized in that Perform sliding window processing on the homography transformation image, including: Obtaining a gradient map of the homography transformed image; Traversing the gradient map of the homography transformation image with a second preset sliding window, wherein a second pixel mean within the second preset sliding window is obtained, and pixel points scanned by the second preset sliding window are assigned corresponding second pixel mean values; The second sliding window processed image is a set of all pixel points in the gradient map of the homography transformed image that are assigned the corresponding second pixel mean value.

12. The method according to claim 8, characterized in that The target binary image is an image marked as 255 when the pixel difference is greater than or equal to the pixel threshold and marked as 0 when the pixel difference is less than the pixel threshold; The pixel difference value is an absolute value of a pixel difference between the first sliding window processed image and the second sliding window processed image.

13. The method according to claim 8, characterized in that The obstacle area is a contour area in the total binary image whose contour area is greater than an area threshold; The total binary image is obtained by performing an AND operation on each of the target binary images, and the contour is obtained by performing a dilation process on the total binary image and performing contour extraction.

14. The method according to claim 1, characterized in that The first camera is a bottom camera located at the bottom of the rail vehicle; the second camera is a top camera located at the top of the rail vehicle.

15. An electronic device, characterized in that: include: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and when the at least one processor executes the computer program, the method for detecting anomalies of a rail transit line according to any one of claims 1 to 13 is implemented.

16. A non-volatile readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the method for detecting rail transit line abnormalities according to any one of claims 1 to 13 is implemented.

17. A rail vehicle, characterized in that: include: a first camera, the first camera being a bottom camera located at the bottom of the rail vehicle; a second camera, the second camera being a top camera located on top of the rail vehicle; A controller is connected to the first camera and the second camera respectively, and is used to execute the method for detecting abnormalities in a rail transit route as described in any one of claims 1-13.

18. The rail vehicle according to claim 16, characterized in that The optical axis of the bottom camera is perpendicular to the track of the rail vehicle travel route.

19. The rail vehicle according to claim 16, characterized in that: The bottom camera is a thermal imaging camera.

20. The rail vehicle according to claim 16, characterized in that The resolution of the lens of the bottom camera is higher than the resolution of the standard lens.

21. The rail vehicle according to claim 16, characterized in that The optical axis of the top camera is parallel to the track of the rail vehicle travel route.

22. The rail vehicle according to claim 16, characterized in that The top camera is a thermal imaging camera.

23. The rail vehicle according to claim 16, characterized in that The focal length of the lens of the top camera is greater than the focal length of the standard lens.

Images