A method for detecting lesions of a track and a tunnel

Abstract

The present application provides a kind of track and tunnel lesion detection method, the detection method specifically is: camera calibration is carried out to camera, the corresponding relationship of the point in track and tunnel and the pixel position in camera imaging image is determined;The continuous frame of track and tunnel is photographed by the several cameras installed on track detection trolley, the three-dimensional image of track and tunnel is reconstructed based on the image photographed, and lesion detection is carried out based on the three-dimensional image of track and tunnel;According to the lesion detection result, the corresponding image frame of the lesion detected is determined, and the position information of the lesion in track and tunnel is determined according to the corresponding image frame of the lesion detected.The present application does not need artificial control detection instrument, and the image photographed by the camera on track detection trolley can realize the detection of the lesion condition in track and tunnel, and the detection accuracy at different speeds is guaranteed by camera calibration, so as to guarantee the detection accuracy of the lesion in track and tunnel.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of track and tunnel detection, and particularly relates to a track and tunnel disease detection method. BACKGROUND

[0002] Currently, track and tunnel detection technologies mainly include two types, one is a traditional static measurement technology, that is, a manual track detection using a total station, a track gauge sensor and other measurement tools, and the other is a dynamic detection technology. The dynamic detection technology integrates the most advanced technologies such as mileage correction, inertial measurement, digital image measurement and wireless communication on a high-speed train, and can measure track parameters, catenary states and other related items in real time while the train is running. In the railway construction and commissioning stages, the mainstream method is still static measurement, and in the static measurement technology, a track detection trolley is most widely used. The track detection trolley mainly includes two parts, one part is a measurement trolley, and the other part is a communication trolley. The measurement trolley is provided with a body prism, an industrial computer and special measurement devices for track gauge, track level and other parameters. The communication trolley is mainly provided with a high-precision total station and a wireless communication module. The existing track detection trolley can not only measure the internal geometric state of the track such as track gauge, track alignment, level, height and twist, but also measure the external geometric state of the track such as track centerline deviation and elevation deviation.

[0003] Both the static measurement technology and the dynamic detection technology can realize the detection of the track and tunnel, but their shortcomings are also obvious. The static measurement technology has low automation degree, low detection efficiency and low detection accuracy, and has more requirements for the external environment. In addition, due to the presence of the communication trolley, the static measurement technology can only be applied to the railway operation stage to realize the detection of the track, which makes the detection process highly dependent on the catenary. The dynamic detection technology needs to realize the detection through a high-speed train, and the driving environment of the high-speed train makes the sensors on the high-speed train prone to damage, and the detection safety cannot be guaranteed. In addition, the high driving speed of the high-speed train also causes problems such as insufficient precision of the captured images, difficult calibration and easy sunlight interference. Therefore, when the static measurement technology or the dynamic detection technology is used to detect the diseases of the track and tunnel, the accuracy of the disease detection is not high. SUMMARY

[0004] The present application aims to overcome the shortcomings in the prior art, and provide a track and tunnel lesion detection method, which reconstructs a three-dimensional image of the track and tunnel by installing a camera on a track detection trolley, and realizes automatic detection of lesions in the track and tunnel based on the reconstructed three-dimensional image, and can obtain the corresponding relationship between the points in the track and tunnel and the pixel positions in the camera imaging image through camera calibration, and realize positioning and type judgment of the lesions in cooperation with the camera position, so as to solve the problems of low lesion detection efficiency and lesion detection accuracy in the static measurement technology when the track detection trolley is manually controlled, and low lesion detection safety and calibration difficulty in the dynamic detection technology when the high-speed train is used for lesion detection, so that the accuracy of track and tunnel lesion detection can be guaranteed.

[0005] The present application aims to overcome the shortcomings in the prior art, and provide a track and tunnel lesion detection method, which reconstructs a three-dimensional image of the track and tunnel by installing a camera on a track detection trolley, and realizes automatic detection of lesions in the track and tunnel based on the reconstructed three-dimensional image, and can obtain the corresponding relationship between the points in the track and tunnel and the pixel positions in the camera imaging image through camera calibration, and realize positioning and type judgment of the lesions in cooperation with the camera position, so as to solve the problems of low lesion detection efficiency and lesion detection accuracy in the static measurement technology when the track detection trolley is manually controlled, and low lesion detection safety and calibration difficulty in the dynamic detection technology when the high-speed train is used for lesion detection, so that the accuracy of track and tunnel lesion detection can be guaranteed.

[0006] A track and tunnel lesion detection method, comprising:

[0007] Camera calibration is performed on the camera, and the corresponding relationship between the points in the track and tunnel and the pixel positions in the camera imaging image is determined according to the camera calibration result;

[0008] A plurality of cameras installed on the track detection trolley are used to continuously frame the track and tunnel, the three-dimensional image of the track and tunnel is reconstructed through the continuous frame shooting image of the camera, and lesion detection is performed based on the three-dimensional image of the track and tunnel;

[0009] The corresponding image frame in which the lesion is detected is determined according to the lesion detection result, and the position information of the lesion in the track and tunnel is determined according to the corresponding image frame in which the lesion is detected;

[0010] The specific process of the camera calibration is as follows: a calibration board is selected, a calibration image is set, and a plurality of fixed regions in the calibration image are determined, the distance between the calibration board plane and the camera coordinate system origin point is tested, and the tested distance is set as the X coordinate value of all pixel points in all fixed regions in the image collected by the camera in the camera coordinate system, a first fixed region is selected, the distance between the upper edge of the first fixed region and the laser point of the measuring laser range finder is calculated, the Y coordinate value of the center of the first fixed region in the image collected by the camera in the camera coordinate system is calculated based on the calculated distance value and the width of the selected fixed region, the Y coordinate value of the center point of all fixed regions in the camera coordinate system is determined based on the Y coordinate value of the center point of the first fixed region in the camera coordinate system, and the corresponding relationship between the points in the track and tunnel and the pixel positions in the camera imaging image is determined based on the X coordinate value and the Y coordinate value of each fixed region center point in the camera coordinate system;

[0011] When determining the correspondence between the points in the track and tunnel and the pixel positions in the camera imaging image based on the X coordinate value and the Y coordinate value of each fixed region center point in the camera coordinate system, the pixel positions in the imaging images of the two linear array cameras are taken as input variables, the actual coordinate values of each fixed region center point in the camera coordinate system are taken as output variables, and the correspondence between the two is found through the input and output variable values of several fixed points.

[0012] Further, the specific process of determining the position information of the lesion in the track and tunnel according to the corresponding image frame in which the lesion is detected is as follows: determining the corresponding shooting position of the camera when each frame of image is shot, performing key frame selection on each frame of image shot by the camera based on the corresponding image frame in which the lesion is detected, and determining the position information of the lesion corresponding to each key frame image according to the corresponding camera shooting position of the key frame image and the correspondence between the points in the track and tunnel and the pixel positions in the camera imaging image.

[0013] Further, the specific process of performing key frame selection on each frame of image shot by the camera based on the corresponding image frame in which the lesion is detected is as follows: determining the number of frames from the last repositioning, selecting image frames that are more than a preset number of frames from the last repositioning, counting the number of feature points of the selected image frames, and selecting image frames with a number of feature points exceeding a preset threshold as key frames.

[0014] Further, after selecting the key frames, the lesion type of the key frame image is identified through a machine learning algorithm, and the type information of the lesion in the track and tunnel is determined according to the lesion type identification result.

[0015] Further, when identifying the lesion type of the key frame image through the machine learning algorithm, the key frame image is also enhanced through a network.

[0016] Further, the specific process of enhancing the key frame image through a network and identifying the lesion type of the key frame image through a machine learning algorithm is as follows: the network enhancement algorithm divides the key frame image into MxN rectangular lattices, each lattice processes the image information falling in its own lattice based on a machine learning algorithm, each lattice returns the processing result of its corresponding image information after completing the image information processing, and the type information of the lesion in the key frame image is determined according to the processing result of the image information returned by each lattice.

[0017] Further, the processing result of the image information includes the pixel center point position coordinates, width, depth and confidence, and the probability information of the image information belonging to the lesion object category.

[0018] Further, when the track and tunnel are continuously shot by the several cameras installed on the track detection trolley, the camera and the inertial navigation are also fused in a tight coupling form.

[0019] Further, the camera is fused with the inertial navigation by visual re-projection error and pre-integration of the inertial navigation.

[0020] The beneficial effects of the present application are:

[0021] The track and tunnel disease detection can be directly realized based on the track detection trolley, and the disease detection precision at different speeds can be ensured through camera calibration. The track and tunnel three-dimensional image reconstruction can be realized directly through the image captured by the camera on the track detection trolley without manual control of the detection instrument, the detection efficiency is obviously improved. The disease condition in the track and tunnel can be detected through the three-dimensional image, the deep disease detection of the track is also supported, and the detection precision is further ensured. The speed of the track detection trolley can be controlled, and the safety of the detection will not be affected due to the too high speed. After the disease detection is completed, the disease positioning and type identification can be directly performed based on the captured image, and the processing efficiency can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flowchart of the present application. DETAILED DESCRIPTION

[0023] The present application is further described below in combination with the drawings and examples.

[0024] Embodiment:

[0025] A track and tunnel disease detection method, as shown in Figure 1 , comprising:

[0026] Camera calibration is performed on the camera, and the corresponding relationship between the points in the track and tunnel and the pixel positions in the camera imaging image is determined according to the camera calibration result;

[0027] A plurality of cameras installed on the track detection trolley are used to continuously frame the track and tunnel, the three-dimensional image of the track and tunnel is reconstructed through the continuous frame image of the camera, and the disease detection is performed based on the three-dimensional image of the track and tunnel;

[0028] The corresponding image frame of the detected disease is determined according to the disease detection result, and the position information of the disease in the track and tunnel is determined according to the corresponding image frame of the detected disease.

[0029] The specific process of the camera calibration is: selecting a calibration board, setting a calibration image, determining a plurality of fixed regions in the calibration image, testing the distance between the calibration board plane and the origin of the camera coordinate system, and setting the distance obtained by the test as the X coordinate value of all pixel points in all fixed regions in the image collected by the camera in the camera coordinate system, selecting a first fixed region, calculating the distance between the upper edge of the first fixed region and the laser point of the laser range finder, calculating the Y coordinate value of the center of the first fixed region in the camera coordinate system based on the calculated distance value and the width of the selected fixed region, determining the Y coordinate values of the center points of all fixed regions in the camera coordinate system based on the Y coordinate value of the center of the first fixed region in the camera coordinate system, and determining the corresponding relationship between the points in the track and tunnel and the pixel positions in the camera imaging image based on the X coordinate value and the Y coordinate value of each fixed region center point in the camera coordinate system.

[0030] Specifically, the selected calibration board in the embodiment is an L-shaped calibration board, and the calibration image is set as a black-and-white pattern, and each boundary of black and white is parallel to each other. The width of each black stripe is 20 mm, the width of each white stripe is 50 mm, and the center line distance between two adjacent black stripes is 70 mm. The fixed region used in the camera calibration is the black stripe.

[0031] After determining the fixed regions in the calibration image, the camera calibration is realized based on the positions of the fixed regions on the calibration board. The purpose of the camera calibration is to take the pixel positions of the fixed points on the calibration board, i.e., the center points of the fixed regions, in the imaging images of the two linear array cameras as input variables, take the actual coordinate values of the fixed points in the camera coordinate system as output variables, and find the corresponding relationship between the input and output variable values of a plurality of fixed points.

[0032] Before calculating the distance value between the upper edge of the selected fixed region and the laser point of the laser range finder, the positions of the camera and the calibration stand need to be set. After the setting is completed, the focusing ring of the camera is adjusted, and the calibration image is photographed by the camera. After the calibration image is acquired by photographing, the distance from the calibration board plane to the origin of the camera coordinate system needs to be measured. This distance is the X coordinate value of the point in the photographed calibration image in the camera coordinate system. The distance detection is specifically realized by a laser range finder which is arranged above the calibration board. After the X coordinate value is determined, the distance value from the laser point of the laser range finder to the upper edge of the first black stripe, i.e., the first fixed region, is measured. Since the widths of the black stripes and the white stripes are known, the Y coordinate value of the center of each black stripe in the camera coordinate system can be calculated based on the widths of the black stripes and the white stripes.

[0033] After determining the X coordinate value and the Y coordinate value of each fixed region center point in the camera coordinate system, the identification of the coefficients can be determined by using a multivariate nonlinear regression method, so as to determine the correspondence between the points in the track and the tunnel and the pixel positions in the camera imaging image.

[0034] The specific process of determining the position information of the lesion in the track and the tunnel according to the corresponding image frame in which the lesion is detected is as follows:

[0035] determining the corresponding shooting position of the camera when each frame of image is shot;

[0036] performing key frame selection on each frame of image shot by the camera based on the corresponding image frame in which the lesion is detected, and determining the position information of the lesion corresponding to each key frame image according to the corresponding camera shooting position of the key frame image and the correspondence between the points in the track and the tunnel and the pixel positions in the camera imaging image;

[0037] The specific process of determining the corresponding shooting position of the camera when each frame of image is shot is as follows:

[0038] obtaining a group of points P in the track and the tunnel, the projection points of the P points in a frame and an adjacent frame being points P1={p j}(j=1, 2,..., n) and points P2={p j ′}(j=1, 2,..., n) respectively, and setting an Euclidean transformation O, under the influence of the Euclidean transformation O, p j =Op j ′ . Defining the error term corresponding to the jth point at this time as e j =p j -Op j ′ , constructing a least square problem based on the error function, the points P1 and the points P2, that is, obtaining the Euclidean transformation O by minimizing the sum of squares of the error terms, and specifically, the formula corresponding to the least square problem is as follows: When solving the least square problem, the centroids of the two groups of points need to be defined first, then the error function is processed to obtain defining q j =p j -p, q j ′ =p j ′ -p ′ , in order to decompose the error term, defining and performing SVD decomposition on T, so as to obtain the specific formula of the Euclidean transformation O.

[0039] In addition, considering that the least square problem can provide a false reference value when calculating the error term, and the square of the two-norm can cause the false value to grow rapidly, thereby causing the algorithm to not converge, resulting in errors in subsequent positioning and classification, in order to improve the robustness of the algorithm, a constraint function Y(e) is added, and the expression of the constraint function Y(e) is:

[0040]

[0041] Wherein, e is the reference value of the error term, and ε is a preset value.

[0042] Based on the calculated Euclidean transformation formula, the corresponding shooting position of the camera when each frame of image is shot can be obtained.

[0043] Solving the least square problem, the Euclidean transformation formula is obtained according to the solving result of the least square problem.

[0044] In the embodiment, 6 cameras are used to detect the tunnel arc surface, and the height of each camera from the detection surface is set to 2.15 meters, and the 6 cameras are installed on the same arc surface, so as to ensure the clarity of the image shot by the linear array camera.

[0045] In addition to the camera, a 3D scanner is also arranged on the track detection trolley, and the 3D scanner can realize three-dimensional reconstruction of the track and the tunnel in cooperation with the image shot by the camera. Since the scanning accuracy of the 3D scanner decreases with the increase of the installation height, the installation height of the 3D scanner is selected to be 0.23m away from the track bottom in the embodiment.

[0046] The speed of the track detection trolley is controllable, and can be set according to the detection requirements and the specific conditions of the track and the tunnel.

[0047] The specific process of key frame selection for each frame of image shot by the camera based on the corresponding image frame of the detected lesion is: determining the number of frames from the last repositioning, selecting the image frame which is more than a preset number of frames from the last repositioning, counting the number of feature points of the selected image frame, and selecting the image frame whose number of feature points is more than a preset threshold as a key frame.

[0048] The key frame selection is performed on the image frame with lesions, and the image frame without lesions is directly identified as a normal frame. When the lesion type recognition is performed by the machine learning algorithm in the subsequent process, the calculation workload can be reduced, thereby improving the recognition accuracy and efficiency.

[0049] After selecting the key frame, the machine learning algorithm is also used to recognize the lesion type of the key frame image, and the type information of the lesion in the track and the tunnel is determined according to the lesion type recognition result.

[0050] When the key frame image is subjected to lesion type recognition by the machine learning algorithm, the key frame image is subjected to network enhancement.

[0051] The specific process of the network enhancement of the key frame image and the lesion type recognition of the key frame image by the machine learning algorithm is that the network enhancement algorithm divides the key frame image into MxN rectangular lattices, each lattice processes the image information falling in the lattice based on the machine learning algorithm, each lattice returns the processing result of the corresponding image information after completing the image information processing, and the category information of the lesion in the key frame image is determined according to the processing result of the image information returned by each lattice.

[0052] The processing result of the image information includes the pixel center point position coordinates, width, depth and confidence, and the probability information that the image information belongs to the lesion object category.

[0053] When the track and the tunnel are subjected to continuous frame shooting by the cameras installed on the track detection trolley, the cameras and the inertial navigation system are fused in the form of tight coupling.

[0054] Since the positioning of the lesion by the camera is not necessarily very accurate, sometimes it will lead to inaccurate positioning and low precision, in order to solve the above problems, the camera and the inertial navigation system are fused, the inertial navigation system is an inertial navigation system, which can assist the camera to better realize positioning when shooting images.

[0055] The camera and the inertial navigation system are fused by the visual re-projection error and the pre-integration of the inertial navigation system.

[0056] The above-mentioned embodiments are only a preferred scheme of the present application, and do not limit the present application in any form, and other variants and modifications can be made without exceeding the technical scheme recorded in the claims.

Claims

1. A method of detecting lesions in a track and tunnel, characterized in that, The application comprises the following steps: camera calibration is performed on the camera, and the correspondence between the points in the track and tunnel and the pixel positions in the camera imaging image is determined according to the camera calibration result; a plurality of cameras installed on the track detection trolley are used to continuously frame the track and tunnel, the three-dimensional image of the track and tunnel is reconstructed through the continuous frame imaging of the cameras, and the lesion detection is performed based on the three-dimensional image of the track and tunnel; the corresponding image frame in which the lesion is detected is determined according to the lesion detection result, and the position information of the lesion in the track and tunnel is determined according to the corresponding image frame in which the lesion is detected. The specific process of the camera calibration is as follows: a calibration board is selected, a calibration image is set, and a plurality of fixed regions in the calibration image are determined, the distance between the plane of the calibration board and the origin of the camera coordinate system is tested, and the distance obtained by the test is set as the X coordinate value of all pixel points in all fixed regions in the image collected by the camera, a first fixed region is selected, the distance between the upper edge of the first fixed region and the laser point of the laser range finder is calculated, the Y coordinate value of the center of the first fixed region in the image collected by the camera in the camera coordinate system is calculated based on the calculated distance value and the width of the selected fixed region, the Y coordinate value of the center of all fixed regions in the camera coordinate system is determined based on the Y coordinate value of the center of the first fixed region in the camera coordinate system, and the correspondence between the points in the track and tunnel and the pixel positions in the camera imaging image is determined based on the X coordinate value and the Y coordinate value of the center of each fixed region in the camera coordinate system. When the correspondence between the points in the track and tunnel and the pixel positions in the camera imaging image is determined based on the X coordinate value and the Y coordinate value of the center of each fixed region in the camera coordinate system, the pixel positions in the imaging images of the two linear array cameras are taken as input variables, the actual coordinate values of the center of each fixed region in the camera coordinate system are taken as output variables, and the correspondence between the two is found through the input and output variable values of a plurality of fixed points. The selected calibration board is an L-shaped calibration board, and the calibration image is set as a black-and-white pattern, and each black and white boundary is parallel to each other. The fixed region used in the camera calibration is a black stripe.

2. The method of claim 1, wherein the method further comprises: The specific process of determining the position information of the lesion in the track and tunnel according to the corresponding image frame in which the lesion is detected is as follows: the corresponding shooting position of the camera when each frame of image is shot is determined, key frames are selected from each frame of image shot by the camera based on the corresponding image frame in which the lesion is detected, and the position information of the lesion corresponding to each key frame image is determined according to the corresponding camera shooting position of the key frame image and the correspondence between the points in the track and tunnel and the pixel positions in the camera imaging image.

3. The method of claim 2, wherein the method further comprises: The specific process of selecting key frames from each frame of image shot by the camera based on the corresponding image frame in which the lesion is detected is as follows: the frame number from the last repositioning is determined, image frames with a frame number exceeding a preset frame number from the last repositioning are selected, the number of feature points in the selected image frames is counted, and image frames with a number of feature points exceeding a preset threshold are selected as key frames.

4. The method of claim 2, wherein the method further comprises: After the key frame is selected, a machine learning algorithm is used to identify the lesion type of the key frame image, and the type information of the lesion in the track and tunnel is determined according to the lesion type identification result.

5. The method of claim 4, wherein the method further comprises: When the machine learning algorithm is used to identify the lesion type of the key frame image, the key frame image is also subjected to network enhancement.

6. The method of claim 5, wherein the method further comprises: The specific process of the network enhancement of the key frame image and the lesion type identification of the key frame image by the machine learning algorithm is that the network enhancement algorithm divides the key frame image into MxN rectangular grids, each grid processes the image information falling in the grid based on the machine learning algorithm, and each grid returns the processing result of the corresponding image information after completing the image information processing, and the type information of the lesion in the key frame image is determined according to the processing result of the image information returned by each grid.

7. The method of claim 6, wherein the method further comprises: The processing result of the image information includes the pixel center position coordinates, width, depth and confidence, and the probability information of the image information belonging to the lesion object category.

8. The method of claim 1, wherein the method further comprises: When the cameras installed on the track detection trolley continuously frame the track and tunnel, the cameras and the inertial navigation are also fused in a tight coupling form.

9. The method of claim 8, wherein the method further comprises: The cameras and the inertial navigation are fused by the visual reprojection error and the pre-integration of the inertial navigation.

Images