Steel rail point inspection method and system

Abstract

The invention relates to a steel rail point inspection method and system, and the method comprises the steps: obtaining the point cloud data of the cross section of a stacked steel rail through a laser radar, obtaining the image of the cross section of the steel rail through a camera, carrying out the recognition of the steel rail image through a deep learning image processing technology for target detection, and marking each steel rail, and matching the labeling information with the point cloud data, and calculating the length of each steel rail. Compared with the prior art, the method has the advantages of realizing deartialization and automation in steel rail point inspection, improving efficiency and accuracy and the like.

Description

Technical Field

[0001] The invention belongs to the field of rail transportation, and in particular relates to a rail inspection method and system. Background Art

[0002] Rail inspection refers to the process of measuring the number and length of stacked rails. Currently, rail inspection mainly relies on manual work, but manual inspection is slow, inaccurate, and troublesome because the rails are stacked together and the length of each rail is different, so it is difficult to measure the length of each rail.

[0003] Chinese patent CN115824070B discloses a method and device for measuring the size of rail parts. The method obtains point cloud data of rail parts, loads point cloud data and displays point cloud files; pre-processes point cloud data; determines the type of point cloud data, which includes standard workpiece point cloud, rail point cloud with end face and rail point cloud without end face; processes point cloud data according to the processing and measurement logic corresponding to the type, measures and obtains three-dimensional data of rail parts, which includes rail height, rail bottom width, rail waist thickness and rail head width; performs dimensionality reduction processing on point cloud data, converts it into a 2D cross-sectional diagram for display, and stores the measured three-dimensional data of rail parts. This patent can only measure one rail, and cannot measure the stacked rails, nor can it count the number of stacked rails. Summary of the invention

[0004] The purpose of the present invention is to provide a rail inspection method and system in order to overcome the defects of the above-mentioned prior art.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides a rail inspection method, comprising the following steps:

[0007] Step S1: Align the cross-section of one side of the stacked rails, set a laser radar and a camera on the other side of the stacked rails, obtain the point cloud data of the cross-section of the stacked rails through the laser radar, and obtain the cross-section image of the stacked rails through the camera

[0008] Step S2: Processing the transverse surface image using deep learning image processing technology to obtain the number of rails in the stacked rails and the identification of each rail;

[0009] Step S3: Match the image information with the point cloud data to obtain the point cloud data within the image range of each rail;

[0010] Step S4: Take the average value of the point cloud data of each rail, and calculate the corresponding length of each rail according to the distance between the laser radar and the aligned side and the average value.

[0011] Furthermore, the laser radar is arranged on a vertical line of a transverse surface on the other side of the aligned side of the stacked rails, and a distance between the laser radar and the aligned transverse surface on one side is greater than a length of any rail in the stacked rails.

[0012] Furthermore, the camera is close to the laser radar.

[0013] Furthermore, in step S2, the lateral surface image is processed using deep learning image processing technology, wherein the deep learning image processing technology used is a deep learning image processing technology used for target detection.

[0014] Furthermore, the output of the image processing in step S2 is the two-dimensional pixel position of each rail cross section in the image, and an image pixel coordinate system including the two-dimensional pixel position of each rail cross section in the image is obtained.

[0015] Furthermore, the mapping step includes:

[0016] Step S301: Obtain the laser radar coordinate system (X L ,Y L ,Z L ), camera coordinate system (X C ,Y C ,Z C ), image pixel coordinate system (X P ,Y P );

[0017] Step S302: Adopt Zhang Zhengyou's camera calibration algorithm and use the camera calibration tool CameraCalibrator in Matlab to calculate the intrinsic parameter matrix K, which is used to realize the conversion between the camera coordinate system and the image pixel coordinate system. The conversion formula is:

[0018]

[0019] Step S303: Convert the camera coordinate system obtained after conversion into the laser radar coordinate system. The conversion formula is:

[0020]

[0021] Where R is a 3×3 rotation matrix; T is a 3×1 translation vector;

[0022] Step S304: Use the open source toolbox to automatically calibrate the converted coordinate system to achieve matching between image information and point cloud data.

[0023] Furthermore, the camera coordinate system after the conversion in step S302 is a coordinate system including two-dimensional pixel position coordinate information of each rail cross section in the image.

[0024] Furthermore, the laser radar coordinate system after the conversion in step S303 is a coordinate system including two-dimensional pixel position coordinate information of each rail cross section in the image and point cloud data information of each rail.

[0025] Furthermore, step S4 specifically includes: calculating the distance from the point cloud of each rail cross section to the laser radar, accumulating and dividing by the number of the rail point clouds to obtain the distance Ti from each rail to the laser radar, obtaining the distance P from the laser radar to the side aligned with the stacked rails, and the corresponding length of each rail Li=P-Ti.

[0026] The present invention also provides a rail inspection system, characterized in that it includes a laser radar ranging and image acquisition module, an image processing module, a matching module, and a rail inspection algorithm module, wherein the laser radar ranging and image acquisition module is used to obtain the point cloud data and the transverse surface image of the stacked rails, the image processing module is used to process the transverse surface image using deep learning image processing technology, the matching module is used to match the image information with the point cloud data, and the rail inspection algorithm module is used to use the point cloud data of the transverse surface of the stacked rails and the identification of each rail of the stacked rails to obtain the number of rails of the stacked rails and the length measurement of each rail.

[0027] Compared with the prior art, the present invention.

[0028] (1) The present invention solves the problem of relying on manual inspection of piled rails in the prior art by combining laser radar with a camera, thereby achieving de-manualization and automation in rail inspection.

[0029] (2) The present invention improves the efficiency of spot inspection of stacked rails. Compared with traditional manual spot inspection, the spot inspection of stacked rails can be completed in a short time.

[0030] (3) The present invention provides a combination of laser radar and camera, as well as deep learning image processing technology for target detection, which improves the accuracy of spot inspection of piled rails.

[0031] (4) The present invention can be flexibly configured according to actual needs and can be applied to different scales and scenarios.

[0032] (5) The present invention reduces the single-point distance measurement error caused by factors such as the laser radar scanning angle and the uneven reflectivity of the rail cross-section surface by averaging the point cloud data of each rail, thereby improving the accuracy and reliability of the overall measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the present invention;

[0034] Figure 2 This is a schematic diagram of the implementation principle of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] like Figure 1 As shown, this embodiment provides a rail inspection method, comprising the following steps:

[0037] Step S1: Align the cross-section of one side of the stacked rails, set a laser radar and a camera on the other side of the stacked rails, obtain the point cloud data of the cross-section of the stacked rails through the laser radar, and obtain the cross-section image of the stacked rails through the camera

[0038] Step S2: Processing the transverse surface image using deep learning image processing technology to obtain the number of rails in the stacked rails and the identification of each rail;

[0039] Step S3: Match the image information with the point cloud data to obtain the point cloud data within the image range of each rail; wherein matching is to map the two-dimensional pixel position of each rail on the image to the point cloud.

[0040] The matching process involves three coordinate systems: lidar coordinate system, camera coordinate system, and image pixel coordinate system.

[0041] The laser radar coordinate system is used to describe the relative position of the detected target and the laser radar, which is expressed as (X L ,Y L ,Z L ), the laser radar geometric center is the coordinate origin;

[0042] The camera coordinate system is used to describe the relative position of the detected object and the camera, expressed as (X C ,Y C ,Z C ), X C Parallel to the image coordinate x-axis, Y C The axis is parallel to the image coordinate y axis, Z C The axis is parallel to the optical axis of the camera, and the optical center of the camera is the coordinate origin;

[0043] The image pixel coordinate system is used to describe the position of the detected target in the two-dimensional image formed by the camera, which is expressed as (X P ,Y P ), with the upper left corner of the image as the coordinate origin.

[0044] (1) Camera internal parameter calibration

[0045] The Zhang Zhengyou camera calibration algorithm is used, and the camera calibration tool Camera Calibrator in Matlab is used to calculate the intrinsic parameter matrix K. It is used to realize the conversion between the camera coordinate system and the image pixel coordinate system. The formula is

[0046]

[0047] (2) Joint external parameter calibration of LiDAR and camera

[0048] The conversion from the laser radar coordinate system to the camera coordinate system is a rigid body transformation, that is, the object will not be deformed, only rotation and translation transformations are performed, and the transformation matrix is

[0049]

[0050] Where R is a 3×3 rotation matrix and T is a 3×1 translation vector.

[0051] Keep the lidar and camera still and record data for 10 to 15 seconds. Use the direct_visual_lidar_calibration open source toolbox to manually select at least 3 corresponding points in the point cloud stream data and the video stream data, and start the registration program in the toolbox to complete the automatic joint calibration.

[0052] The above is the mapping process. The conversion of pixel coordinates to camera coordinates is determined by the camera's internal parameters. The coordinate system of the camera and the coordinate system of the lidar are both three-dimensional, which can be obtained through rotation and translation. This is the camera's external parameter.

[0053] The marked area is the method of automatically obtaining the rotation and translation matrices. Recording the points here is equivalent to knowing the different expressions of a point in two coordinate systems.

[0054] Step S4: Take the average of the point cloud data of each rail, and calculate the corresponding length of each rail based on the distance between the laser radar and the aligned side and the average value. Each rail cross section is a set of point clouds, each point cloud is a three-dimensional coordinate, and the distance from each point to the device is calculated (actually the distance along the axis of the laser radar coordinate system perpendicular to the cross section), and then divided by the number of point clouds in the group after accumulation.

[0055] The reason for taking the average is to reduce the single-point distance measurement error caused by factors such as the lidar scanning angle and the uneven reflectivity of the rail cross-section surface, thereby improving the accuracy and reliability of the overall measurement.

[0056] like Figure 2 As shown, the laser radar is arranged on a vertical line of a transverse surface on the other side of one side of the stacked rails, and the distance between the laser radar and the aligned side is greater than the length of any rail in the stacked rails.

[0057] The camera is close to the laser radar.

[0058] In step S2, the transverse surface image is processed using deep learning image processing technology, wherein the deep learning image processing technology used is a deep learning image processing technology for target detection, and a large amount of transverse surface image data of rails is collected for training using open source deep learning image processing technology - target detection. The trained model can take an image or video stream as input, and output the two-dimensional pixel position of each rail cross section in the image.

[0059] Furthermore, the output of the image processing in step S2 is the two-dimensional pixel position of each rail cross section in the image, and an image pixel coordinate system including the two-dimensional pixel position of each rail cross section in the image is obtained.

[0060] Furthermore, the mapping step includes:

[0061] Step S301: Obtain the laser radar coordinate system (X L ,Y L ,Z L ), camera coordinate system (X C ,Y C ,Z C ), image pixel coordinate system (X P ,Y P );

[0062] Step S302: Adopt Zhang Zhengyou's camera calibration algorithm and use the camera calibration tool CameraCalibrator in Matlab to calculate the intrinsic parameter matrix K, which is used to realize the conversion between the camera coordinate system and the image pixel coordinate system. The conversion formula is:

[0063]

[0064] Step S303: Convert the camera coordinate system obtained after conversion into the laser radar coordinate system. The conversion formula is:

[0065]

[0066] Where R is a 3×3 rotation matrix; T is a 3×1 translation vector;

[0067] Step S304: Use the open source toolbox to automatically calibrate the converted coordinate system to achieve matching between image information and point cloud data.

[0068] Furthermore, the camera coordinate system after the conversion in step S302 is a coordinate system including two-dimensional pixel position coordinate information of each rail cross section in the image.

[0069] Furthermore, the laser radar coordinate system after the conversion in step S303 is a coordinate system including two-dimensional pixel position coordinate information of each rail cross section in the image and point cloud data information of each rail.

[0070] Furthermore, step S4 specifically includes: calculating the distance from the point cloud of each rail cross section to the laser radar, accumulating and dividing by the number of the rail point clouds to obtain the distance Ti from each rail to the laser radar, obtaining the distance P from the laser radar to the side aligned with the stacked rails, and the corresponding length of each rail Li=P-Ti.

[0071] The present embodiment also provides a rail inspection system, characterized in that it includes a laser radar ranging and image acquisition module, an image processing module, a matching module, and a rail inspection algorithm module, wherein the laser radar ranging and image acquisition module is used to obtain the point cloud data and the transverse surface image of the stacked rails, the image processing module is used to process the transverse surface image using deep learning image processing technology, the matching module is used to match the image information with the point cloud data, and the rail inspection algorithm module is used to obtain the number of rails of the stacked rails and the length measurement of each rail using the point cloud data of the transverse surface of the stacked rails and the identification of each rail of the stacked rails.

[0072] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A rail inspection method, characterized in that: The following steps are involved: Step S1: aligning the cross-section of one side of the stacked rails, setting a laser radar and a camera on the other side of the stacked rails, obtaining point cloud data of the cross-section of the stacked rails by the laser radar, and obtaining a cross-section image of the stacked rails by the camera; Step S2: Processing the transverse surface image using deep learning image processing technology to obtain the number of rails in the stacked rails and the identification of each rail; Step S3: Match the image information with the point cloud data to obtain the point cloud data within the image range of each rail; Step S4: Take the average value of the point cloud data of each rail, and calculate the corresponding length of each rail according to the distance between the laser radar and the aligned side and the average value.

2. A rail inspection method according to claim 1, characterized in that: The laser radar is arranged on a vertical line of a transverse surface on the other side of the stacked rails, and a distance between the laser radar and the transverse surface on the aligned side is greater than a length of any rail in the stacked rails.

3. A rail inspection method according to claim 1, characterized in that: The camera is close to the laser radar.

4. A rail inspection method according to claim 1, characterized in that: In step S2, the lateral surface image is processed using deep learning image processing technology, wherein the deep learning image processing technology used is a deep learning image processing technology used for target detection.

5. A rail inspection method according to claim 1, characterized in that: The output of the image processing in step S2 is the two-dimensional pixel position of each rail cross section in the image, and an image pixel coordinate system including the two-dimensional pixel position of each rail cross section in the image is obtained.

6. A rail inspection method according to claim 1 or 5, characterized in that: The mapping step comprises: Step S301: Obtain the laser radar coordinate system (X L ,Y L ,Z L ), camera coordinate system (X C ,Y C ,Z C ), image pixel coordinate system (X P ,Y P ); Step S302: Adopt Zhang Zhengyou's camera calibration algorithm and use the camera calibration tool CameraCalibrator in Matlab to calculate the intrinsic parameter matrix K, which is used to realize the conversion between the camera coordinate system and the image pixel coordinate system. The conversion formula is: Step S303: Convert the camera coordinate system obtained after conversion into the laser radar coordinate system. The conversion formula is: Where R is a 3×3 rotation matrix; T is a 3×1 translation vector; Step S304: Use the open source toolbox to automatically calibrate the converted coordinate system to achieve matching between image information and point cloud data.

7. A rail inspection method according to claim 6, characterized in that: The camera coordinate system converted in step S302 is a coordinate system containing two-dimensional pixel position coordinate information of each rail cross section in the image.

8. A rail inspection method according to claim 6, characterized in that: The laser radar coordinate system after the conversion in step S303 is a coordinate system containing two-dimensional pixel position coordinate information of each rail cross section in the image and point cloud data information of each rail.

9. A rail inspection method according to claim 1, characterized in that: The step S4 specifically includes: calculating the distance from the point cloud of each rail cross section to the laser radar, accumulating and dividing by the number of the rail point clouds to obtain the distance Ti from each rail to the laser radar, obtaining the distance P from the laser radar to the side aligned with the stacked rails, and the corresponding length of each rail Li=P-Ti.

10. A rail inspection system, characterized in that: The invention comprises a laser radar ranging and image acquisition module, an image processing module, a matching module and a rail inspection algorithm module. The laser radar ranging and image acquisition module is used to acquire the point cloud data and the transverse surface image of the stacked rails. The image processing module is used to process the transverse surface image by using deep learning image processing technology. The matching module is used to match the image information with the point cloud data. The rail inspection algorithm module is used to obtain the number of rails of the stacked rails and the length measurement of each rail by using the point cloud data of the transverse surface of the stacked rails and the identification of each rail of the stacked rails.

Images