Rail surface light band detection method and equipment

Abstract

The present invention provides a method and device for detecting light bands on a rail surface, comprising a light band area identification step, a light band two-dimensional information acquisition step, a rail point cloud generation step, a rail center point extraction step, and a light band offset acquisition step, wherein the light band area in a two-dimensional image is identified and segmented by a trained semantic segmentation model, and the identification is comprehensive, fast, and highly accurate; partial contour point position data corresponding to the two-dimensional image is extracted to generate a rail point cloud, and a small range of point clouds is further extracted based on the light band center point for further calculation and analysis. Therefore, only a relatively small amount of data collected by a line laser sensor needs to be analyzed and processed, with a small amount of calculation, and the detected rail surface light band and its offset can be quickly output, which is beneficial to on-site detection and analysis; based on the extracted light band effective point cloud, the coordinates of the rail center point are obtained by fitting a plane, and the fitting plane can effectively eliminate the influence of some other types of rail surface damage, and obtain an accurate light band offset.

Description

Technical Field

[0001] The present invention belongs to the technical field of track defect detection, and in particular relates to a method and device for detecting light bands on a rail surface. Background Art

[0002] After a period of operation, the rolling and sliding of the wheel tread on the rail surface, as well as the interaction between the wheel flange and the rail's active edge, will leave bright marks on the rail, namely the top surface light band and the active edge scratches. If the force and point of action between the wheel and rail change due to track unevenness, the rail light band will produce corresponding abnormalities, including changes in light band width, uneven width, and light band position deviation. Therefore, it is necessary to detect and further analyze the light band on the rail surface to promptly detect the presence of various types of track unevenness and ensure the safe and smooth operation of trains.

[0003] Currently, in practical applications, the size and offset of the top surface light band are mainly measured manually using corresponding detection tools, which is inefficient. Although there are some rail surface light band detection methods in the existing technology, these methods still have some problems and are difficult to promote and apply in practice: some methods use trained neural network models to identify the rail surface light band in track photos. Such methods can only identify the rail surface light band from the photos, but it is difficult to obtain the accurate size and position information of the light band; other methods obtain the size and position information of the light band by projecting the collected image data onto a preset standard track model, such as the method disclosed in CN107264570A. However, the rail may not only have light band anomalies, but also other defects such as wear, which will affect the accuracy of the projection matching, making it difficult to obtain accurate light band size and position information, which in turn affects subsequent further analysis.

[0004] Therefore, in order to improve the automation and efficiency of rail surface light band detection and obtain accurate light band size and position information, a new rail surface light band detection method is needed. Summary of the Invention

[0005] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a rail surface light band detection method and equipment that can accurately and comprehensively identify light band anomalies on the rail surface and obtain their size and position information. The present invention adopts the following technical solutions:

[0006] The present invention provides a method for detecting light bands on rail surfaces, characterized in that the rail surface light bands on the rail surface are detected based on a two-dimensional image of the rail surface captured by a camera, contour point data of the rail captured by a line laser sensor, and corresponding mileage information. The method comprises: a light band area segmentation step: inputting the two-dimensional image into a rail surface light band segmentation model, which outputs the shape and position information of the segmented rail surface light band, wherein the rail surface light band segmentation model is a trained semantic segmentation model; a light band two-dimensional information acquisition step: calculating the shape and position information of the rail surface light band by sampling based on the shape and position information of the rail surface light band Two-dimensional information, including the coordinates of the center point of the light band; a rail point cloud generating step: based on the mileage information, extracting multiple frames of the contour point data corresponding to the two-dimensional image from the collected contour point data, and generating a rail point cloud of the rail; a rail center point extracting step: with the center point of the light band as the center, extracting a point cloud in a predetermined range from the rail point cloud as a light band effective point cloud, and based on the light band effective point cloud, obtaining the coordinates of the rail center point of the rail by fitting a plane; a light band offset acquiring step: calculating the offset of the track surface light band based on the center point of the light band and the center point of the rail.

[0007] The rail surface light band detection method provided by the present invention may also have the following technical features, wherein the light band area segmentation step includes: a binary image generation sub-step, inputting the two-dimensional image into the rail surface light band segmentation model, and the model outputting a binary image of the rail surface light band; a binary image conversion sub-step, converting the pixel points of the light band area in the binary image into a polygonal surface composed of inflection points; a connected domain filling sub-step, filling the holes in the connected domain of the polygonal surface according to a predetermined hole filling algorithm to obtain a filled target polygonal surface; a false detection filtering sub-step, judging whether the pixel area of ​​the target polygonal surface is smaller than a predetermined area, and filtering out the target polygonal surface smaller than the predetermined area as a false detection.

[0008] The rail surface light band detection method provided by the present invention may also have such technical features, wherein the light band two-dimensional information acquisition step includes: a light band area sampling sub-step: in the middle of the length direction of the light band area in the binary image, a predetermined number of samplings are performed along the Y axis at a predetermined sampling interval to obtain multiple light band area samples; a light band center point calculation sub-step: calculating the difference in the X-axis coordinates of the two endpoints of each of the light band area samples, and calculating the average value of the multiple differences as the X-axis coordinate of the light band center point, and taking the Y-axis coordinate of the midpoint of the length direction of the light band area as the Y-axis coordinate of the light band center point.

[0009] The rail surface light band detection method provided by the present invention may also have such technical features, and before the rail center point extraction step, it also includes: a light band center point extraction step: based on the camera intrinsic parameters of the camera, the calibration relationship between the camera and the line laser sensor, the coordinates of the light band center point are converted to the coordinate system of the rail point cloud, wherein, in the light band offset acquisition step, based on the converted light band center point coordinates and the rail center point coordinates, the offset of the rail surface light band relative to the central axis of the rail is calculated.

[0010] The rail surface light band detection method provided by the present invention may also have such technical features, wherein the rail center extraction step includes: a light band effective point cloud extraction sub-step: based on the converted coordinates of the light band center point and the mileage information, with the light band center point as the center, a point cloud within a predetermined mileage range is cut out from the rail point cloud as the light band effective point cloud; a rail surface fitting sub-step: based on the light band effective point cloud, the rail surface plane is fitted by the RANSAC algorithm; a rail surface highest point calculation sub-step: the coordinates of the center point of the rail surface plane are calculated as the coordinates of the highest point of the rail surface; a non-active edge fitting sub-step: based on the light band effective point cloud, the non-active edge plane is fitted by the RANSAC algorithm; a lower jaw point calculation sub-step: based on the coordinates of the highest point of the rail surface and the non-active edge plane, the coordinates of the lower jaw point are calculated; a rail center point calculation sub-step: based on the coordinates of the lower jaw point, the coordinates of the rail center point are calculated.

[0011] The rail surface light band detection method provided by the present invention may also have such a technical feature, wherein the RANSAC algorithm includes the following steps: Step 1: select a partial point cloud of the corresponding part in the effective point cloud of the light band; Step 2: randomly sample at least three points in the partial point cloud as initial points; Step 3: set the initial equation of the plane model AX+BY+CZ+D=0, and fit the plane model based on the multiple initial points to obtain the values ​​of the plane model parameters A, B, C, and D; Step 4: select a point other than the initial point from all points of the selected partial point cloud, and calculate the distance d1 of the point to the plane model:

[0012]

[0013] Wherein, x0, y0, z0 are the coordinates of the selected point; Step 5: Determine whether the distance d1 is less than a preset distance threshold, and if so, add the point to the inner group; Step 6: Calculate the ratio of the number of points in the inner group to the total number of points in the effective point cloud of the light band, and determine whether the ratio is greater than a preset ratio threshold; Step 7: If the judgment in step 6 is yes, update the plane model parameters based on the points in the inner group to obtain updated parameters a, b, c, d; Step 8: Based on the updated parameters, calculate the Z-axis coordinate of the center point of the partial point cloud as the Z-axis coordinate of the highest point on the rail surface:

[0014] Z max =-(ax+by+d) / c.

[0015] The rail surface light band detection method provided by the present invention may also have the following technical features: wherein, in the jaw point calculation sub-step, the X-axis coordinate of the jaw point is:

[0016] z jaw =z max -34

[0017] x jaw =-(by+cz jaw +d) / a

[0018] Where x jaw 、z jaw are the X-axis coordinate and the Z-axis coordinate of the mandible point respectively. In the rail center point calculation sub-step, the coordinates of the rail center points of the rails on both sides are respectively:

[0019] x rail =x jaw -36.5

[0020] x rail ′=x jaw +36.5

[0021] In the light band offset acquisition step, the offsets of the rails on both sides are respectively:

[0022] offset=x lightband -x rail

[0023] offset′=x rail -x lightband

[0024] Where x lightband is the X-axis coordinate of the center point of the light band after conversion.

[0025] The rail surface light band detection method provided by the present invention may also have such technical features, wherein, in the light band two-dimensional information acquisition step, after the light band area sampling sub-step, it also includes: an average width calculation sub-step, calculating the width of each light band area sample, and calculating the average width of multiple widths, and after the light band two-dimensional information acquisition step, it also includes: a light band width acquisition step, based on the camera intrinsic parameters of the camera, the calibration relationship between the camera and the line laser sensor, converting the average width to the coordinate system of the rail point cloud, thereby obtaining the physical width of the rail surface light band.

[0026] The rail surface light band detection method provided by the present invention may also have such a technical feature, after the light band width acquisition step and the light band offset acquisition step, it also includes: a light band abnormality type and level determination step: based on the physical width of the rail surface light band, the offset and the predetermined rail surface light band abnormality threshold information, the abnormality type and level of the rail surface light band are determined.

[0027] The present invention provides a rail surface light band detection device, which is characterized in that the light band area on the rail surface is detected based on the two-dimensional image of the rail surface collected by a camera, the contour point data of the rail collected by a line laser sensor, and the corresponding mileage information. The device includes: a model storage unit for storing a rail surface light band segmentation model, which is a trained semantic segmentation model; a light band area recognition unit for inputting the two-dimensional image into the rail surface light band segmentation model to obtain the shape and position information of the segmented light band area; a light band two-dimensional information acquisition unit for calculating the rail surface light band area by sampling based on the shape and position information of the rail surface light band. The two-dimensional information of the belt, including the coordinates of the center point of the light belt; a rail point cloud generating unit, which is used to extract multiple frames of the contour point data corresponding to the two-dimensional image from the collected contour point data based on the mileage information, and generate the rail point cloud of the rail; a rail center point extracting unit: which is used to extract a point cloud in a predetermined range from the rail point cloud with the center point of the light belt as the effective point cloud of the light belt, and obtain the coordinates of the rail center point of the rail by fitting a plane based on the effective point cloud of the light belt; and a light belt offset acquiring unit, which is used to calculate the offset of the track surface light belt based on the center point of the light belt and the center point of the rail.

[0028] Functions and effects of the invention

[0029] According to the present invention, the method and apparatus for detecting light bands on a rail surface include a light band region identification step, a light band two-dimensional information acquisition step, a rail point cloud generation step, a rail center point extraction step, and a light band offset acquisition step. A trained semantic segmentation model is used to identify and segment the light band region in a two-dimensional image, enabling comprehensive, rapid, and highly accurate identification of the rail surface light band. Partial contour point data corresponding to the two-dimensional image is extracted to generate a rail point cloud, and a small point cloud in the middle of the light band is further extracted based on the light band center point as a valid light band point cloud for further calculation and analysis. Therefore, only a relatively small amount of data collected by a line laser sensor needs to be analyzed and processed, resulting in a low computational load and the ability to quickly output the detected rail surface light band and its offset, facilitating on-site detection and analysis. On-site rail surfaces may not only have light bands but may also have other types of damage. In the method of the present invention, the coordinates of the rail center point are obtained by fitting a plane based on the extracted effective light band point cloud, thereby calculating the light band offset. The plane fitting effectively eliminates the influence of other types of rail surface damage on the light band offset calculation, thereby obtaining an accurate light band offset. As described above, the method and device of the present invention can accurately and comprehensively identify the light band on the rail surface and calculate its accurate offset with a small amount of calculation, which is very beneficial for on-site track detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of a method for detecting light bands on rail surfaces according to an embodiment of the present invention;

[0031] Figure 2 is a perspective view of a track inspection vehicle according to an embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of the distribution of sensors in a vehicle body unit according to an embodiment of the present invention;

[0033] Figure 4 is a perspective view of a load-bearing wheel assembly according to an embodiment of the present invention;

[0034] Figure 5 is a cross-sectional view of a load-bearing wheel assembly according to an embodiment of the present invention;

[0035] Figure 6 is a side view of a compression assembly according to an embodiment of the present invention;

[0036] Figure 7 is a three-dimensional diagram of a compression assembly according to an embodiment of the present invention;

[0037] Figure 8 is a flow chart of the rail data collection steps in an embodiment of the present invention;

[0038] Figure 9 Flowchart of the light band area segmentation steps in an embodiment of the present invention

[0039] Figure 10 is a binary image of the rail surface light band in an embodiment of the present invention;

[0040] Figure 11 is a schematic diagram of marking a light band area in an original two-dimensional image based on a binary image in an embodiment of the present invention;

[0041] Figure 12 is a flow chart of steps for obtaining two-dimensional information of a light band in an embodiment of the present invention;

[0042] Figure 13 is a flow chart of the steps for generating a track point cloud in an embodiment of the present invention;

[0043] Figure 14 This is a schematic diagram of a frame of point data in an embodiment of the present invention;

[0044] Figure 15 is a schematic diagram of a rail point cloud according to an embodiment of the present invention;

[0045] Figure 16 Flowchart of the rail center point extraction step in an embodiment of the present invention;

[0046] Figure 17 Schematic diagram of a cropped effective point cloud of a light band in an embodiment of the present invention;

[0047] Figure 18 is a flow chart of fitting a plane using the RANSCAC algorithm in an embodiment of the present invention;

[0048] Figure 19 is a block diagram of a rail surface light band detection device according to an embodiment of the present invention;

[0049] Figure 20 It is a cross-sectional view of the mainline rail in an embodiment of the present invention.

[0050] Reference numerals:

[0051] Track inspection vehicle 100; vehicle body 20; vehicle body unit 21; housing 211; oblique end portion 2111a; first line laser sensor 217a; third line laser sensor 217c; carrying mechanism 30; load-bearing wheel assembly 31; load-bearing wheel bracket 311; load-bearing wheel shaft 312; load-bearing wheel 313; encoder accommodating groove 3133; bearing 314; brake component 315; clamping assembly 32; clamping bracket 321; mounting block 3211; guide rod 3212; guide component 322; guide rail 3221; slider 3222; elastic member 323; wheel body bracket 324; clamping wheel 326; locking component 327; locking fitting 3271; locking component 3272; wrench portion Component 328; pushing mechanism 40; rail surface light band detection equipment 60; model storage unit 61; detection data acquisition unit 602; image preprocessing unit 603; light band area segmentation unit 604; light band two-dimensional information calculation unit 605; rail point cloud generation unit 606; light band width acquisition unit 607; light band center point extraction unit 608; rail center point extraction unit 609; light band offset acquisition unit 610; light band abnormality type and level determination unit 611; light band information storage unit 612; light band detection communication unit 613; light band detection control unit 614; mainline rail 9; rail head 91; rail surface 911; active edge 912; inactive edge 913; rail waist 92; rail bottom 93. DETAILED DESCRIPTION

[0052] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following is a detailed description of the rail surface light band detection method and equipment of the present invention in combination with embodiments and drawings.

[0053] <Example>

[0054] This embodiment provides a method for detecting light bands on a rail surface, which is used to detect the size and position of light bands on the rail surface. The rails may be mainline rails, basic rails at switches, or wing rails. In this embodiment, a mainline rail is used as an example for specific description.

[0055] Figure 19 It is a cross-sectional view of the mainline rail in this embodiment.

[0056] like Figure 19 As shown, the mainline rail 9 includes a rail head 91, a rail waist 92, and a rail bottom 93. The upper end surface of the rail head 91 is the rail surface 911, the side of the rail head 91 facing between the two rails is the active side 912, and the other side is the inactive side 913. When a train travels along the two mainline rails 9, its wheels contact the rail surface 911 (also called the tread) and the active side 912, resulting in a rail surface light band on the rail surface 911 and scratches on the active side 912.

[0057] Based on the parameters of the train and the track, the rail surface light band should be formed with approximately equal width on the side of the rail surface 911 close to the active edge 912, and its width and offset relative to the central axis of the rail should be within a predetermined numerical range.

[0058] Figure 1 4 is a flow chart of the rail surface light band detection method in this embodiment.

[0059] like Figure 1 As shown in FIG, the rail surface light band detection method specifically includes the following steps:

[0060] Rail data collection step S1: The rails are inspected by a track inspection vehicle, a two-dimensional image of the rails is obtained by the onboard camera, point data of the rail contour is obtained by a line laser sensor, and mileage information is obtained by a mileage encoder.

[0061] Image preprocessing step S2: preprocessing the acquired two-dimensional image.

[0062] Light band area recognition step S3: The pre-processed two-dimensional image is input into the track surface light band recognition model, and the model outputs the light band position information.

[0063] Rail point cloud generation step S4: Based on the mileage information, multiple frames of point data corresponding to the two-dimensional image are extracted from the collected point data to generate a rail point cloud of the rail.

[0064] Light band two-dimensional information acquisition step S5: Based on the shape and position information of the track surface light band, the two-dimensional information of the track surface light band is calculated by sampling, including the average width of the light band and the coordinates of the center point of the light band.

[0065] Light band width acquisition step S6: Based on the camera intrinsic parameters and the calibration relationship between the camera and the line laser sensor, the average width of the rail surface light band is converted to the coordinate system of the rail point cloud, and the physical width of the rail surface light band is obtained based on the converted average width.

[0066] Light band center point extraction step S7: Based on the camera intrinsic parameters and the calibration relationship between the camera and the line laser sensor, the coordinates of the light band center point of the rail surface light band are converted to the coordinate system of the rail point cloud.

[0067] Rail center point extraction step S8: With the light band center point as the center, extract a point cloud within a predetermined range from the rail point cloud as the light band effective point cloud, and obtain the rail center point coordinates of the rail by fitting a plane.

[0068] Light band offset acquisition step S9: Based on the converted light band center point coordinates and the rail member center point coordinates, the offset of the rail surface light band relative to the rail center axis is calculated.

[0069] The light band abnormality determination step S10 is to determine the abnormality type and level of the track surface light band based on the width and offset of the track surface light band and predetermined abnormality threshold information of the track surface light band.

[0070] The above steps will be described in detail below.

[0071] Rail data collection step S1: The rails are inspected by a track inspection vehicle, a two-dimensional image of the rails is obtained by the onboard camera, point data of the rail contour is obtained by a line laser sensor, and mileage information is obtained by a mileage encoder.

[0072] As an example, a track inspection vehicle equipped with a camera, a line laser sensor, and an odometer encoder for collecting rail data on site is shown below.

[0073] Figure 2 It is a three-dimensional diagram of the track inspection vehicle in this embodiment.

[0074] like Figure 2 As shown, the track inspection vehicle 100 includes a vehicle body 20, a supporting mechanism 30 and a pushing mechanism 40. The vehicle body 20 is movably placed on two main line rails 9 through the supporting mechanism 30 at the bottom. The pushing mechanism 40 is installed on the vehicle body 20 for inspection personnel to push the vehicle body 20.

[0075] The vehicle body 20 is a two-part structure, comprising two mirror-symmetrical vehicle body units 21. Each vehicle body unit 21 comprises a housing 211 and sensors, electronic control components, etc. disposed within the housing 211.

[0076] Figure 3 Schematic diagram of the distribution of sensors in the vehicle body unit in this embodiment.

[0077] like Figure 3 As shown, a camera 215, a refractor 216, a first line laser sensor 217a, a second line laser sensor 217b, a third line laser sensor 217c, and an illumination unit (not shown) are provided within the housing 211 of each vehicle body unit 21. The housing 211 has an opening below, through which the camera and the line laser sensor can detect the mainline rail 9 below.

[0078] Camera 215 is mounted horizontally at the top of housing 211 using a corresponding bracket, with its lens facing horizontally. Refractor 216 is a 45° refractor, also mounted at the top of housing 211 using a corresponding bracket, in front of the lens of camera 215. Refraction by refractor 216 allows camera 215 to capture the rails beneath it. This provides a more stable installation for camera 215 and avoids obstruction of the line laser sensor.

[0079] The lighting unit includes multiple light tubes, which are also installed on the top of the shell 211 through corresponding brackets to provide sufficient lighting for the camera 215 to shoot. The shielding of the shell 211 can reduce the impact of changes in external environmental lighting conditions on the camera 215 shooting.

[0080] The first line laser sensor 217a is mounted within the housing 211 via a corresponding bracket and is located within the oblique end 2111a of the housing 211. The first line laser sensor 217a is mounted horizontally. When the track inspection vehicle 100 is placed on the rail 9, the first line laser sensor 217a is located outside the mainline rail 9, slightly above the mainline rail 9, and facing the mainline rail 9. The projected line laser covers the inactive side of the mainline rail 9, the lower jaw portion of the inactive side, and a portion of the rail surface, enabling the acquisition of corresponding point position data.

[0081] The second line laser sensor 217b is obliquely mounted on the top of the housing 211 via a corresponding bracket. When the track inspection vehicle 100 is placed on the mainline rail 9, the second line laser sensor 217b is located above and outside the mainline rail and faces the mainline rail 9. The projected line laser covers the inactive side of the mainline rail 9, the waist of the inactive side, and the rail surface, and can obtain corresponding point position data.

[0082] The third line laser sensor 217c is mounted at an angle on the top of the housing 211 via a corresponding bracket, closer to the center of the vehicle body 20 than the second line laser sensor 217b. When the track inspection vehicle 100 is placed on the mainline rail 9, the third line laser sensor 217c is located above and inside the mainline rail 9 and faces the mainline rail 9. The projected line laser covers the active edge of the mainline rail 9, the waist of the active edge, and the rail surface, and can obtain corresponding point position data.

[0083] In this embodiment, the models of the several line laser sensors on the track inspection vehicle 100 are all the same.

[0084] like Figure 2 As shown, the supporting mechanism 30 includes four supporting wheel assemblies 31 and four clamping assemblies 32. The four supporting wheel assemblies 31 are mounted on the lower portions of the vehicle body 20 in the width direction, and are arranged in pairs near one end of the vehicle body 20 in the length direction, corresponding to the two rails 9. The four clamping assemblies 32 are also mounted on the lower portions of the vehicle body 20 in the width direction, and are located next to the four supporting wheel assemblies 31. The track inspection device 100 is movably mounted on the two rails 9 via the four supporting wheel assemblies 31. At this time, the four clamping assemblies 32 are located on the inner side of the rails 9 on the corresponding side.

[0085] Figure 4 3D is a perspective view of the load-bearing wheel assembly in this embodiment.

[0086] Figure 5 2 is a cross-sectional view of the load-bearing wheel assembly in this embodiment.

[0087] like Figures 4 and 5 As shown, the load-bearing wheel assembly 31 includes a load-bearing wheel bracket 311 , a load-bearing wheel shaft 312 , a load-bearing wheel 313 , a bearing 314 and a brake component 315 .

[0088] The ends of the load-bearing wheel shaft 312 are mounted on the load-bearing wheel bracket 311 via two bearings 314. The load-bearing wheel 313 is mounted on the load-bearing wheel shaft 312 and is restrained by the two bearings 314. In this embodiment, the load-bearing wheel 313 is made of an insulating material, preferably ceramic or POM plastic, and has an insulation resistance of no less than 1 MΩ. This prevents the detection device 100 from connecting the two rails and energizing them, thereby preventing interference with the track's electrical system.

[0089] The load-bearing wheels 313 have encoder accommodating slots 3133 within them. Two of the load-bearing wheels 313 have distance detection encoders 317 installed within the encoder accommodating slots 3133. These encoders are used to obtain mileage information as the load-bearing wheels 313 roll along the rails 9. The encoder accommodating slots 3133 are slightly larger than the outer dimensions of the distance detection encoders 317. Therefore, the distance detection encoders 317 and the load-bearing wheels 313 do not directly contact each other, preventing friction from affecting the rotation of the load-bearing wheels 313.

[0090] It should be noted that the structure of the vehicle body 20 and the installation position of the line laser sensors enable multiple line laser sensors to scan and obtain profile data for the same cross-section of the mainline rail 9. Under the control of the corresponding controller, during the movement of the cart, the multiple line laser sensors perform a synchronous scan after a predetermined time or distance. Each frame of point data (point cloud) collected by the line laser sensors contains hundreds to thousands of coordinate points, each of which contains mileage information (Y-axis), height information (Z-axis), width information (X-axis), and brightness information.

[0091] Similarly, under the control of the corresponding controller, during the movement of the cart, each time it travels a predetermined distance or moves a predetermined distance, the camera captures a frame of 2D image. In this embodiment, a 2D image is captured every 500 mm, and a frame of contour point data is captured every 2 mm.

[0092] In addition, before starting data acquisition, the camera and all line laser sensors were calibrated and the corresponding calibration parameters were obtained.

[0093] Figure 6 2 is a side view of the clamping assembly in this embodiment.

[0094] Figure 7It is a three-dimensional diagram of the clamping assembly in this embodiment.

[0095] like Figures 6 and 7 As shown, the clamping assembly 32 includes a clamping bracket 321, a guide component 322, an elastic member 323, a wheel body bracket 324, a clamping wheel shaft (not shown in the figure), a clamping wheel 326, a locking component 327, and a wrench component 328.

[0096] The pressing bracket 321 includes a mounting block 3211 and a guide rod 3212. The guide rod 3212 is a cylindrical rod, and its extending direction is consistent with the length direction of the vehicle body 20.

[0097] The guide member 322 includes a guide rail 3221 and a slider 3222. The guide rail 3221 is fixedly mounted on the housing 211 and extends in the same direction as the guide rod 3212. The slider 3222 is slidably mounted on the guide rail 3221.

[0098] The elastic member 323 is a spring, which is also mounted on the guide rod 3212 , and one end of the spring abuts against the mounting block 3211 , and the other end is embedded in the cylindrical groove on the wheel bracket 324 and abuts against the bottom of the groove.

[0099] The pinch wheel shaft is mounted in the notch at the end of the wheel body bracket 324, and the pinch wheel 326 is rotatably mounted on the pinch wheel shaft and partially embedded in the notch. The pinch wheel 326 is also made of the above-mentioned insulating material.

[0100] Therefore, under the action of the spring force of the elastic member 323, the wheel support 324 and the clamping wheel 326 thereon can be pressed toward the inner side of the rail, so that the clamping wheel 326 is closely attached to the active edge of the rail. In this embodiment, when the track inspection vehicle 100 is placed on the rail, the clamping wheel 326 is located 16 cm below the rail surface.

[0101] The locking component 327 includes a locking fitting 3271 and a locking member 3272 .

[0102] The locking member 3272 is a locking screw, which is installed at the corresponding locking member hole on the mounting block 3211, and the end of the screw can move along the hole.

[0103] The locking fitting 3271 is in the shape of a strip plate with multiple circular holes along its length. One end of the locking fitting 3271 is mounted on the wheel support 324, and the other end passes through the locking fitting hole in the mounting block 3211 and can move along the hole. The locking hole is connected to the locking fitting hole. When the screw end of the locking member 3272 extends downward, the screw end passes through the locking hole to the locking fitting hole and then penetrates the circular hole on the end of the locking fitting 3271, thereby fixing (locking) the relative position of the wheel support 324 and the mounting block 3211.

[0104] In this embodiment, the locking members 3272 of the two clamping assemblies 32 corresponding to one rail 9 are unlocked, and their elastic members 323 are free to move. In the two clamping assemblies 32 on the other side, the locking members 3272 are locked, and the positions of the wheel brackets 324 and the mounting blocks 3211 are fixed, that is, the positions of the clamping wheels 326 relative to the vehicle body 20 are fixed. In other words, in this embodiment, the two clamping assemblies 32 on one side are in the form of fixed side wheels, and the two on the other side are in the form of spring side wheels.

[0105] The wrench component 328 includes two combined connecting rods, which are used for the inspection personnel to adjust the relative position of the clamping wheel 326 with respect to the vehicle body 20, so as to make it easier to place the track inspection vehicle 100 on the rail.

[0106] The pushing mechanism 40 comprises a push rod and a notebook support installed at the end of the push rod. The angle of the push rod is adjustable, and the angle of the notebook support relative to the push rod is also adjustable.

[0107] Figure 8 4 is a flow chart of the data collection steps in this embodiment.

[0108] like Figure 8 As shown, based on the above-mentioned track inspection vehicle 10, in this embodiment, the data collection step S1 specifically includes the following sub-steps:

[0109] In the data acquisition preparation sub-step S1-1, the inspection personnel place the track inspection vehicle on the two main line rails and start the sensors and data collectors on the track inspection vehicle.

[0110] The inspector places the four load-bearing wheels 313 of the track inspection vehicle 10 on two main rails, and presses the four clamping assemblies 32 toward the inner sides of the corresponding rails. The onboard camera, line laser sensor, data collector, and inspection equipment are then activated.

[0111] In the data acquisition sub-step S1-2, the inspection personnel push the track inspection vehicle along the main line rail. During the movement, the track inspection vehicle obtains the two-dimensional image, point data and mileage data of the main line rail through the on-board camera, line laser sensor and mileage encoder.

[0112] Image preprocessing step S2: preprocessing the acquired two-dimensional image.

[0113] In this embodiment, the preprocessing of the two-dimensional image includes adjusting the size of the image so that it can be used as an input to the neural network model, and adaptively adjusting the brightness of the image to minimize the impact of changes in ambient lighting conditions on detection.

[0114] Light band area segmentation step S3: the pre-processed two-dimensional image is input into the track surface light band segmentation model, and the model outputs the shape and position information of the segmented light band area.

[0115] Among them, the rail surface light band segmentation model is a trained neural network model, whose input is a two-dimensional image of the rail, and its output is a binary image representing the rail surface light band area in the two-dimensional image.

[0116] In this embodiment, the rail surface light band segmentation model is an improved YOLOP convolutional neural network model, which is a semantic segmentation model. The rail surface light band recognition model includes a Backbone module, a Neck module and a Head module. Among them, the Backbone module includes a CSP-Darknet structure, which is used to extract the feature map of the two-dimensional image. The Neck module includes a spatial pyramid pooling layer (SPP) and a feature pyramid structure. The extracted multi-scale feature maps are fused through the SPP, and then the feature maps with different semantics are fused through the feature pyramid structure. The structure of the Backbone module and the Neck module is consistent with the original YOLOP model. The Head module includes a segmentation detection head and a target detection head, that is, one segmentation detection head is cropped relative to the original YOLOP model. The fused feature map passes through the target detection head to generate light band positioning information, including the coordinates of the rail surface anchor frame for identifying the rail surface area in the two-dimensional image, and the coordinates of the light band anchor frame for identifying the light band area on the rail surface. The fused feature map passes through the segmentation detection head to generate a binary map of the light band area. During this process, the light band positioning information generated by the target detection head is also provided to the segmentation detection head as auxiliary detection information, making the light band segmentation detection results more accurate and smooth.

[0117] To train this model, a large number of rail surface light band images and corresponding light band labels are first prepared as training data. The rail surface light band images are two-dimensional images of the rails acquired using the same method described above. Experienced inspectors manually delineate the light band areas within the 2D images, and corresponding training labels are generated based on these delineations. Data augmentation is then performed on the training data to improve the robustness and generalization capabilities of the trained model. This augmented training data is then used to train the track surface chipping area recognition model.

[0118] In this embodiment, the data augmentation algorithm performs geometric transformations on the two-dimensional track images, including cropping, scaling, translation, inversion, and rotation, to generate datasets of varying physical sizes. The two-dimensional track images are then subjected to color-shifting data transformations, including noise addition, color perturbation, and blurring. This data augmentation algorithm generalizes to a new dataset several times larger than the original training dataset.

[0119] Figure 9 4 is a flow chart of the light band area segmentation steps in this embodiment.

[0120] like Figure 9 As shown, the light band area segmentation step S3 specifically includes the following sub-steps:

[0121] Binary image generation sub-step S3-1: input the two-dimensional image into the trained track surface light band segmentation model, and the model outputs a binary image of the segmented track surface light band.

[0122] Binary image conversion sub-step S3-2: According to a predetermined image conversion algorithm, the pixel points of the light band area in the binary image are converted into a polygonal surface composed of inflection points.

[0123] Connected domain filling sub-step S3-3: Fill the holes in the connected domain of each polygonal surface according to a predetermined hole filling algorithm to obtain a target polygonal surface (target block).

[0124] False detection filtering sub-step S3-4: determine whether the pixel area of ​​the target polygonal surface is smaller than a predetermined area, and filter the target polygonal surface smaller than the predetermined area as a false detection.

[0125] In this embodiment, target polygonal faces with a pixel area less than 5000 are filtered as false detections.

[0126] Figure 10 It is a binary image of the rail surface light band in this embodiment.

[0127] like Figure 10 As shown, in the binary image, the background is black and the identified light band area is white. It can be seen that the edges of the white light band area on both sides in the width direction are roughly straight, while the edges on both sides in the length direction are irregular.

[0128] Figure 11 This is a schematic diagram of marking light band areas in an original two-dimensional image based on a binary image in this embodiment.

[0129] like Figure 11 As shown in the figure, a frame is generated based on the boundaries of the light band area in the binary image and superimposed on the original two-dimensional image for visualization, allowing the track surface light band portion identified from the original two-dimensional image to be intuitively seen. It can be seen that the irregularities on both sides of the longitudinal edges of the white light band area in the binary image are due to the presence of some overexposed areas in the original two-dimensional image. The track surface light band segmentation model filters out these overexposed areas during identification and segmentation, which means that the identified track surface light band portion is incomplete.

[0130] Step S4 of acquiring two-dimensional information of the light band: Based on the shape and position information of the light band on the track surface, the two-dimensional information of the light band on the track surface is calculated by sampling, including the average width of the light band and the coordinates of the center point of the light band. The coordinates here refer to the pixel coordinates of the two-dimensional image.

[0131] like Figure 11 As shown in the figure, the two ends of the track surface light strip in the longitudinal direction are incomplete. Therefore, in order to eliminate the influence of the two ends, when calculating the width of the track surface light strip, the width of the middle 60% length area in the longitudinal direction is calculated, and then the average value of the width is calculated to obtain a more accurate light strip width.

[0132] Figure 12 4 is a flow chart of the steps for obtaining two-dimensional information of the light band in this embodiment.

[0133] like Figure 12 As shown, the light band two-dimensional information acquisition step S4 specifically includes the following sub-steps:

[0134] Light band area sampling sub-step S4-1: In the middle 60% area of ​​the length direction of the light band area in the binary image, a predetermined number of samplings are performed along the Y axis at a predetermined sampling interval to obtain multiple light band area samples.

[0135] Among them, it is preset to perform n samplings, and the sampling interval stride is:

[0136]

[0137] Where Y lightband It is the Y-axis coordinate of the center point of the light band, that is, the Y-axis coordinate of the midpoint of the length direction of the light band area.

[0138] Average width calculation sub-step S4-2: Calculate the width of each light band area sample and calculate the average width of these multiple widths.

[0139] Among them, the average width MeanWidth is:

[0140] y=0.2*Y lightband +y+i*stride

[0141]

[0142] Where i represents the i-th sampling.

[0143] Light band center point calculation sub-step S4-3: Calculate the difference in X-axis coordinates of the two endpoints of each light band area sampled above, and calculate the average of these multiple differences as the X-axis coordinate of the light band center point; the Y-axis coordinate of the light band center point is as described above, that is, the Y-axis coordinate of the midpoint of the length direction of the light band area.

[0144] Among them, the X-axis coordinate of the center point of the light band is:

[0145] y=0.2*Y lightband +y+i*stride

[0146]

[0147] Rail point cloud generation step S5: Based on the mileage information, multiple frames of point data corresponding to the two-dimensional image are extracted from the collected point data to generate a rail point cloud of the rail.

[0148] Figure 13 It is a flow chart of the steps for generating the track point cloud in this embodiment.

[0149] like Figure 13 As shown, the rail point cloud generation step S5 specifically includes the following sub-steps:

[0150] Point data extraction sub-step S5-1: Based on the mileage information corresponding to the two-dimensional image, corresponding multi-frame point data are extracted from all collected point data.

[0151] Figure 14 It is a schematic diagram of a frame of point data in this embodiment.

[0152] like Figure 14 As shown, in this embodiment, three line laser sensors are used. The first line laser sensor 217a acquires point data for the contours of the inactive rail edge, the rail web flank, and the lower jaw. The second line laser sensor 217b acquires point data for the contours of the rail surface and inactive rail edge. The third line laser sensor 217c acquires point data for the contours of the rail surface, the active rail edge, and the rail web flank. This yields three sections of contour data for the same rail cross section. Based on the calibration relationship, these three sections can be combined to obtain a single frame of point data for the contour of a single rail cross section.

[0153] Point data stacking sub-step S5-2: Based on the corresponding mileage information, the extracted multiple frames of point data are stacked along the Y-axis direction.

[0154] Each frame of point data is in the XZ coordinate system, and each point contains corresponding mileage information. Multiple frames of point data are stacked along the Y axis according to this mileage information. The coordinates here refer to the world coordinates of the 3D point cloud.

[0155] Rail point cloud generation sub-step S5-3: convert the stacked multi-frame point data into a rail point cloud. The specific conversion algorithm is prior art and will not be described in detail.

[0156] Figure 15 Schematic diagram of the rail point cloud in this embodiment.

[0157] like Figure 15 As shown in the figure, since the line laser projected by the line laser sensor also covers part of the sleepers, the base beside the rails and other locations, the track point cloud contains not only the point cloud of the rail contour, but also the point clouds of these parts. Therefore, some cropping is required to crop out the rail contour point cloud.

[0158] Rail point cloud clipping sub-step S5-4: Based on the distance between the line laser sensor and the rail surface, the rail point cloud is clipped to obtain the rail point cloud.

[0159] Light band width acquisition step S6: Based on the camera intrinsic parameters and the calibration relationship between the camera and the line laser sensor, the average width of the rail surface light band is converted to the coordinate system of the rail point cloud, and the physical width of the rail surface light band is obtained based on the converted average width.

[0160] Among them, the average width of the rail surface light band is first converted into the width in camera coordinates based on the camera intrinsic parameters, and then the width in camera coordinates is converted into the width in world coordinates of the three-dimensional point cloud based on the calibration relationship between the camera and the line laser sensor.

[0161] That is, the coordinates of the pixel points are corrected through the camera intrinsic parameter matrix to obtain their true pixel positions, and then the pixel positions are projected into the world coordinate system of the three-dimensional point cloud to obtain the physical width of the rail surface light band.

[0162] Specifically, let the camera intrinsic parameter matrix be:

[0163]

[0164] The coordinates (u, v) of the pixel points within the light band anchor frame are converted to camera coordinates (X, Y, Z) according to the intrinsic parameter matrix:

[0165]

[0166] The camera coordinates (X, Y, Z) can be obtained as follows:

[0167]

[0168]

[0169] Furthermore, the camera coordinate P C Generate world coordinates P by rotating the matrix R and offsetting the matrix t W :

[0170] P C =RP W +t

[0171] Among them, the rotation matrix R and the offset matrix t can be obtained according to the calibration parameters.

[0172] Solve the world coordinate P by the following matrix W :

[0173]

[0174]

[0175] Light band center point extraction step S7: Based on the camera intrinsic parameters and the calibration relationship between the camera and the line laser sensor, the coordinates of the light band center point of the rail surface light band are converted to the coordinate system of the rail point cloud.

[0176] The specific conversion process is the same as step S6 and will not be repeated.

[0177] Rail center point extraction step S8: With the light band center point as the center, extract a point cloud within a predetermined range from the rail point cloud as the light band effective point cloud, and based on the light band effective point cloud, obtain the rail center point coordinates of the rail by fitting a plane.

[0178] Figure 16 Flowchart of the rail center point extraction steps in this embodiment.

[0179] like Figure 16 As shown, the rail center point extraction step S8 specifically includes the following sub-steps:

[0180] Light band valid point cloud extraction sub-step S8-1: Based on the world coordinates of the light band center point and the mileage information, with the light band center point as the center, the point cloud within a predetermined mileage range is cropped from the track point cloud as the light band valid point cloud.

[0181] That is to say, cut out the middle section of the track surface light strip for further calculation and analysis.

[0182] Figure 17 Schematic diagram of the effective point cloud of the light band cut out in this embodiment.

[0183] like Figure 17 As shown, in this embodiment, based on the mileage corresponding to the center point of the light strip, the point cloud of the mileage 20 mm before and after the center point of the light strip along the Y-axis direction is cropped from the rail point cloud, and the point cloud of the rail head part is roughly cropped along the Z-axis as the effective point cloud of the light strip.

[0184] Rail surface fitting sub-step S8-2: Based on the effective point cloud of the light band, the rail surface plane is fitted using the RANSAC algorithm.

[0185] Figure 18 4 is a flow chart of plane fitting using the RANSCAC algorithm in this embodiment.

[0186] like Figure 18As shown in Figure 1, the process of fitting the rail surface plane using the RANSAC algorithm specifically includes the following steps:

[0187] Step S8-2-1: Roughly select the part of the point cloud corresponding to the rail surface in the effective point cloud of the light band.

[0188] Step S8-2-2: In this part of the point cloud, randomly sample at least three points (the minimum number of points that can fit the plane model) as initial points.

[0189] Step S8-2-3: Assume that the initial equation of the plane model is AX+BY+CZ+D=0, and fit the plane model based on multiple initial points to obtain the values ​​of the plane model parameters A, B, C, and D.

[0190] Step S8-2-3a: Set k=1, that is, specify the first point.

[0191] Step S8-2-4: Select the kth point other than the initial point from all points in the selected partial point cloud, and calculate the distance d1 from the point to the plane model:

[0192]

[0193] Where x0, y0, and z0 are the coordinates of the selected point.

[0194] Step S8-2-5: Determine whether the distance d1 is less than a preset distance threshold, and if so, add the point to the inner group.

[0195] Step S8-2-6: Calculate the ratio of the number of points in the inner group to the total number of points in the effective point cloud of the light band, and determine whether the ratio is greater than a preset ratio threshold.

[0196] Step S8-2-7: When the judgment in step S8-2-6 is yes, the plane model parameters a, b, c, and d are obtained based on the inner group points (i.e., fitting is performed again based on the inner group points to update the parameters).

[0197] Step S8-2-8: Based on the updated parameters, calculate the Z-axis coordinate of the center point of this part of the point cloud as the Z-axis coordinate of the highest point on the rail surface:

[0198] Z max =-(ax+by+d) / c

[0199] Step S8-2-9: When the judgment in step S8-2-6 is no, set k=k+1 and return to step S8-2-4.

[0200] Rail surface highest point calculation sub-step S8-3: Calculate the coordinates of the center point of the fitted rail surface plane as the coordinates of the highest point of the rail surface.

[0201] Inactive edge fitting sub-step S8-4: Based on the effective point cloud of the light strip, the inactive edge plane is fitted using the RANSAC algorithm.

[0202] The fitting process using the RANSAC algorithm is the same as step S8-2 and will not be repeated.

[0203] Mandible point calculation sub-step S8-5: Calculate the coordinates of the mandible point based on the coordinates of the highest point of the rail surface and the fitted non-active edge plane.

[0204] Specifically, the Y-axis coordinate of the jaw point is the Y value of the center point of the light band, and its Z-axis coordinate is the Z-axis coordinate of the highest point of the rail surface minus 34 (the distance from the top surface of the standard rail head to the jaw point is 34mm). Its X-axis coordinate is calculated by the plane equation:

[0205] z jaw =z max -34

[0206] x jaw =-(by+cz jaw +d) / a

[0207] Rail center point calculation sub-step S8-6: Based on the coordinates of the lower jaw point, the coordinates of the rail center point of the rail are calculated.

[0208] Specifically, the width of the lower jaw point of the standard rail head is 73 mm at both ends. Therefore, after obtaining the X-axis coordinate of the lower jaw point, it is offset 36.5 mm in the predetermined direction as the center point of the rail. That is, based on the train travel direction, the X-axis coordinate of the center point of the left rail is:

[0209] x rail =x jaw -36.5

[0210] The X-axis coordinate of the center point of the right rail is:

[0211] x rail ′=x jaw +36.5

[0212] Light band offset acquisition step S9: Based on the converted light band center point coordinates and the rail member center point coordinates, the offset of the rail surface light band relative to the rail center axis is calculated.

[0213] That is, calculate the difference in the world coordinates of the X axis between the center point of the light strip and the center point of the track surface as the deviation of the track surface light strip. Set the deviation to a negative value when the track surface light strip deviates toward the active edge, and a positive value when it deviates toward the inactive edge.

[0214] Specifically, the offset of the left rail is:

[0215] offset=xlightband -x rail

[0216] The offset of the right rail is:

[0217] offset′=x rail -x lightband

[0218] The light band abnormality type and level determination step S10 determines the abnormality type and level of the track surface light band based on the physical width and offset of the track surface light band and predetermined abnormality threshold information of the track surface light band.

[0219] In this embodiment, the preset abnormal threshold information includes multiple abnormal types, the level of each abnormal type and the corresponding threshold information. The abnormal types include increased light band width, decreased light band width, inward movement of the light band, and outward movement of the light band. Each abnormal type includes multiple levels, and each level has corresponding threshold information. When the calculated light band width and deviation amount fall within the corresponding threshold range, it is determined that the track surface light band belongs to the level of the abnormal type.

[0220] In this embodiment, the two-dimensional image, mileage information, the binary image of the track surface light band, the effective point cloud of the light band, the width and offset of the track surface light band, and the abnormality type and level of the track surface light band are also stored as corresponding light band information. This makes it easier for inspection personnel to verify and further analyze the automatically identified track surface light band and its information. For example, inspection personnel can filter out a certain type of light band abnormality from the stored light band information and process it first; or inspection personnel can call up the two-dimensional image and point cloud data of the track surface light band for review; or inspection personnel can locate the corresponding position on the rail based on the mileage information corresponding to the identified track surface light band and go to that position for manual inspection.

[0221] Figure 19 4 is a block diagram of the rail surface light band detection device in this embodiment.

[0222] like Figure 19 As shown, this embodiment also provides a rail surface light band detection device 60 corresponding to the above method, which includes a model storage unit 61, a detection data acquisition unit 602, an image preprocessing unit 603, a light band area segmentation unit 604, a light band two-dimensional information calculation unit 605, a rail point cloud generation unit 606, a light band width acquisition unit 607, a light band center point extraction unit 608, a rail center point extraction unit 609, a light band offset acquisition unit 610, a light band abnormality type and level determination unit 611, a light band information storage unit 612, a light band detection communication unit 613, and a light band detection control unit 614.

[0223] The model storage unit 61 stores the track surface light band segmentation model. The detection data acquisition unit 602 acquires the two-dimensional image captured by the camera, the contour point data collected by the line laser sensor, and the mileage information collected by the mileage encoder through the data collector. The image preprocessing unit 603 preprocesses the two-dimensional image according to the method of step S2 above. The light band area segmentation unit 604 identifies and segments the light band area from the two-dimensional image according to the method of step S3 above, and obtains a binary image. The light band two-dimensional information calculation unit 605 calculates the two-dimensional information of the light band based on the binary image according to the method of step S4 above. The track point cloud generation unit 606 extracts multiple frames of point data and generates a track point cloud according to the method of step S5 above. The light band width acquisition unit 607 calculates the physical width of the light band according to the method of step S6 above. The light band center point extraction unit 608 obtains the world coordinates of the light band center point according to the method of step S7 above. The track center point extraction unit 609 obtains the world coordinates of the track center point according to the method of step S8 above. The light band offset acquisition unit 610 calculates the offset of the light band center point relative to the track center point using the method described in step S9. The light band abnormality type and level determination unit 611 determines the type and level of the light band abnormality using the method described in step S10. The light band information storage unit 612 stores the detected and calculated light band information. The light band detection communication unit 613 communicates with other devices. The light band detection control unit 614 controls the operation of each of the aforementioned functional units.

[0224] As one example, the rail surface light strip detection device 60 is a laptop computer with a corresponding detection program installed. The laptop computer is placed on a laptop holder on the rail inspection vehicle 10 and connected to the vehicle's data acquisition device via a cable. Inspection personnel can use the laptop computer to conveniently obtain light strip detection results.

[0225] In this embodiment, parts not described in detail are well-known technologies in the art.

[0226] Example Function and Effect

[0227] According to the rail surface light band detection method and equipment provided by this embodiment, the method includes a light band area identification step, a light band two-dimensional information acquisition step, a rail point cloud generation step, a rail center point extraction step, and a light band offset acquisition step, wherein the light band area in the two-dimensional image is identified and segmented by a trained semantic segmentation model, and the rail surface light band is identified comprehensively, quickly and accurately; partial contour point position data corresponding to the two-dimensional image is extracted to generate a rail point cloud, and a small range of point cloud in the middle of the light band is further extracted based on the light band center point as the light band effective point cloud for further Calculation and analysis, therefore, only a relatively small amount of data collected by line laser sensors needs to be analyzed and processed, the amount of calculation is small, and the detected rail surface light band and its offset can be quickly output, which is beneficial to on-site detection and analysis; the on-site rail surface not only has light bands, but may also have other types of damage. In the method of the present invention, based on the extracted light band effective point cloud, the coordinates of the rail center point are obtained by fitting the plane, thereby calculating the light band offset. By fitting the plane, the influence of some other types of rail surface damage on the calculation of the light band offset can be effectively eliminated, thereby obtaining an accurate light band offset. As described above, the method and equipment of the present invention can accurately and comprehensively identify the light band on the rail surface, calculate its accurate offset, and the amount of calculation is small, which is very beneficial to on-site track detection.

[0228] In the embodiment, a track inspection vehicle equipped with a camera, a line laser sensor, and an mileage encoder is used to collect rail data on site. The inspection personnel only need to start the equipment and push the vehicle along the rail to collect data. The operation is convenient, and a variety of data required for the inspection can be obtained with one movement of the vehicle.

[0229] Furthermore, the camera is arranged in the shell of the inspection vehicle, and a lighting unit is also provided in the shell, thereby reducing the impact of changes in external light conditions on camera shooting and providing sufficient lighting for camera shooting, thereby improving the quality of the two-dimensional image captured by the camera.

[0230] Furthermore, a laptop computer is placed on the handle of the track inspection vehicle. The laptop computer contains a preset inspection program and is connected to the camera and sensor through a data collector. Therefore, the inspection personnel can easily obtain the real-time data collection status and the rail block drop detection results through the laptop computer, discover problems in time and make adjustments or re-measurements, etc.

[0231] In this embodiment, an improved YOLOP convolutional neural network model is used as a semantic segmentation model to automatically identify and segment track light bands. The segmentation decoder of the YOLOP network model has high-precision output and very fast inference speed, which helps improve detection efficiency. Furthermore, the training data used during the model training has undergone data augmentation, which can improve the model's robustness and generalization ability.

[0232] In the embodiment, based on the calculated center point of the light strip, only the point cloud data of the middle 40 mm of the light strip is extracted, and the point cloud of the rail head part is cropped to retain only the point cloud for calculating the coordinates and offset of the rail center point. The amount of point cloud data is small, so the amount of calculation is small, and the detection results can be quickly obtained during on-site detection, which is beneficial to reducing the time for on-site line stop detection.

[0233] Furthermore, based on the extracted effective point cloud of the light band, the rail surface plane and the non-active edge plane are fitted using a plane equation fitting algorithm. The coordinates of the highest point on the rail surface are extracted from the rail surface plane. The coordinates of the lower jaw point on the non-active edge are then extracted from the non-active edge plane and the highest point on the rail surface. The coordinates of the rail center point are then obtained based on the coordinates of the lower jaw point. The plane fitting eliminates the impact of other types of rail surface damage on the detection. Furthermore, since the non-active edge does not contact the wheel, the lower jaw point on the non-active edge is not subject to wear. Therefore, the accurate coordinates of the rail center point can be calculated, resulting in a more accurate offset, which helps to improve the accuracy of further analysis of the light band and track.

[0234] In the embodiment, the abnormal type and level of the track surface light band are automatically determined based on the physical width, offset and preset threshold information of the track surface light band, and the two-dimensional image, point cloud data, binary image, calculated width and offset and other information of the light band are stored accordingly. Therefore, the inspection personnel can easily review and confirm the identified track surface light band or further analyze it, and can, for example, screen out a certain type of light band abnormality for priority processing, thereby further improving the efficiency and accuracy of track surface light band detection, and the high degree of automation can reduce the burden on inspection personnel.

[0235] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the description scope of the above embodiments.

[0236] In the above embodiment, the mainline rail is used as an example for specific description. It can be understood that since the structures of the basic rail and wing rail at the switch are similar to those of the mainline rail, the above method can also be applied to the basic rail and wing rail at the switch. Accordingly, when training the track surface light band segmentation model (semantic segmentation model), the corresponding two-dimensional image of the switch is used as training data.

[0237] In the above embodiment, the abnormal types and corresponding threshold information of the increase in the width of the track surface light band, the decrease in the width of the light band, the inward movement of the light band, and the outward movement of the light band are set. In an alternative solution, different abnormal types and corresponding threshold information can also be set according to actual detection needs.

[0238] In the above embodiment, the detection and analysis of the light bands of each corresponding part are performed based on a single two-dimensional image. In an alternative solution, further calculation and analysis can be performed based on the saved light band information. For example, based on the light band information corresponding to multiple consecutive two-dimensional images, the changes in the light band edges can be further calculated and analyzed.

[0239] In the above embodiment, for the convenience of description, two-dimensional image preprocessing, light band area segmentation and other steps are performed after the rail data collection step is completed. It can be understood that if the computing power is sufficient, two-dimensional image preprocessing, light band area segmentation and other steps can be performed on the collected data during the data collection process, thereby further improving the real-time performance of the detection.

[0240] In the above embodiment, some conventional exception handling steps are omitted. For example, if no light band region is identified and segmented in step S2, then the subsequent steps are unnecessary and processing can proceed directly to the next 2D image. This means that the 2D image is used for prejudgment, and if no light band region is identified, the corresponding 3D data is not processed, effectively reducing the amount of computation. A corresponding warning message can also be generated for further verification by the inspector. Similarly, in step S3, if the sampled width is 0, a corresponding warning message is generated and subsequent steps are not performed. The same applies to the other steps and will not be described in detail.

[0241] In the above embodiment, the rail surface light band detection device is a laptop computer mounted on a track inspection vehicle. In an alternative solution, the device may also be other devices mounted on a track inspection vehicle, such as a tablet computer; the device may also include a remote server and multiple terminals connected to the remote server for communication. The data obtained by detection is transmitted to the remote server, and the remote server then sends the detection results obtained by calculation and analysis to the terminal held by the inspection personnel, which can also achieve the corresponding technical effect.

Claims

1. A method for detecting light bands on rail surfaces, characterized in that: The method includes detecting a rail surface light band on the rail surface based on a two-dimensional image of the rail surface acquired by a camera, contour point position data of the rail acquired by a line laser sensor, and corresponding mileage information. Light band area segmentation step: inputting the two-dimensional image into a track surface light band segmentation model, which outputs the shape and position information of the segmented track surface light band, wherein the track surface light band segmentation model is a trained semantic segmentation model; The step of acquiring two-dimensional information of the light band: based on the shape and position information of the light band on the track surface, calculating the two-dimensional information of the light band on the track surface, including the coordinates of the center point of the light band, by sampling; A rail point cloud generating step: extracting multiple frames of contour point data corresponding to the two-dimensional image from the collected contour point data based on the mileage information, and generating a rail point cloud of the rail; Light band center point extraction step: based on the camera intrinsic parameters of the camera and the calibration relationship between the camera and the line laser sensor, converting the coordinates of the light band center point to the coordinate system of the track point cloud; The rail center point extraction step includes extracting a predetermined range of point clouds from the rail point cloud with the center point of the light band as the center as the light band effective point cloud, and obtaining the coordinates of the rail center point of the rail by fitting a plane based on the light band effective point cloud; The light band offset acquisition step is as follows: based on the center point of the light band and the center point of the track, the offset of the track surface light band is calculated. The rail center extraction step includes: Light band effective point cloud extraction sub-step: based on the converted coordinates of the center point of the light band and the mileage information, cutting out a point cloud within a predetermined mileage range from the track point cloud with the center point of the light band as the light band effective point cloud; Rail surface fitting sub-step: fitting the rail surface plane based on the effective point cloud of the light band by using the RANSAC algorithm; The rail surface highest point calculation sub-step: calculating the coordinates of the center point of the rail surface plane as the coordinates of the rail surface highest point; Non-active edge fitting sub-step: fitting the non-active edge plane by the RANSAC algorithm based on the effective point cloud of the light band; A jaw point calculation sub-step: calculating the coordinates of the jaw point based on the coordinates of the highest point of the rail surface and the inactive edge plane; Rail center point calculation sub-step: based on the coordinates of the mandibular point, calculate the coordinates of the rail center point.

2. The rail surface light band detection method according to claim 1, Its characteristics are: The light band area segmentation step includes: A binary image generating sub-step is to input the two-dimensional image into the track surface light band segmentation model, and the model outputs a binary image of the track surface light band; A binary image conversion sub-step, converting the pixel points of the light band area in the binary image into a polygonal surface composed of inflection points; a connected domain filling sub-step, filling the holes in the connected domain of the polygonal surface according to a predetermined hole filling algorithm to obtain a filled target polygonal surface; The false detection filtering sub-step determines whether the pixel area of ​​the target polygonal surface is smaller than a predetermined area, and filters out the target polygonal surface smaller than the predetermined area as a false detection.

3. The rail surface light band detection method according to claim 2, Its characteristics are: The step of acquiring the two-dimensional information of the light band includes: Light band area sampling sub-step: performing predetermined sampling times along the Y axis at a predetermined sampling interval in the middle of the length direction of the light band area in the binary image to obtain a plurality of light band area samples; Light band center point calculation sub-step: calculate the difference in X-axis coordinates of the two endpoints of each light band area sample, and calculate the average value of multiple differences as the X-axis coordinate of the light band center point, and use the Y-axis coordinate of the midpoint of the length direction of the light band area as the Y-axis coordinate of the light band center point.

4. The rail surface light band detection method according to claim 3, characterized in that: in, In the light band offset acquisition step, the offset of the rail surface light band relative to the central axis of the rail is calculated based on the converted coordinates of the light band center point and the coordinates of the rail member center point.

5. The rail surface light band detection method according to claim 1, Its characteristics are: The RANSAC algorithm includes the following steps: Step 1: Select the corresponding part of the point cloud in the effective point cloud of the light band; Step 2: Randomly sample at least three points in the partial point cloud as initial points; Step 3: Assume that the initial equation of the plane model is AX+BY+CZ+D=0, and fit the plane model based on the multiple initial points to obtain the values ​​of the plane model parameters A, B, C, and D; Step 4: Among all the points in the selected part of the point cloud, select a point other than the initial point and calculate the distance d1 from the point to the plane model: Where x0, y0, z0 are the coordinates of the selected point; Step 5: Determine whether the distance d1 is less than a preset distance threshold, and if so, add the point to the inner group; Step 6: Calculate the ratio of the number of points in the inner group to the total number of points in the effective point cloud of the light band, and determine whether the ratio is greater than a preset ratio threshold; Step 7: When the answer in step 6 is yes, update the plane model parameters based on the points in the inner group to obtain updated parameters a, b, c, and d; Step 8: Based on the updated parameters, calculate the Z-axis coordinate of the center point of the partial point cloud as the Z-axis coordinate of the highest point on the rail surface: Z max =-(ax+by+d) / c。 6. The rail surface light band detection method according to claim 5, characterized in that: in, In the mandibular point calculation sub-step, the X-axis coordinate of the mandibular point is: With jaw =z max -34 x jaw =-(by+cz jaw +d) / a Where x jaw 、z jaw are the X-axis coordinate and Z-axis coordinate of the mandibular point respectively, In the rail center point calculation sub-step, the coordinates of the rail center points of the rails on both sides are respectively: x rail =x jaw -36.5 x rail ′=x jaw +36.5 In the light band offset acquisition step, the offsets of the rails on both sides are respectively: offset=x lightband -x rail offset′=x rail -x lightband Where x lightband is the X-axis coordinate of the center point of the light band after conversion.

7. The rail surface light band detection method according to claim 3, characterized in that: in, The light band two-dimensional information acquisition step, after the light band area sampling sub-step, further includes: The average width calculation sub-step is to calculate the width of each of the light band area samples and calculate the average width of the multiple widths. After the step of acquiring the two-dimensional information of the light band, the method further includes: The light band width acquisition step converts the average width into the coordinate system of the rail point cloud based on the camera intrinsic parameters of the camera and the calibration relationship between the camera and the line laser sensor, thereby obtaining the physical width of the rail surface light band.

8. The rail surface light band detection method according to claim 7, characterized in that: After the light band width acquisition step and the light band offset acquisition step, the method further includes: The step of determining the abnormality type and level of the track light band is as follows: determining the abnormality type and level of the track light band based on the physical width of the track light band, the offset and predetermined abnormality threshold information of the track light band.

9. A rail surface light band detection device, characterized in that: The device detects the light band area on the rail surface based on the two-dimensional image of the rail surface collected by the camera, the contour point data of the rail collected by the line laser sensor, and the corresponding mileage information. The device includes: A model storage unit is used to store a track light band segmentation model, which is a trained semantic segmentation model; a light band area recognition unit, configured to input the two-dimensional image into the track surface light band segmentation model to obtain shape and position information of the segmented light band area; a light band two-dimensional information acquisition unit, configured to calculate the two-dimensional information of the track surface light band, including the coordinates of the center point of the light band, by sampling based on the shape and position information of the track surface light band; A rail point cloud generating unit is configured to extract multiple frames of contour point data corresponding to the two-dimensional image from the collected contour point data based on the mileage information, and generate a rail point cloud of the rail; A light band center point extraction unit is configured to convert the coordinates of the light band center point into the coordinate system of the track point cloud based on the camera intrinsic parameters of the camera and the calibration relationship between the camera and the line laser sensor; A rail center point extraction unit is configured to extract a point cloud within a predetermined range from the rail point cloud with the light band center point as the light band effective point cloud, and obtain the coordinates of the rail center point of the rail by fitting a plane based on the light band effective point cloud; and A light band offset acquiring unit is used to calculate the offset of the track surface light band based on the center point of the light band and the center point of the track member. The rail center point extraction unit obtains the coordinates of the rail center point by: Based on the converted coordinates of the center point of the light strip and the mileage information, a point cloud within a predetermined mileage range is cropped from the track point cloud with the center point of the light strip as the effective point cloud of the light strip; Based on the effective point cloud of the light strip, the rail surface plane is fitted by the RANSAC algorithm; Calculate the coordinates of the center point of the rail surface plane as the coordinates of the highest point of the rail surface; Based on the effective point cloud of the light band, fitting the inactive edge plane by using the RANSAC algorithm; Calculating the coordinates of the lower jaw point based on the coordinates of the highest point of the rail surface and the inactive edge plane; Based on the coordinates of the mandible point, the coordinates of the center point of the rail are calculated.

Images