A robot-based track fastener intelligent inspection and maintenance system and a maintenance method

Abstract

The application discloses a kind of track fastener intelligent inspection maintenance systems and maintenance methods based on robot.The fastener inspection maintenance is one of the daily key work of railway department.The application includes self-walking platform, positioning detection system module, control system module, tight subsystem module, rust removal and oiling subsystem module, safety protection system and embedded data processing system.The application combines with laser radar positioning technology, laser ranging technology and machine vision image recognition technology, utilizes motion control technology, realizes the intelligent tightening or loosening of the screw bolt of the loosened fastener inspected automatically, and the rust removal, oiling and drying operation of the fastener parts with unqualified surface state inspected automatically;Realize the full automation and intelligentization of track fastener state detection and maintenance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of track fastener detection and maintenance and repair, and particularly relates to a track fastener intelligent inspection and maintenance system based on a robot and a maintenance method. BACKGROUND

[0002] Safety is the first element of rail transit. As an important connecting part for maintaining the safety of railway transportation, track fasteners play an important role in the safe operation of railway transportation, and fastener inspection and maintenance is also one of the daily key work of railway maintenance departments.

[0003] At present, in the equipment maintenance of existing high-speed railways, freight special lines, subways and other rail transit lines, the fastener state detection methods usually have the following several kinds: the first kind is manual visual inspection, which uses a track hammer to visually inspect the fasteners along the line one by one, which has the problems of high labor intensity, low efficiency and rough diagnosis and evaluation. The second kind is a method based on image comparison, which can effectively identify whether the fastener parts are missing or the surface state of the parts, but cannot detect the tightness of the fasteners and bolts, and the detection efficiency is low. The third kind is a method based on vibration signals, which is easily affected by input excitation, has low sensitivity and poor recognition effect. SUMMARY

[0004] In order to make up for the shortcomings of the prior art, the present application provides a track fastener intelligent inspection and maintenance system based on a robot and a maintenance method, which combines the use of laser radar positioning technology, laser ranging technology and machine vision image recognition technology to realize automatic detection of the state of track fasteners.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0006] A track fastener intelligent inspection and maintenance system based on a robot, characterized in that:

[0007] It comprises a self-walking platform, a positioning and detection system module, a control system module, a tightness subsystem module, a rust removal and oiling subsystem module, a safety protection system and an embedded data processing system.

[0008] The self-walking platform is provided with a laser radar, which is connected with the embedded data processing system, for collecting the contour size information and position information of the fasteners and surrounding objects, monitoring the vehicle body running information, and sending the contour size information, position information and vehicle body running information to the embedded data processing system.

[0009] The positioning detection system module comprises at least two state detection modules, the state detection module comprises a laser sensor, a visual camera and a light supplement lamp; the state detection module is arranged in pairs below the left and right sides of the front end of the vehicle body, corresponds to the fasteners on both sides of the steel rail respectively, and is connected with the embedded data processing system, for collecting state-related quantitative parameters of the fasteners, and transmitting the state-related quantitative parameters to the embedded data processing system;

[0010] The control system module is arranged above the vehicle body and connected with the embedded data processing system, for controlling the vehicle body to travel on the steel rail according to the control instruction of the embedded data processing system, controlling the state detection module to detect the state of the fastener to be detected, and controlling the tensioning subsystem module and the rust removal and oiling subsystem module to automatically maintain the fastener to be maintained according to the detection result.

[0011] Further, the tensioning subsystem module comprises an electric torque wrench and a torque angle sensor; the tensioning subsystem module is installed on both sides of the vehicle body through a telescopic device, the telescopic device controls the tensioning subsystem module to position the fastener to be detected according to the fastener positioning information and the control instruction, and performs the tensioning operation according to the control instruction.

[0012] Further, the rust removal and oiling subsystem module comprises a driving motor, a transmission shaft, a hard brush, a nozzle, an oil tank and an oil and gas circuit device;

[0013] The oil tank is arranged above the vehicle body, and the remaining devices of the rust removal and oiling subsystem module are installed on both sides of the vehicle body through telescopic devices, the telescopic devices control the tensioning subsystem module and the rust removal and oiling subsystem module to position the fastener to be detected according to the fastener positioning information and the control instruction, and perform the rust removal and oiling operation according to the control instruction.

[0014] Further, the rust removal and oiling subsystem module further comprises a heating plate, and the heating plate generates heat energy to dry the fastener after oiling.

[0015] A rail fastener intelligent inspection and maintenance method based on a robot, characterized in that it comprises the following steps:

[0016] Step S10: The embedded data processing system sends a motion control instruction to the control system module, controls the robot to travel on the steel rail, calibrates the position after the laser radar identifies and positions the first fastener, and transmits the fastener position information to the embedded data processing system;

[0017] Step S20: The embedded data processing system sends a motion control instruction to the control system module, controls the positioning detection system module to detect the state of the fastener to be detected according to the positioning information, collects quantized data related to the state of the fastener, and transmits the state data to the embedded data processing system for collection and processing, analyzes the unqualified fastener, and records the fastener state information in the database for storage;

[0018] Step S30: The embedded data processing system sends a motion control instruction to the control system module, controls the tensioning subsystem to perform tensioning operation on the pre-tightening force unqualified fastener calibrated, and records the maintenance operation information in the database for storage;

[0019] Step S40: The embedded data processing system sends a motion control instruction to the control system module, controls the rust removal and oiling subsystem to perform rust removal, oiling and drying operation on the surface state unqualified fastener calibrated, and records the maintenance operation information in the database for storage;

[0020] Step S50: After the above series of operations are completed, the control system sends a feedback signal to the embedded data processing system, and the fastener state inspection and maintenance operation is completed; the embedded data processing system sends a motion control instruction to the control system module, controls the robot to move forward on the rail, and continues to detect the next fastener, and the cycle is repeated.

[0021] Further, the embedded data processing system is used to run a multi-level point cloud feature learning network model to realize point cloud data clustering, feature extraction and point cloud data segmentation on the 3D point cloud data collected by the state detection module, obtain local features, segment the rail fastener region according to the local features, and obtain the fastener bolt and the fastener spring strip;

[0022] The multi-level point cloud feature learning network model is composed of a series of point set abstraction layers, and the point set abstraction layer includes an adoption layer, a grouping layer and a 3D visual target segmentation layer; wherein the input of the point set abstraction layer is N×(d+C); that is, N points, wherein each point has d-dimensional coordinates and C-dimensional point features, and the output of the point set abstraction layer is N'×(d+C'), d represents that the coordinate dimension is unchanged, and the specific steps are as follows:

[0023] The adoption layer adopts farthest point sampling, randomly selects a point from the point cloud data, then selects the farthest point from the point as the starting point, and performs continuous iteration until the required number of center points is selected, so as to select N1(<N) center points;

[0024] The grouping layer adopts a grouping method to find the nearby points of the center point, and combines the found nearby points to obtain a point set of a local region, so as to realize input of an N×(d+C) point set and selection of an N'×d center point set by the adoption layer, and output of an N'×K×(d+C) point set of a local region, wherein each point is provided with d-dimensional coordinates and C-dimensional point features, and K is the number of points in a neighborhood ball;

[0025] The 3D visual target segmentation layer converts the coordinates of the points in the K local regions into coordinates relative to the center point of the region according to the N'×K×(d+C) point set data of the local region, so as to obtain a local feature N'×(d+C'), and the rail fastener region is segmented according to the local feature, so as to obtain a fastener bolt and a fastener elastic strip, wherein C' is the length of the local feature.

[0026] Further, the vertical height difference between the lower plane of the fastener bolt and the upper plane of the fastener elastic strip is calculated, and the vertical height difference is compared with a standard value to determine whether the bolt is loose;

[0027] The Hough circle transformation algorithm is adopted to locate the coordinates of the lead-in and attack-in parts of the bolt, specifically: the Hough gradient method is adopted to traverse and accumulate all non-zero point corresponding circle centers, and the circle center is considered, and the number of intersection of the upper vector of the circle center and a threshold value is compared to determine the circle center of the lead-in and attack-in parts of the bolt.

[0028] The coordinates of the end of the robot arm executing the tightening and loosening bolt instruction of the tightening and loosening subsystem are aligned with the coordinates of the lead-in and attack-in parts of the bolt, so as to execute the tightening and loosening bolt instruction of the tightening and loosening subsystem, and the end of the robot arm can be a tightening wrench.

[0029] Further, the laser sensor collects images of the robot arm and the fastener in real time, extracts a light bar from the image collected by the laser sensor by using a color gamut conversion algorithm, and extracts light bar breakpoint pixel coordinates by using horizontal projection and vertical projection algorithms; the specific steps are as follows:

[0030] 1) Convert the original image from the RGB color gamut to the HSV color gamut

[0031] V = max(R, G, B)

[0032]

[0033]

[0034] 2) Extract the light bar according to the red HSV color gamut value

[0035] The red HSV color value is as follows:

[0036]

[0037] 3) Extracting light bar breakpoint pixel coordinates

[0038] The reason for the formation of the light bar breakpoint is the height difference between the upper edge of the track fastener fixing bolt and the lower edge of the track fastener fixing bolt tightening wrench, and the breakpoint coordinates are located by horizontal projection and vertical projection of the image after extracting the light bar, and the horizontal projection algorithm formula is as follows:

[0039]

[0040] The vertical projection algorithm calculation formula is as follows:

[0041]

[0042] Where (m, n) is the image size, p(i, y) and p(x, j) are the pixel values of (i, y) and (x, j) in the image respectively;

[0043] Set the pixel value threshold of horizontal projection and vertical projection as T; for vertical projection, traverse from left to right, when the vertical projection value h x ≥T, the current x is the horizontal coordinate of the breakpoint pixel coordinate; for horizontal projection, traverse from top to bottom, when the horizontal projection value v y <T, the current y-1 is the vertical coordinate of the breakpoint (lower edge of the track fastener fixing bolt tightening wrench) pixel coordinate, when the horizontal projection value v y ≥T, the current y is the vertical coordinate of the breakpoint (upper edge of the track fastener fixing bolt) pixel coordinate.

[0044] Further, the relationship between the camera coordinate system and the base coordinate system is calculated by the robot hand-eye calibration algorithm, and the relationship result is obtained, and the pose result is converted into the base coordinate according to the relationship result; specifically:

[0045] The corresponding relationship between the space point in the camera coordinate system and the point in the robot base coordinate system is as follows:

[0046]

[0047] Where (X, Y, Z) is the coordinate of the space point in the base coordinate system, (x, y, z) is the coordinate of the same space point in the camera coordinate system, R and T are the hand-eye transformation matrix;

[0048] The relationship between the camera coordinate system and the image coordinate system is as follows:

[0049]

[0050] Where (u, v) is the pixel coordinate, (x c , y c , z c ) is the camera coordinate.

[0051] The camera intrinsic matrix is:

[0052]

[0053] dx is the physical size of the pixel point in the axis direction of the world coordinate system, dy is the physical size of the pixel point in the axis direction of the world coordinate system, (u0, v0) is the origin of the image coordinate system;

[0054] The conversion relationship between the image coordinate system and the camera coordinate system is:

[0055]

[0056] f is the distance from the origin of the camera coordinate system to the image plane;

[0057] The above calibration is performed by using a 9-point calibration board to obtain the world coordinates of the center of the robot base coordinate system, so as to calculate the two hand-eye transformation matrices R and T.

[0058] Further, when the vertical height difference exceeds the standard value, it is judged that the bolt is loose, and then according to the bolt guide and attack part coordinates, the lower edge coordinates of the bolt tightening wrench and the correlation between the camera coordinate system and the robot base coordinate system, the embedded data processing system sends a motion control instruction to the control system module to control the tightening subsystem to perform tightening operation on the unqualified fastener.

[0059] The beneficial effects of the present application are:

[0060] 1) The track fastener intelligent inspection and maintenance system based on the robot of the present application realizes automatic detection of the state of the track fastener by combining the use of laser radar positioning technology, laser ranging technology and machine vision image recognition technology.

[0061] 2) The track fastener intelligent inspection and maintenance system based on the robot of the present application realizes intelligent tightening or loosening of the fastener bolt that is automatically inspected to be loose, and rust removal, oiling and drying operation on the fastener parts that are automatically inspected to have an unqualified surface state, by using motion control technology.

[0062] 3) The track fastener intelligent inspection and maintenance integrated method of the present application combines the use of database technology, human-computer interaction technology and deep learning algorithm to analyze and store the collected fastener state data, bolt torque value data and vehicle body running data, intelligently analyzes and determines the state of the fastener and the bolt, establishes a fastener maintenance ledger along the entire track, automatically generates a fastener inspection and maintenance operation process and result information statistical report, and realizes full automation and intelligentization of track fastener state detection and maintenance and repair. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1An architecture schematic diagram of a track fastener intelligent inspection and maintenance integrated robot in an embodiment of the present application;

[0064] Figure 2 A flowchart schematic diagram of a track fastener intelligent inspection and maintenance integrated method provided by the present application. DETAILED DESCRIPTION

[0065] The present application will be described in detail below with specific embodiments.

[0066] As shown in Figure 1 、 2 , a track fastener intelligent inspection and maintenance system based on a robot of the present application includes a self-walking platform, a positioning detection system module, a control system module, a tightness subsystem module, a rust removal and oiling subsystem module, a safety protection system, and an embedded data processing system;

[0067] The hardware content of the embedded data processing system includes a microprocessor module, an information processing module, a data storage module, a communication module, a power module, and a display module, which are used to receive detection data and robot motion state data collected by the detection system, determine state information of the fastener to be detected through a deep learning algorithm, and issue motion instructions to control the robot motion and operation;

[0068] The self-walking platform includes a vehicle body, a driving motor, a transmission unit, a laser radar, a storage battery, an auxiliary safety device, and a state detection module. The driving motor drives the vehicle body to realize bidirectional self-walking through the transmission unit under the control of the controller. The laser radar is arranged at the front end of the vehicle body. The self-walking platform further includes a reserved module mounting interface for module addition when the equipment is upgraded later. The self-walking platform further includes a parking mechanism for ensuring that the robot remains stationary on the steel rail without external force. The self-walking platform further includes a tool box for storing related operation tools by the inspection personnel;

[0069] The laser radar is arranged on the self-walking platform and connected with the embedded data processing system, for collecting contour size information and position information of the fastener and surrounding objects, monitoring vehicle body running information, and sending the contour size information, position information, and vehicle body running information to the embedded data processing system;

[0070] The positioning detection system module includes at least two state detection modules, and the state detection module includes a laser sensor, a vision camera, and a fill light. The state detection modules are arranged in pairs below the left and right sides of the front end of the vehicle body, correspond to the fasteners on both sides of the steel rail respectively, and are connected with the embedded data processing system, for collecting state-related quantitative parameters of the fasteners and sending the state-related quantitative parameters to the embedded data processing system;

[0071] The control system module is arranged above the vehicle body and is connected with the embedded data processing system, and is used for controlling the vehicle body to travel on the steel rail according to the control instruction of the embedded data processing system, controlling the state detection module to detect the state of the fastener to be detected, and controlling the tensioning subsystem module and the rust removal and oiling subsystem module to automatically maintain the fastener to be maintained according to the detection result.

[0072] The control system module comprises an automatic control device and a manual control device; in the automatic control state, the inspection personnel can control the robot to walk, detect and maintain through the handheld device in the remote control mode. A portable foldable handrail interface is arranged on the vehicle body, and is used for the inspection personnel to push the robot to realize bidirectional walking in the case that the device is not powered; the tensioning subsystem module is arranged in front of the rust removal and oiling subsystem module.

[0073] The safety protection system is arranged above the vehicle body and is connected with the embedded data processing system, and is used for receiving the motion limiting instruction of the embedded data processing system, limiting the motion of the robot, and monitoring the motion state of the robot in real time.

[0074] The tensioning subsystem module comprises an electric torque wrench and a torque angle sensor; the tensioning subsystem module is arranged on both sides of the vehicle body through the telescopic device, the telescopic device controls the tensioning subsystem module to position the fastener to be detected according to the fastener positioning information and the control instruction, and performs the tensioning operation according to the control instruction.

[0075] The rust removal and oiling subsystem module comprises a driving motor, a transmission shaft, a hard brush, a nozzle, an oil tank and an oil and gas circuit device; the oil tank is arranged above the vehicle body, and the remaining devices of the rust removal and oiling subsystem module are arranged on both sides of the vehicle body through the telescopic device, the telescopic device controls the tensioning subsystem module and the rust removal and oiling subsystem module to position the fastener to be detected according to the fastener positioning information and the control instruction, and performs the rust removal and oiling operation according to the control instruction. The rust removal and oiling subsystem module further comprises a heating plate, and the heating plate generates heat to dry the fastener that has been oiled.

[0076] A rail fastener intelligent inspection and maintenance method based on a robot, comprising the following steps:

[0077] Step S10: The embedded data processing system sends a motion control instruction to the control system module, controls the robot to travel on the steel rail, the laser radar performs position calibration after identifying and positioning the first fastener, and transmits the fastener position information to the embedded data processing system.

[0078] Step S20: The embedded data processing system sends a motion control instruction to the control system module, controls the positioning detection system module to detect the state of the fastener to be detected according to the positioning information, collects quantized data related to the state of the fastener, and transmits the state data to the embedded data processing system for collection and processing, analyzes the unqualified fastener, and records the fastener state information in the database for storage;

[0079] Step S30: The embedded data processing system sends a motion control instruction to the control system module, controls the tensioning subsystem to perform tensioning operation on the pre-tightening force unqualified fastener calibrated, and records the maintenance operation information in the database for storage;

[0080] Step S40: The embedded data processing system sends a motion control instruction to the control system module, controls the rust removal and oiling subsystem to perform rust removal, oiling and drying operation on the surface state unqualified fastener calibrated, and records the maintenance operation information in the database for storage;

[0081] Step S50: After the above series of operations are completed, the control system sends a feedback signal to the embedded data processing system, and the fastener state inspection and maintenance operation is completed; the embedded data processing system sends a motion control instruction to the control system module, controls the robot to move forward on the rail, and continues to detect the next fastener, and the cycle is repeated.

[0082] Specifically, the embedded data processing system is used to run a multi-level point cloud feature learning network model to realize point cloud data clustering, feature extraction and point cloud data segmentation on the fastener 3D point cloud data collected by the state detection module, obtain local features, and segment the rail fastener region according to the local features to obtain the fastener bolt and the fastener spring strip;

[0083] The multi-level point cloud feature learning network model is composed of a series of point set abstraction layers, and the point set abstraction layer includes a selection layer, a grouping layer and a 3D visual target segmentation layer; wherein the input of the point set abstraction layer is N×(d+C); that is, N points, each point has d-dimensional coordinates and C-dimensional point features, and the output of the point set abstraction layer is N'×(d+C'), d represents that the coordinate dimension is unchanged, and the specific steps are as follows:

[0084] The selection layer selects the farthest point sampling, randomly selects a point from the point cloud data, then selects the farthest point from the point as the starting point, and iterates continuously until the required number of center points is selected, so as to select N1(<N) center points;

[0085] The grouping layer adopts a ball to find the neighboring points of the center point by using a grouping method, and combines the found neighboring points to obtain a point set of a local region, so as to realize inputting an N x (d+C) point set and an N' x d center point set selected by the layer, and obtaining an output N' x K x (d+C) point set of the local region, wherein each point has a d-dimensional coordinate and a C-dimensional point feature, and K is the number of points in the neighboring ball;

[0086] The 3D vision target segmentation layer converts the coordinates of the points in the K local regions into coordinates relative to the center point of the region according to the N' x K x (d+C) point set data of the local region, so as to obtain a local feature N' x (d+C'), and according to the local feature, the rail fastener region is segmented to obtain a fastener bolt and a fastener spring strip, wherein C' is the length of the local feature.

[0087] Specifically, by calculating the vertical height difference between the lower plane of the fastener bolt and the upper plane of the fastener spring strip, and comparing the vertical height difference with a standard value, it is judged whether the bolt is loose or not;

[0088] The Hough circle transformation algorithm is used to locate the coordinates of the lead-in and attack-in parts of the bolt, specifically: the Hough gradient method is used to traverse and accumulate all non-zero point corresponding circle centers, and the number of intersection of the upper vector of the circle center is compared with a threshold value to judge the circle center of the lead-in and attack-in parts of the bolt.

[0089] The coordinates of the end of the robot arm executing the tightening and loosening subsystem bolt instruction are aligned with the coordinates of the lead-in and attack-in parts of the bolt, so as to execute the tightening and loosening subsystem bolt instruction, and the end of the robot arm can be a tightening wrench.

[0090] Specifically, the laser sensor collects images of the robot arm and the fastener in real time, extracts the light bar from the image collected by the laser sensor through a color domain conversion algorithm, and then extracts the light bar breakpoint pixel coordinates by using horizontal projection and vertical projection algorithms; the specific steps are as follows:

[0091] 1) Convert the original image from RGB color domain to HSV color domain

[0092] V = max (R, G, B)

[0093]

[0094]

[0095] 2) Extract the light bar according to the red HSV color domain value

[0096] The red HSV color value is as follows:

[0097]

[0098] 3) Extract the light bar breakpoint pixel coordinates

[0099] The light bar breakpoint is caused by the height difference between the upper edge of the track fastener fixing bolt and the lower edge of the track fastener fixing bolt fastening wrench, and the breakpoint coordinates are located by horizontal projection and vertical projection of the image after extracting the light bar, and the horizontal projection algorithm formula is as follows:

[0100]

[0101] The vertical projection algorithm calculation formula is as follows:

[0102]

[0103] Where (m, n) is the image size, p(i, y) and p(x, j) are the pixel values of (i, y) and (x, j) in the image respectively;

[0104] The pixel value threshold of horizontal projection and vertical projection is set as T; for vertical projection, when the vertical projection value h x ≥T, the current x is the horizontal coordinate of the breakpoint pixel coordinate; for horizontal projection, when the horizontal projection value v y <T, the current y-1 is the vertical coordinate of the breakpoint (the lower edge of the track fastener fixing bolt fastening wrench); when the horizontal projection value v y ≥T, the current y is the vertical coordinate of the breakpoint (the upper edge of the track fastener fixing bolt).

[0105] Specifically, the correlation between the camera coordinate system and the base coordinate system is calculated by the robot hand-eye calibration algorithm, and the correlation result is obtained, and the pose result is converted into the base coordinate according to the correlation result; specifically:

[0106] The corresponding relationship between the space point in the camera coordinate system and the point in the robot base coordinate system is as follows:

[0107]

[0108] Where (X, Y, Z) is the coordinate of the space point in the base coordinate system, (x, y, z) is the coordinate of the same space point in the camera coordinate system, R and T are the hand-eye transformation matrix;

[0109] The relationship between the camera coordinate system and the image coordinate system is as follows:

[0110]

[0111] Where (u, v) is the pixel coordinate, (x c , y c , z c ) is the camera coordinate;

[0112] The camera intrinsic matrix is:

[0113]

[0114] dx is the physical size of the pixel point in the axial direction of the world coordinate system, dy is the physical size of the pixel point in the axial direction of the world coordinate system, (u0, v0) is the origin of the image coordinate system;

[0115] The conversion relationship between the image coordinate system and the camera coordinate system is:

[0116]

[0117] f is the distance from the origin of the camera coordinate system to the image plane;

[0118] The above calibration is performed by using a 9-point calibration board to obtain the world coordinates of the center of the robot base coordinate system, so as to calculate the two hand-eye transformation matrices R and T.

[0119] When the vertical height difference exceeds the standard value, it is judged that the bolt is loose, then according to the coordinates of the bolt leading-in and attacking-in part, the coordinates of the lower edge of the bolt tightening wrench and the correlation relationship between the camera coordinate system and the robot base coordinate system, the embedded data processing system sends a motion control instruction to the control system module, and the loose-tight subsystem controls the loose-tight operation of the unqualified fastener subjected to calibration.

[0120] The content of the application is not limited to the examples listed, and any equivalent transformation of the technical solutions of the application made by a person skilled in the art by reading the specification of the application is covered by the claims of the application.

Claims

1. A maintenance method for a robot-based intelligent inspection and maintenance system for track fasteners, characterized in that: Includes the following steps: Step S10: The embedded data processing system sends motion control commands to the control system module to control the robot to walk on the rail. After the lidar identifies and locates the first fastener, it performs position calibration and transmits the fastener position information to the embedded data processing system. Step S20: The embedded data processing system sends motion control commands to the control system module, which controls the positioning and detection system module to perform status detection on the fastener under inspection based on the positioning information, collects quantitative data related to the fastener status, transmits the status data to the embedded data processing system for collection and processing, analyzes and calibrates unqualified fasteners, and records the fastener status information into the database for storage. Step S30: The embedded data processing system sends motion control commands to the control system module to control the tensioning subsystem to perform tensioning operations on fasteners with unqualified preload, and records the maintenance operation information into the database for storage. Step S40: The embedded data processing system sends motion control commands to the control system module to control the rust removal and oiling subsystem to perform rust removal, oiling and drying operations on the fasteners with unqualified surface conditions, and to record and store the maintenance operation information in the database. Step S50: After the above series of operations are completed, the control system sends a feedback signal to the embedded data processing system, and the fastener status inspection and maintenance operation is completed; the embedded data processing system sends a motion control command to the control system module to control the robot to move forward on the rail and continue to inspect the next fastener, and so on. The intelligent inspection and maintenance system for track fasteners includes a self-propelled platform, a positioning and detection system module, a control system module, a tensioning subsystem module, a rust removal and oiling subsystem module, a safety protection system, and an embedded data processing system. The self-propelled platform is equipped with a lidar, which is connected to the embedded data processing system. This lidar is used to collect the outline dimensions and position information of the fasteners and surrounding objects, monitor the vehicle's operating information, and send the outline dimensions, position information, and vehicle operating information to the embedded data processing system. The positioning detection system module includes at least two status detection modules, each including a laser sensor, a vision camera, and a supplementary light. The status detection modules are arranged in pairs on the lower left and right sides of the front end of the vehicle body, corresponding to the fasteners on both sides of the rail, and connected to the embedded data processing system. They are used to collect quantitative parameters related to the status of the fasteners and send the quantitative parameters related to the status to the embedded data processing system. The control system module is located on top of the vehicle body and connected to the embedded data processing system. It is used to control the vehicle body to move on the rail according to the control instructions of the embedded data processing system, control the status detection module to perform status detection on the fasteners to be inspected, and control the tensioning subsystem module and the rust removal and oiling subsystem module to perform automatic maintenance operations on the fasteners that need maintenance based on the detection results.

2. The maintenance method of the robot-based intelligent inspection and maintenance system for track fasteners according to claim 1, characterized in that: The tensioning subsystem module includes an electric torque wrench and a torque angle sensor. The tensioning subsystem module is installed on both sides of the vehicle body via a telescopic device. The telescopic device controls the tensioning subsystem module to position the fastener to be tested based on the fastener positioning information and control commands, and performs tensioning operations according to the control commands.

3. The maintenance method of the robot-based intelligent inspection and maintenance system for track fasteners according to claim 1, characterized in that: The rust removal and oiling subsystem module includes a drive motor, a transmission shaft, a hard brush, a nozzle, an oil tank, and oil and air circuit equipment. The oil tank is located on the top of the vehicle body. The remaining devices of the rust removal and oiling subsystem module are installed on both sides of the vehicle body via telescopic devices. The telescopic devices control the tensioning subsystem module and the rust removal and oiling subsystem module to position the fasteners to be inspected according to the fastener positioning information and control commands, and perform rust removal and oiling operations according to the control commands.

4. The maintenance method of the robot-based intelligent inspection and maintenance system for track fasteners according to claim 1, characterized in that: The rust removal and oiling subsystem module also includes a heating plate, which generates heat to dry the oiled fasteners.

5. The intelligent inspection and maintenance method for track fasteners based on robots according to claim 4, characterized in that: The embedded data processing system is used to run a multi-level point cloud feature learning network model to perform point cloud data clustering, feature extraction, and point cloud data segmentation on the 3D point cloud data of the fastener collected by the state detection module to obtain local features. Based on the local features, the track fastener area is segmented to obtain fastener bolts and fastener springs. The multi-layer point cloud feature learning network model consists of a series of point set abstraction layers, including an scalar layer, a grouping layer, and a 3D visual object segmentation layer. The input to each point set abstraction layer is N×(d+C), representing N points, where each point has d-dimensional coordinates and C-dimensional point features. The output of the point set abstraction layer is N'×(d+C'), where d represents the coordinate dimension remaining constant. The specific steps are as follows: The layer employs farthest point sampling, randomly selecting a point from the point cloud data, and then selecting the point farthest from that point as the starting point, performing continuous iterations until the required number of center points is selected, thereby selecting N1 (< N ) center points; The grouping layer uses a sphere grouping method to find the nearest points of the center point, and combines the found nearest points to obtain a point set of the local region. Thus, it can input an N×(d+C) point set and the N'×d center point set selected by the grouping layer to obtain an output N'×K×(d+C) point set of the local region, where each point has d-dimensional coordinates and C-dimensional point features, and K is the number of points in the neighboring spheres. The 3D visual target segmentation layer converts the coordinates of K points within a local region into coordinates relative to the center point of that region based on the N'×K×(d+C) point set data of the local region, thereby obtaining local features N'×(d+C'). Based on the local features, the track fastener region is segmented to obtain fastener bolts and fastener springs, where C' is the length of the local feature.

6. The intelligent inspection and maintenance method for track fasteners based on robots according to claim 5, characterized in that: By calculating the vertical height difference between the lower plane of the fastener bolt and the upper plane of the fastener spring strip, and comparing the vertical height difference with the standard value, it can be determined whether the bolt is loose. The coordinates of the bolt's entry and penetration points are located using the Hough circle transform algorithm. Specifically, the Hough gradient method is used to traverse and accumulate the center of all non-zero points, and the center of the circle is considered. The number of intersections of the modulus vectors on the center of the circle is compared with a threshold to determine the center of the bolt's entry and penetration points. The coordinates of the end of the robotic arm that executes the bolt tightening command of the tightening subsystem are aligned with the coordinates of the bolt's insertion and tapping parts in order to execute the bolt tightening command of the tightening subsystem. The end of the robotic arm can be a tightening wrench for tightening bolts.

7. A robot-based intelligent inspection and maintenance method for track fasteners according to claim 6, characterized in that: The laser sensor acquires images of the robotic arm and fasteners in real time. A color gamut conversion algorithm is used to extract light stripes from the images acquired by the laser sensor. Then, horizontal and vertical projection algorithms are used to extract the pixel coordinates of the light stripe breakpoints. The specific steps are as follows: 1) Convert the original image from the RGB color gamut to the HSV color gamut. 2) Extract light bars based on red HSV color gamut values The red HSV color values ​​are as follows: 3) Extract the pixel coordinates of the light stripe breakpoint. The breakpoint in the light stripe is caused by the height difference between the upper edge of the track fastener fixing bolt and the lower edge of the track fastener fixing bolt tightening wrench. The breakpoint coordinates are located by performing horizontal and vertical projections on the image after extracting the light stripe. The formula for the horizontal projection algorithm is as follows: The formula for calculating vertical projection is as follows: Where (m, n) is the image size, and p(i, y) and p(x, j) are the pixel values ​​of (i, y) and (x, j) in the image, respectively; Set the pixel value thresholds for horizontal and vertical projections to T; for vertical projection, traverse from left to right, and when the vertical projection value h... x If x ≥ T, then the current x is the x-coordinate of the breakpoint pixel; for horizontal projection, traverse from top to bottom, when the horizontal projection value v y < T The current y-1 is the ordinate of the pixel coordinate of the breakpoint (the lower edge of the track fastener fixing bolt tightening wrench). When the horizontal projection value v y ≥T, where y is the ordinate of the pixel coordinate of the breakpoint (the upper edge of the track fastener fixing bolt).

8. The intelligent inspection and maintenance method for track fasteners based on robots according to claim 7, characterized in that: The robot's hand-eye calibration algorithm is used to calculate the correlation between the camera coordinate system and the base coordinate system, obtaining the correlation result. The pose result is then converted to the base coordinate system based on the correlation result. Specifically: The correspondence between points in the camera coordinate system and points in the robot's base coordinate system is shown below: Where (X,Y,Z) are the coordinates of a point in the base coordinate system, (x,y,z) are the coordinates of the same point in the camera coordinate system, and R and T are the hand-eye transformation matrices; The relationship between the camera coordinate system and the image coordinate system is shown below: Where (u, v) are pixel coordinates, (x) c y c , z c () represents the camera coordinates; The camera intrinsic parameter matrix is: dx is the physical dimension of the pixel in the world coordinate system along the central axis, dy is the physical dimension of the pixel in the world coordinate system along the central axis, and (u0, v0) is the origin of the image coordinate system. The transformation relationship between the image coordinate system and the camera coordinate system is as follows: f is the distance from the origin of the camera coordinate system to the image plane; The above calibration was performed using a 9-point calibration board to obtain the world coordinates of the center of the robot's base coordinate system, thereby calculating the two hand-eye transformation matrices R and T.

9. A robot-based intelligent inspection and maintenance method for track fasteners according to claim 8, characterized in that: When the vertical height difference exceeds the standard value, the bolt is judged to be loose; then, based on the coordinates of the bolt insertion and insertion part, the coordinates of the lower edge of the bolt tightening wrench, and the correlation between the camera coordinate system and the robot base coordinate system, the embedded data processing system sends motion control commands to the control system module to control the tightening subsystem to tighten or loosen the calibrated unqualified fasteners.

Images