Unmanned railway ballast inspection robot for train coupler state recognition

Abstract

The invention discloses a railway ballast unmanned inspection robot for train coupler state recognition, and can be applied to the technical field of railway automatic detection. According to the invention, the satellite navigation receiving antenna in the railway ballast unmanned inspection robot receives the satellite positioning data of the railway ballast unmanned inspection robot, and the inertial measurement unit detects the position attitude data of the railway ballast unmanned inspection robot in real time; after a first camera collects a real-time sky image of the position where the ballast unmanned inspection robot is located in real time, a processing unit carries out position analysis according to the satellite positioning data, the position posture data and the real-time sky image to obtain the target position of the ballast unmanned inspection robot, and the inspection positioning precision of the inspection robot can be effectively improved; and after the real-time coupler image of the front train is collected in real time through the second camera, the coupler connection state information of the front train is obtained through analysis of the processing unit, and the train coupler state recognition accuracy and recognition efficiency of the inspection robot can be improved.

Description

Technical Field

[0001] This application relates to the field of railway automated inspection technology, and in particular to an unmanned ballast inspection robot for recognizing the condition of train couplers. Background Technology

[0002] In related technologies, train couplers are key components connecting carriages, and their connection status directly affects the safety and reliability of train operation. However, current coupler status inspections mainly rely on manual visual inspection, which is not only inefficient but also poses safety hazards. Furthermore, various connection states of the coupler and locking pins (locked, returned, etc.) require manual visual identification. In ballast environments, due to confined space and worker fatigue, missed or false inspections frequently occur. Moreover, some ballast tracks are located in harsh environments such as canyons, where side-by-side carriages create a canyon effect, leading to poor visibility of traditional GNSS satellites and decreased positioning accuracy. Additionally, ballast tracks are unstructured surfaces, making low-cost inertial navigation systems (INS) prone to drift over time, hindering the long-term stable output of high-precision position information. Simultaneously, the lack of ground features in canyon environments reduces the positioning performance of conventional laser radar (SLAM), while wireless signal-based positioning requires extensive base station deployment, which is costly and susceptible to electromagnetic interference. It can be seen that the positioning accuracy of the inspection robots in the existing technology is poor and the accuracy and efficiency of the coupler identification results are low, which reduces the efficiency of checking the coupler status in the ballast area before the train departs and the safety after the train departs.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to propose an unmanned ballast inspection robot for train coupler status recognition, which can effectively improve the inspection and positioning accuracy of the inspection robot, and improve the accuracy and efficiency of the inspection robot in recognizing the status of train couplers.

[0005] To achieve the above objectives, this application proposes an unmanned ballast inspection robot for train coupler status identification, the unmanned ballast inspection robot comprising: Mobile chassis; The vehicle body is mounted above the mobile chassis; A satellite navigation receiving antenna is disposed above the mobile chassis and is used to receive satellite positioning data of the ballast unmanned inspection robot. An inertial measurement unit, which is installed inside the vehicle body, is used to detect the position and attitude data of the unmanned ballast inspection robot in real time. The first camera is located above the vehicle body and is used to collect real-time sky images of the location of the unmanned ballast inspection robot. The second camera is located at the front end of the vehicle body and is used to capture real-time images of the couplers of the train ahead. A radar unit is installed on the top of the vehicle body and is used to collect obstacle information in real time in the corresponding direction of movement of the ballast unmanned inspection robot. The processing unit is configured to perform position analysis based on the satellite positioning data, the position and attitude data, and the real-time sky image to obtain the target position of the unmanned ballast inspection robot; analyze the real-time coupler image to obtain the coupler connection status information of the train ahead; and control the movement state of the unmanned ballast inspection robot based on the obstacle information. The communication unit connects the processing unit to the monitoring platform, which is used to display the target position, position and attitude data, real-time coupler image, or coupler connection status information of the ballast unmanned inspection robot in real time.

[0006] In some embodiments, obtaining the target position of the ballast unmanned inspection robot by performing position analysis based on the satellite positioning data, the position and attitude data, and the real-time sky image includes: The satellite positioning data and the position and attitude data are fused to obtain corrected position data; The real-time sky image is processed to obtain non-direct line-of-sight position data; The target position of the ballast unmanned inspection robot is obtained by adjusting the corrected position data using the non-direct line-of-sight position data.

[0007] In some embodiments, fusing the satellite positioning data and the position and attitude data to obtain corrected position data includes: Construct yaw angle constraints based on the position and attitude data; In the position measurement phase, a lever correction is introduced, and the satellite positioning data is corrected based on the yaw angle constraint to obtain the corrected position data.

[0008] In some embodiments, processing the real-time sky image to obtain non-line-of-sight distance position data includes: The real-time sky image is preprocessed by an alternating on-off sequential filter to obtain a preprocessed sky image. The preprocessed sky image is clustered using a clustering algorithm to obtain candidate sky region seed points; The candidate sky region seed points are filtered to obtain the cluster center sky region; An occlusion interference analysis is performed on the sky region of the cluster center to obtain the non-direct line-of-sight location data.

[0009] In some embodiments, the step of filtering the candidate sky region seed points to obtain the cluster center sky region includes: Calculate the mean of the grayscale values ​​of the regions corresponding to the candidate sky region seed points and the sky grayscale reference values; Calculate the weighted average deviation between the grayscale values ​​of the regions corresponding to the candidate sky region seed points and the sky grayscale reference values; The cluster center sky region is determined based on the ratio of the mean to the weighted average deviation.

[0010] In some embodiments, obtaining the coupler connection status information of the train ahead based on the real-time coupler image analysis includes: The real-time coupler image is subjected to a second preprocessing to obtain a preprocessed coupler image. The second preprocessing includes Laplacian sharpening and processing based on the lighting system. The preprocessed coupler image is subjected to bidirectional principal component analysis to obtain a dimension-reduced feature image; The dimensionality-reduced feature image is subjected to hybrid feature extraction using a self-attention convolutional neural network model to obtain hybrid features; The hybrid features are input into a preset neural network model to identify the vehicle connection status, thereby obtaining the coupler connection status information of the train ahead.

[0011] In some embodiments, the second preprocessing of the real-time coupler image to obtain a preprocessed coupler image includes: Calculate the average grayscale value of the real-time coupler image to quantize the illumination value; Calculate the illumination adaptive weight matrix based on the quantized illumination value according to the average gray value; Edge enhancement is performed on the real-time coupler image based on the illumination adaptive weight matrix; The illumination level of the real-time coupler image is determined by quantizing the illumination value based on the average gray value. The structural unit size of the real-time coupler image after edge enhancement is adjusted according to the illumination level; The real-time coupler image after structural unit size adjustment is processed by closing operation to obtain the preprocessed coupler image.

[0012] In some embodiments, performing bidirectional principal component analysis on the preprocessed coupler image to obtain a dimensionality-reduced feature image includes: Calculate the first covariance matrix along the row direction of the preprocessed coupler image, and extract the first principal components of the preprocessed coupler image based on the first covariance matrix to form a principal component image; The second covariance matrix is ​​calculated along the column direction of the principal component image, and the second principal components of the principal component image are extracted based on the second covariance matrix to form the dimensionality-reduced feature image.

[0013] In some embodiments, the step of extracting mixed features from the dimensionality-reduced feature image using a self-attention convolutional neural network model to obtain mixed features includes: Local texture features in the reduced-dimensional feature image are extracted using depthwise separable convolution in the self-attention convolutional neural network model. The global features are obtained by extracting the global spatial relationship between the locking pin and the hook tongue in the dimensionality-reduced feature image through the self-attention encoder in the self-attention convolutional neural network model; The hybrid feature is obtained by integrating the local texture features and the global features through element-wise addition.

[0014] In some embodiments, the step of inputting the hybrid features into a preset neural network model for vehicle connection status recognition to obtain the coupler connection status information of the train ahead includes: The mixed features are weighted by the joint attention mechanism in the preset neural network model to obtain weighted features. The weighted features are input into the classification layer of the preset neural network model to obtain the coupler connection status information of the train ahead.

[0015] The embodiments of this application include at least the following beneficial effects: This application provides an unmanned ballast inspection robot for train coupler status identification. This solution, by setting up a mobile chassis, vehicle body, satellite navigation receiving antenna, inertial measurement unit, first camera, second camera, radar unit, and processing unit in the unmanned ballast inspection robot, receives satellite positioning data from the unmanned ballast inspection robot via the satellite navigation receiving antenna, detects the position and attitude data of the unmanned ballast inspection robot in real time via the inertial measurement unit, and collects real-time sky images of the robot's location via the first camera. The processing unit then performs position analysis based on the satellite positioning data, position and attitude data, and real-time sky images to obtain the target position of the unmanned ballast inspection robot. This effectively reduces the impact of poor satellite signal or inertial navigation signal on positioning data, improving the efficiency of the inspection robot. The robot improves the accuracy of human inspection and positioning. Furthermore, by using a second camera to capture real-time images of the train's couplers, the processing unit analyzes these images to determine the coupler connection status, enhancing the robot's accuracy and efficiency in identifying train couplers. Simultaneously, the radar unit collects real-time obstacle information along the robot's movement direction, and the processing unit controls the robot's movement based on this information, improving its safety during movement. Additionally, the robot connects to a monitoring platform via a communication unit, allowing real-time display of its target location, position and attitude data, real-time coupler images, and coupler connection status information. This enables staff to access coupler-related data and robot-related data promptly. Attached Figure Description

[0016] Figure 1 This is a structural schematic diagram of the unmanned ballast inspection robot for train coupler status recognition provided in the embodiments of this application from a first-view perspective. Figure 2 This is a structural schematic diagram of the unmanned ballast inspection robot for train coupler status recognition provided in the embodiments of this application from a second perspective. Figure 3 This is a schematic diagram of the visual display interface provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0018] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0019] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] In related technologies, train couplers are key components connecting carriages, and their connection status directly affects the safety and reliability of train operation. However, current coupler status inspections mainly rely on manual visual inspection, which is not only inefficient but also poses safety hazards. Furthermore, various connection states of the coupler and locking pins (locked, returned, etc.) require manual visual identification. In ballast environments, due to confined space and worker fatigue, missed or false inspections frequently occur. Moreover, some ballast tracks are located in harsh environments such as canyons, where side-by-side carriages create a canyon effect, leading to poor visibility of traditional GNSS satellites and decreased positioning accuracy. Additionally, ballast tracks are unstructured surfaces, making low-cost inertial navigation systems (INS) prone to drift over time, hindering the long-term stable output of high-precision position information. Simultaneously, the lack of ground features in canyon environments reduces the positioning performance of conventional laser radar (SLAM), while wireless signal-based positioning requires extensive base station deployment, which is costly and susceptible to electromagnetic interference. It can be seen that the positioning accuracy of the inspection robots in the existing technology is poor and the accuracy and efficiency of the coupler identification results are low, which reduces the efficiency of checking the coupler status in the ballast area before the train departs and the safety after the train departs.

[0022] In view of this, this application provides an unmanned ballast inspection robot for train coupler status recognition, which can effectively improve the inspection positioning accuracy of the inspection robot and improve the accuracy and efficiency of the inspection robot in recognizing the status of train couplers.

[0023] The embodiments of this application will be described in detail below with reference to the accompanying drawings: Reference Figure 1 and Figure 2This application provides an unmanned ballast inspection robot for train coupler status identification. The unmanned ballast inspection robot includes a mobile chassis 1, a vehicle body 10, a satellite navigation receiving antenna 2, an inertial measurement unit 3, a first camera 4, a second camera 5, a radar unit 6, a processing unit 7, and a communication unit 9. The system comprises: a vehicle body mounted on top of a mobile chassis; a satellite navigation receiving antenna mounted on top of the mobile chassis for receiving satellite positioning data from the unmanned ballast inspection robot; an inertial measurement unit located inside the vehicle body for real-time detection of the robot's position and attitude data; a first camera mounted on top of the vehicle body for real-time acquisition of sky images of the robot's location; a second camera mounted at the front of the vehicle body for real-time acquisition of images of the train couplers ahead; a radar unit mounted on top of the vehicle body for real-time acquisition of obstacle information in the robot's direction of movement; a processing unit for performing position analysis based on satellite positioning data, position and attitude data, and real-time sky images to obtain the robot's target position; analyzing the real-time coupler images to obtain the coupler connection status information of the train ahead; controlling the robot's movement based on obstacle information; and connecting to a monitoring platform via a communication unit so that the monitoring platform can display the robot's target position, position and attitude data, real-time coupler images, or coupler connection status information in real-time.

[0024] It is understood that the mobile chassis in this embodiment can adopt an electric track or a four-wheel structure, which can be selected according to the actual track and ballast environment. The robot is also equipped with shock absorption and guide rail structures to adapt to side-walking scenarios. The satellite navigation receiving antenna can be a Global Navigation Satellite System (GNSS) receiving antenna. The first camera can be a fisheye camera, which can acquire 360° sky images. By segmenting the sky image into sky / non-sky regions, it can identify satellites in non-line-of-sight (NLOS) conditions and reduce the weight of the satellite's observation signal to improve GNSS single-point positioning accuracy. The second camera is a forward-looking camera, installed at the front of the vehicle body, used to capture images of the train couplers ahead. The radar unit can be a two-dimensional lidar, installed at the front or top of the vehicle body, used to detect obstacles ahead and perform collision avoidance. The processing unit can be a central processing unit, which can be connected to various modules within the robot via cables, responsible for data processing and command issuance. Furthermore, a power supply unit 8 can be installed within the robot in this embodiment, which can supply power to the mobile platform and various modules within the robot. The communication unit is used to upload the robot's inspection results (including positioning information and coupler status) to a ground monitoring platform or visualization platform for real-time display. For example... Figure 3As shown, the display interface of the ground monitoring platform or visualization platform can display the coupler status, robot positioning information, real-time path map, coupler anomaly rate statistics, historical data detection, etc. in real time.

[0025] It is understood that the second camera in this embodiment can detect the coupler status of the train ahead from a 45° side angle, breaking through the limitations of the undercarriage view detection, eliminating blind spots, and achieving full coverage of the locking pin without any blind spots. This embodiment solves the problem of blind spots caused by bogie obstruction by completely covering the entire locking pin from a side-view perspective; it fully exposes the locking pin's reset state (the depth of the locking pin embedded in the coupler tongue groove); and it clearly captures the locking characteristics of the coupler tongue (the gap size between the coupler tongue and the locking pin contact surface).

[0026] It is understandable that, in this embodiment, when obtaining the target position of the ballast unmanned inspection robot based on satellite positioning data, position and attitude data, and real-time sky imagery, the corrected position data can be obtained by fusing the satellite positioning data and position and attitude data. After processing the real-time sky imagery to obtain non-line-of-sight position data, the corrected position data is adjusted using the non-line-of-sight position data to obtain the target position of the ballast unmanned inspection robot. The process of fusing satellite positioning data and position and attitude data to obtain the corrected position data in this embodiment can involve constructing a yaw angle constraint based on the position and attitude data, then introducing lever correction during the position measurement stage, and finally correcting the satellite positioning data based on the yaw angle constraint to obtain the corrected position data.

[0027] Specifically, this embodiment employs a vision-assisted GNSS / INS integrated navigation method. The output signal of the inertial measurement unit (IMU) is first fused using a Kalman filter to obtain the inertial navigation solution; the GNSS receiver receives satellite signals to obtain the absolute positioning result, and the two are fused using an extended Kalman filter to calculate the robot's pose on the track in real time. When the robot travels into a canyon area where the number of visible GNSS satellites decreases, this embodiment introduces vehicle kinematic constraints to stabilize the inertial navigation algorithm: using a 2DOF vehicle model, and utilizing the yaw rate and vehicle speed measured by the IMU, translational and yaw angle integral models are established respectively to compensate for the inertial navigation output, thereby extending the stabilization time of the pose calculation and improving the continuity of the inertial navigation output.

[0028] Simultaneously, this embodiment utilizes a fisheye camera to periodically capture sky images and performs sky / non-sky region segmentation on the images. For this segmentation process, this embodiment combines edge detection, Flood Fill, and K-Means++ clustering algorithms for initial image preprocessing to obtain a preprocessed sky image. Then, based on the clustering algorithm, the preprocessed sky image is clustered to extract candidate sky region seed points. These candidate sky region seed points are then filtered to obtain the cluster center sky region. Occlusion interference analysis is performed on the cluster center sky region to obtain non-line-of-sight (NLOS) position data. By analyzing the sky segmentation results, this embodiment can identify which GNSS satellites are obstructed by buildings or train carriages (i.e., non-line-of-sight satellites) and reduce the weight of their observation signals in the Kalman filter. This sky-map-assisted GNSS positioning technology effectively eliminates NLOS satellite observations with large errors, improving the accuracy of single-point positioning.

[0029] During the combined positioning process, the system also introduces visual odometry constraints: continuous frame images are acquired by a fisheye camera, the relative pose change between two adjacent frames is calculated, and this is used as an observation to add to the Kalman filter, thereby optimizing the fusion result of inertial navigation and GNSS, further suppressing inertial navigation drift and smoothing the positioning trajectory. This enables centimeter-level continuous positioning accuracy in complex environments such as ballast track canyons, meeting the requirements of coupler status recognition for real-time high-precision positioning.

[0030] It is understood that in this embodiment, when the system built based on the inertial detection unit performs continuous and uninterrupted position estimation, velocity, position, and attitude can be updated through INS mechanical orchestration. Specifically, the process of updating velocity, position, and attitude during INS mechanical orchestration can be represented by the following formula: ; For speed updates, by Time's up Executable steps for updating the temporal bisample hypothesis rate: ; ; ; ; ; As shown in the above equation, to complete the velocity update at time K based on the bisample assumption, it is necessary to input the velocities of the b-frame relative to the n-frame at times K-1 and K-2 (i.e., the velocity in the navigation state), the attitude of the b-frame relative to the n-frame at time K-1 (i.e., the attitude in the navigation state), and the position of the b-frame relative to the e-frame at the previous time (i.e., the geodetic coordinates, used to calculate the rotational angular velocity of the n-frame relative to the e-frame). Therefore, velocity updates must be completed before attitude and position updates.

[0031] The change of position over time can be described by the following set of differential equations: ; ; ; In the formula, , These are the meridian radius and the tropospheric radius of the carrier's location, respectively. The carrier's location can be represented by latitude. ,longitude and ellipsoid height express. , and For the carrier velocity vector in It consists of projection components in the three directions of north, east, and ground.

[0032] From this set of differential equations, we can see that updating latitude and longitude... Elevation is needed, and latitude is needed to update longitude; therefore, the position coordinates are discretized and updated in the order of elevation, latitude, and longitude. Solving the differential equation for elevation yields the following formula: ; Assuming within the integration period As the equation changes linearly with time, the integral of the above equation simplifies to the trapezoidal integral of the following formula: ; In the formula, and They represent and Vertical velocity at any given time. Simplification during the integration period when updating latitude. Let the elevation h be a constant, and let the elevation h equal the average elevation over the integration period. This yields the following formula: ; In the formula, Indicates using Calculating the radius of the meridian at a given time position ; .

[0033] Similarly, update longitude The process is as follows: ; In the formula, This indicates that the latitude is used for calculation at the intermediate time. , .

[0034] For the attitude update process, this embodiment uses the attitude quaternion method. Specifically, the attitude update process can be represented by the following formula: ; In the formula, Indicates the current state to be solved The attitude quaternion of the system, Representing the navigation coordinate system Tied to arrive The attitude change quaternion at time t, Represents the carrier coordinate system From arrive The inverse change of attitude at any given moment Represents the carrier coordinate system The attitude quaternion of time.

[0035] To improve pose estimation accuracy, this embodiment decouples yaw and roll motions. Specifically, this embodiment uses the IMU angular velocity and the velocity of the 2DOF vehicle model. Construct six-DOF pose constraints and use the angular velocities of the 2DOF vehicle model. Establish yaw angle constraints. Data synchronization is achieved based on linear interpolation and the IMU frequency.

[0036] Specifically, this embodiment combines a traditional kinematic model with a data-driven model. The traditional 2DOF vehicle model is used to describe the basic motion characteristics of the vehicle, while the data-driven model corrects model errors by learning from historical motion data. Based on the above model, this embodiment constructs the following formula for hybrid vehicle dynamics-kinematic constraints: ; In the formula, The angular velocity output by the hybrid model. The angular velocity output by a traditional 2DOF vehicle model. The angular velocity predicted by the LSTM network. These are adaptive weighting coefficients. The system dynamically adjusts based on the vehicle's motion status; when the vehicle is moving smoothly... The values ​​are relatively large, and traditional models are the main approach; however, when the vehicle performs complex movements... The values ​​are relatively small, and the model is mainly driven by data.

[0037] The six-degree-of-freedom pose integral model is as follows: ; ; ; In the formula, They represent Translation in the vehicle coordinate system at any given moment; This is the rotation matrix from the IMU coordinate system to the vehicle coordinate system, and its expression is as follows: ; In the formula, Let represent the rotation matrix obtained at time t using the 2DOF vehicle model in vehicle coordinates.

[0038] Construct the kinematic residuals between the system states of frame k and k+1. Its expression is as follows: ; In the formula, q represents the rotation quaternion, used to characterize the rotational attitude in three-dimensional space; p represents the position vector, used to characterize the coordinate position of the object in three-dimensional space; G represents the global coordinate system; L represents the lidar coordinate system; K is the adaptive adjustment coefficient, which is related to the vehicle's steering angle and the number of feature points extracted from the point cloud. The specific calculation process is as follows: ; In the formula, It indicates the instantaneous steering angle of the vehicle, reflecting the current degree of curvature of the vehicle's trajectory. This represents the maximum instantaneous steering angle among all vehicles; This indicates the number of feature points extracted from the current point cloud; This represents the maximum number of feature points extracted across all time periods. Indicates the number of successfully matched associated feature points; This represents the maximum number of successfully matched associated feature points across all time periods.

[0039] This embodiment applies the Discrete Kalman Filter (DKF) algorithm to implement GNSS / INS loose combination solutions. Within one filtering cycle, the Kalman filter is divided into a time update (state prediction) and a measurement update process. Specifically, this embodiment performs one-step prediction of the state and state variance using the following formula: ; ; In the formula, This represents the state vector of a one-step prediction system. This represents the filtered estimate from the previous time step. This represents the predicted value of the system covariance at the current moment. This represents the system covariance filter estimate from the previous time step.

[0040] Then, the measurement is updated, first by calculating the gain matrix. Then update the system status. and systematic error covariance This embodiment directly provides the formulas for estimating the system state variables and the system covariance: ; ; ; The above formula is the standard Kalman filter update formula. As long as the initial value is given... and According to any time Measurement The state estimate at the corresponding time can then be derived. .

[0041] To apply Kalman filtering to this embodiment, it needs to be improved to become a tool capable of performing or approximating nonlinear filtering. Therefore, this embodiment employs an extended Kalman filter with error feedback to address the nonlinearity issue in the integrated navigation system.

[0042] To obtain the system state equations, we first need to write out the error state... The continuous-time differential equation is given by the following formula: ; Using Taylor expansion down to the first-order terms, assume the objective equation is as follows: ; The system's one-step prediction state transition matrix can be obtained from the first-order Taylor expansion as follows: ; The second-order matrix (covariance matrix) is given by the following formula: ; In the short integral interval Inside It can be simplified to the trapezoidal integral of the following formula: ; Since the sequence is indeed white noise, the equivalent discrete system described by the equation satisfies the requirements of standard Kalman filtering. The Kalman filter update can then be implemented simply by providing the observation equation.

[0043] Before deriving the position observation equation, it's crucial to clarify a physical fact: GNSS positioning provides the position coordinates and velocity of the antenna phase center (or a reference point with pre-set corrections on the receiver), while the position obtained from inertial navigation system (INS) mechanical arrangement reflects the navigation state (position, velocity, attitude) of the IMU measurement center (the origin of the IMU coordinate system). These two are physically distinct. Therefore, lever correction is necessary during data fusion. The formula for correcting the INS position (N-system position) to the GNSS antenna phase center is: ; in This means converting the differences in northward, eastward, and vertical positions under the N-system into differences in latitude, longitude, and elevation components. Represented as: ; Ignore position error Due to the influence of the matrix, the attitude error definition is rewritten as follows: ; ; The two formulas can be rearranged to obtain the following formula: ; Therefore, the GNSS observation equation can be expressed by the following formula: ; In the formula, This represents the observation error, and its standard deviation can be output by the receiver. The matrix representation is as follows: ; Since inertial navigation sampling epochs are usually not synchronized with satellite navigation sampling epochs, inertial navigation... Health Guide Taking the sampling rate as an example, the maximum position error that can occur over a time span of 0.1 seconds is possible. When the carrier moves at low speeds but with significant changes in relative attitude amplitude, this error will further amplify the impact of gyroscope noise. Therefore, time alignment is also required in the algorithm design. This embodiment addresses this issue by extrapolating the inertial navigation-calculated position to the GNSS update epoch. This strategy ensures that the filtering update process does not use epoch data that has not actually been obtained. In addition to being applicable to loosely combined post-processing, it can also be used in real-time processing, demonstrating good algorithm robustness.

[0044] In practical applications, it has been found that due to the non-negligible system modeling and high-order approximation truncation errors, position updates have limited effectiveness in estimating and compensating for the velocity and attitude errors of the integrated navigation system. Therefore, this embodiment introduces velocity updates to achieve error estimation of INS velocity and attitude. Specifically, the observation matrix for velocity updates is as follows: ; ; Furthermore, this embodiment introduces particle filtering to handle nonlinear motion states. When the robot performs complex movements on the ballast track, particle filtering can better handle the nonlinear characteristics of the system and improve positioning accuracy. By adding an LSTM network to predict and compensate for sensor drift, the LSTM network utilizes its memory function to learn the patterns of sensor drift and predict and compensate for INS drift errors. Simultaneously, this embodiment uses a fisheye camera to acquire sky images synchronized with the GNSS signal, effectively capturing and reflecting the environment around the GNSS antenna and the geometric distribution of surrounding buildings. To detect NLOS signals, it is necessary to segment the sky and non-sky regions of the image.

[0045] Specifically, the sky image segmentation process in this embodiment can be achieved by combining edge detection, the FloodFill algorithm, and the K-Means++ clustering algorithm. Since strong sunlight and clouds may be misidentified as obstructions due to weather conditions, affecting the image segmentation results, smoothing and simplifying the image preprocessing is crucial. This embodiment uses an alternating sequential filter (ASF) to process the image. The image output using ASF has less noise than the original image and better preserves the edges of the sky and non-sky regions.

[0046] In this context, ASF performs opening and closing operations alternately in an incremental manner, and its expression is as follows: ; In the formula, This indicates that an m-order alternating opening and closing sequential filter is composed of a series of different structure elements (from...). arrive The mathematical morphology operations γ are performed alternately and sequentially using opening and closing operations. Here, γ represents the structuring element. In mathematical morphology operations, it is typically a small matrix or template used to probe or alter the shape and structure of an image. The subscript m indicates a series of operations using structuring elements of different sizes or shapes.

[0047] In this embodiment, the preprocessed sky image is converted into a grayscale image, and then the Canny edge detection algorithm is used to obtain an edge mask.

[0048] Specifically, this embodiment uses the K-Means++ clustering algorithm to cluster the preprocessed RGB image, obtaining a clustered binary image. Connectivity analysis is performed on the clustered binary image, and the centroids of the connected components are used as seed points for candidate sky regions. A lightweight U-Net segmentation network is added. This lightweight U-Net segmentation network takes a fisheye image as input and outputs a probability mask for the sky region. Through training, the network can learn the features of sky and non-sky regions.

[0049] Under complex weather conditions, deep networks are used as lightweight U-Net segmentation networks for segmentation processing, which can better cope with the degradation of image quality under complex weather conditions such as fog, haze, and rain, and improve the accuracy of image segmentation.

[0050] Specifically, the process of processing sky images using deep networks is as follows: ; In the formula, For the final weather mask, For traditional sky masks, The sky mask obtained by the deep network. To optimize weighting, the weights are dynamically adjusted based on weather conditions, under normal weather conditions. Large values, complex weather The value is relatively small.

[0051] Under normal weather conditions, this embodiment divides the sky image into sky and non-sky regions according to the segmentation requirements. Therefore, the number of cluster centers K is set to 2, and a pixel is randomly selected from the image as the first cluster center. The distance D between each pixel in the image and the nearest cluster center is calculated. After that, the probability of each pixel being selected as a cluster center is calculated using the following formula. And select the next cluster center based on the highest probability.

[0052] ; Repeat the above steps until K cluster centers are selected. Then calculate the distance D between each pixel and the K cluster centers. Each pixel is assigned to the class containing the nearest cluster center, and then the sum of squared errors (SSE) for each class is calculated using the following formula: ; If the SSE approaches stability, the clustering result is obtained; otherwise, the steps are repeated until the SSE approaches stability. The clustering results of K-Means++ are represented using a binary image. Then, connected component statistics are performed on the binary image, the centroids of the connected components are labeled, and the area of ​​each connected component is calculated. Set a threshold for the area of ​​connected components. ,like > Then the centroid of the connected region is used as the seed point for the candidate sky region.

[0053] It is understood that, in this embodiment, when screening candidate sky region seed points, the cluster center sky region can be determined by calculating the mean of the gray level of the image corresponding to the candidate sky region seed point and the gray level of the sky reference value, and by calculating the weighted average deviation of the gray level of the image corresponding to the candidate sky region seed point and the gray level of the sky reference value, and then determining the cluster center sky region based on the ratio of the mean and the weighted average deviation.

[0054] Specifically, the screening process in this embodiment uses preset evaluation rules. These preset evaluation rules include: The grayscale reference value of the entire image S and the sky is calculated using the following formula. Mean deviation D: ; A circle C, centered at the seed point (x, y) and with radius R, is used as the evaluation region for that seed point. The region C is then compared with the evaluation region using the following formula. Weighted average deviation M: ; In the formula, This represents the number of pixels in region C with a grayscale value of i.

[0055] Set the brightness threshold for the sky area. The luminance coefficient is calculated using the following formula. (That is, the ratio of the mean to the weighted average deviation): ; like If the cluster center is located in the sky region, then the sky region of the cluster center can be obtained.

[0056] As can be seen from the above, this embodiment can effectively improve the positioning accuracy and stability of GNSS / INS fusion by introducing vehicle kinematic constraints and sky image assistance technology during the positioning process.

[0057] The aforementioned vision-assisted GNSS / INS integrated navigation technology effectively mitigates the effects of poor signal visibility, sensor drift, and environmental occlusion, enabling continuous high-precision positioning of the robot in complex conditions such as ballast track canyons. Specifically, the positioning results from the forward channel guide visual recognition of the Region of Interest (ROI). Based on the robot's position and attitude obtained from GNSS / INS fusion positioning, combined with the train's position information, the approximate position of the coupler in the image is calculated, thereby determining the ROI for visual recognition, reducing interference from irrelevant areas, and improving recognition efficiency. The coupler recognition results from the feedback channel correct positioning deviations. After the coupler is identified, the robot's positioning results are corrected based on the deviation between the actual position of the coupler in the image and the predicted position based on the positioning results.

[0058] Joint optimization objective function: ; In the formula, J is the joint optimization objective function. This is the positioning error function, used to measure the accuracy of the positioning result. The Euclidean distance between the positioning location and the actual location can be used. To identify the error function, which is used to measure the accuracy of coupler status identification, indicators such as classification error rate can be used; and This is a weighting coefficient, adjusted according to the importance of positioning and identification. During the critical stage of coupler status identification, it can be increased. The weight of [the parameter] can be increased during the critical stages of robot navigation. The weights are determined. By minimizing the joint optimization objective function, coordinated optimization of localization and identification is achieved.

[0059] To verify the effectiveness of the method proposed in this application, a series of experiments were conducted. The experiments were carried out in an actual ballast track environment, selecting different time periods and weather conditions, including sunny days, cloudy days, and foggy days, to comprehensively test the system's performance under various environments.

[0060] The system's positioning accuracy was tested in both static and dynamic scenarios. In the static scenario, the robot was fixed in a specific position, and positioning data was continuously collected and compared with the actual position measured by a high-precision total station. In the dynamic scenario, the robot was allowed to travel along a track, and the positioning results were recorded and compared with the actual position on the track. The results show that after adopting multi-model collaborative GNSS / INS fusion positioning, the positioning accuracy can be improved by more than 50% in static scenarios and by more than 45% in dynamic scenarios, with an average positioning accuracy on the order of 8 cm.

[0061] It is understood that in this embodiment, after acquiring a real-time image of the coupler of the train ahead, the coupler connection status information is analyzed based on the real-time coupler image. Specifically, the analysis process in this embodiment may involve performing a second preprocessing on the real-time coupler image to obtain a preprocessed coupler image, then performing bidirectional principal component analysis on the preprocessed coupler image to obtain a dimensionality-reduced feature image, then using a self-attention convolutional neural network model to extract mixed features from the dimensionality-reduced feature image to obtain mixed features, and finally inputting the mixed features into a preset neural network model to identify the vehicle connection status and obtain the coupler connection status information of the train ahead. The second preprocessing includes Laplacian sharpening and processing based on the lighting system. Specifically, the second preprocessing process in this embodiment may involve quantizing the illumination value by calculating the average gray value of the real-time coupler image, calculating the illumination adaptive weight matrix based on the quantized illumination value, performing edge enhancement on the real-time coupler image based on the illumination adaptive weight matrix, determining the illumination level of the real-time coupler image based on the quantized illumination value, adjusting the structural unit size of the edge-enhanced real-time coupler image based on the illumination level, and then performing a closing operation on the real-time coupler image with the adjusted structural unit size to obtain the preprocessed coupler image.

[0062] In this embodiment, to address the issue of image quality fluctuations in couplers caused by complex lighting conditions in ballast environments (strong light, shadows, fog, etc.), a dynamic preprocessing module based on illumination intensity is designed to adaptively adjust the weights of the Laplacian sharpening convolution kernel and the size of the morphological structural unit. Specifically, the average grayscale value quantizes the illumination value. The calculation process is as follows: ; In the formula, The grayscale value of the coupler image. This refers to the image size.

[0063] The process of classifying light intensity levels can be done using the following formula: Low light: ; Normal lighting: ; Strong sunlight: .

[0064] This embodiment designs an adaptive weighting matrix for illumination based on nonlinear adjustment of illumination. The calculation formula is as follows: ; In the formula, the illumination coefficient is adaptively weighted. The calculation formula is as follows: ; In the formula, This indicates the low light threshold, and its value can be 50 (grayscale value). This represents the strong light threshold or high light threshold, and its value can be 200 (grayscale value). These two thresholds are used to calculate the average grayscale value of the image. The lighting is divided into three levels (low light, normal light, and strong light), which is the core basis for the condition judgment of the entire dynamic preprocessing module.

[0065] During this process, sharpening is enhanced in low light conditions, standard sharpening is performed in normal light conditions, and sharpening is reduced in strong light conditions to avoid excessive noise.

[0066] In this embodiment, after calculating the average grayscale value of the real-time coupler image G and quantizing the illumination value, the corresponding illumination coefficient adaptive weights are matched, and then edge enhancement is achieved through convolution operation: In this embodiment, a 3x3 convolution kernel can be used. By using morphological closing operations, a 5*5 rectangular structuring element is used to dilate and then erode the real-time coupler image to connect the broken edges, thereby achieving edge enhancement.

[0067] To determine the edge fracture strength under different lighting conditions, this embodiment dynamically adjusts the rectangular structural unit size S using the following formula: ; The formula for closure operation is as follows: ; In the formula, Represents structural elements or morphological structural units; This represents the preprocessed image of the coupler. This represents the image after edge enhancement.

[0068] This embodiment uses the Laplacian operator to sharpen the coupler image and performs morphological closing operations to extract edge features of key parts of the coupler and remove background interference. It is understood that in this embodiment, when obtaining the preprocessed coupler image, bidirectional principal component analysis is performed on the preprocessed coupler image to obtain a dimensionality-reduced feature image, thereby reducing the data dimensionality and highlighting important features. Specifically, this embodiment can calculate a first covariance matrix along the row direction of the preprocessed coupler image, and extract the first principal components of the preprocessed coupler image based on the first covariance matrix to form a principal component image; calculate a second covariance matrix along the column direction of the principal component image, and extract the second principal components of the principal component image based on the second covariance matrix to form a dimensionality-reduced feature image. This embodiment calculates the covariance matrix and extracts principal components along the row direction of the image, and then repeats the operation along the column direction, compressing the 500×500 pixel image to a 128×128 feature map as the dimensionality-reduced feature image through double dimensionality reduction, thereby reducing the number of model parameters by more than 90%.

[0069] It is understood that, after dimensionality reduction, this embodiment uses a self-attention convolutional neural network model to extract hybrid features from the dimensionality-reduced feature image to obtain hybrid features. Specifically, this embodiment can extract local texture features from the dimensionality-reduced feature image through depthwise separable convolution in the self-attention convolutional neural network model; extract global features from the global spatial relationship between the locking pin and the hook in the dimensionality-reduced feature image through the self-attention encoder in the self-attention convolutional neural network model; and then integrate the local texture features and global features by element-wise addition to obtain hybrid features. In this embodiment, the dimensionality-reduced feature image is divided into N=16 non-important overlapping image blocks ((8*8) pixels / block), and sequence features are obtained through linear projection. Then enter the Transformer encoder: ; ; The MultiHeadAttention module models the global positional relationship between the locking pin and the latch (the spatial offset of the locking pin relative to the latch) and outputs global features. (Mapping sequence features to local texture features using an egg roll machine) Features of the same size, Then, local texture features and global features are integrated by adding them element by element: ; In the formula, This indicates mixed features.

[0070] Understandably, in this embodiment, after obtaining the mixed features, the mixed features are input into a preset neural network model to identify the vehicle connection status and obtain the coupler connection status information of the train ahead. Specifically, this embodiment can use the joint attention mechanism in the preset neural network model to weight the mixed features to obtain weighted features, and then input the weighted features into the classification layer of the preset neural network model to classify and obtain the coupler connection status information of the train ahead. The preset neural network model includes two convolutional layers (using 32 and 64 3×3 convolutional kernels respectively), two max-pooling layers (2×2 pooling windows), and two fully connected layers, employing the ReLU activation function and cross-entropy loss function, achieving a coupler status information identification accuracy of 97.56%.

[0071] In this embodiment, a joint attention mechanism is designed to highlight key areas (locking pin edges, hook contact points). This joint attention mechanism includes channel attention and spatial attention. Channel attention is applied to mixed features. The channel dimensions are globally averaged, and the channel weights are output through the MLP. : ; Spatial attention is a feature of mixed features The channel dimensions are globally averaged, and the channel weights are output through the MLP. : ; Then, the weighted features are obtained by weighting the features corresponding to channel attention and the features corresponding to spatial attention: ; In the formula, This indicates a weighted feature.

[0072] Then weighted features The input is fed into the classification layer for classification and recognition to obtain the coupler connection status information: ; Among them, FC is a fully connected layer, and Focal loss is used to solve the class imbalance problem: ; In the formula, Indicates a label; Label prediction probability; Indicates weight; This represents a coefficient, which can be 2.

[0073] As described above, in this embodiment, the dimensionality-reduced feature vectors are input into a pre-trained convolutional neural network model, which uses the CNN to classify and identify whether the coupler tongue is locked and whether the locking pin is in place. This method fully combines the advantages of 2DPCA in feature extraction and dimensionality reduction with the powerful image classification capabilities of CNN, achieving an integrated output that quickly locates the damaged area and accurately identifies the state type.

[0074] Therefore, the method of this application solves the two key problems of positioning accuracy and coupler recognition, realizing the function of continuous high-precision positioning and efficient status recognition of inspection robots in complex ballast environments. Experiments show that the method of this embodiment improves positioning accuracy by up to 42.62% in static scenes and 41.24% in dynamic scenes compared to traditional solutions, with an average positioning accuracy on the order of 10cm. The accuracy of coupler status recognition in this embodiment reaches 97.56%. By collecting a large number of coupler images in different states, including normal state, hook tongue locked, and locking pin not returned to position, a hybrid vision architecture based on Transformer-CNN was used for recognition testing. The results show that the method of this embodiment achieves a coupler status recognition accuracy of 99.6%, an improvement of 2.04 percentage points compared to the original CNN model. Furthermore, under extreme lighting conditions, the dynamic adaptive preprocessing system improves the recognition accuracy by more than 30%. Furthermore, tests conducted under different weather conditions on sky segmentation and NLOS identification performance show that the dual-path segmentation network in this embodiment improves sky segmentation accuracy to 98.7% and reduces the NLOS satellite misjudgment rate by 60%, effectively improving the reliability of GNSS positioning.

[0075] Therefore, the method of this application embodiment significantly improves the automation and accuracy of coupler clamping status inspection, greatly reduces the labor cost and safety risks of manual inspection, and is of great significance for ensuring the safe operation of railway freight and promoting the automation of railway maintenance.

[0076] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0077] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0078] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0080] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0081] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A ballast unmanned inspection robot for recognizing the condition of train couplers, characterized in that, The ballast unmanned inspection robot includes: Mobile chassis; The vehicle body is mounted above the mobile chassis; A satellite navigation receiving antenna is disposed above the mobile chassis and is used to receive satellite positioning data of the ballast unmanned inspection robot. An inertial measurement unit, which is installed inside the vehicle body, is used to detect the position and attitude data of the unmanned ballast inspection robot in real time. The first camera is located above the vehicle body and is used to collect real-time sky images of the location of the unmanned ballast inspection robot. The second camera is located at the front end of the vehicle body and is used to capture real-time images of the couplers of the train ahead. A radar unit is installed on the top of the vehicle body and is used to collect obstacle information in real time in the corresponding direction of movement of the ballast unmanned inspection robot. The processing unit is configured to perform position analysis based on the satellite positioning data, the position and attitude data, and the real-time sky image to obtain the target position of the unmanned ballast inspection robot; analyze the real-time coupler image to obtain the coupler connection status information of the train ahead; and control the movement state of the unmanned ballast inspection robot based on the obstacle information. The communication unit connects the processing unit to the monitoring platform, which is used to display the target position, position and attitude data, real-time coupler image, or coupler connection status information of the ballast unmanned inspection robot in real time.

2. The unmanned ballast inspection robot according to claim 1, characterized in that, The step of obtaining the target position of the unmanned ballast inspection robot by performing position analysis based on the satellite positioning data, the position and attitude data, and the real-time sky image includes: The satellite positioning data and the position and attitude data are fused to obtain corrected position data; The real-time sky image is processed to obtain non-direct line-of-sight position data; The target position of the ballast unmanned inspection robot is obtained by adjusting the corrected position data using the non-direct line-of-sight position data.

3. The unmanned ballast inspection robot according to claim 2, characterized in that, The step of fusing the satellite positioning data and the position and attitude data to obtain corrected position data includes: Construct yaw angle constraints based on the position and attitude data; In the position measurement phase, a lever correction is introduced, and the satellite positioning data is corrected based on the yaw angle constraint to obtain the corrected position data.

4. The unmanned ballast inspection robot according to claim 2, characterized in that, The process of processing the real-time sky image to obtain non-line-of-sight distance position data includes: The real-time sky image is preprocessed by an alternating on-off sequential filter to obtain a preprocessed sky image. The preprocessed sky image is clustered using a clustering algorithm to obtain candidate sky region seed points; The candidate sky region seed points are filtered to obtain the cluster center sky region; An occlusion interference analysis is performed on the sky region of the cluster center to obtain the non-direct line-of-sight location data.

5. The unmanned ballast inspection robot according to claim 4, characterized in that, The step of filtering the candidate sky region seed points to obtain the cluster center sky region includes: Calculate the mean of the grayscale values ​​of the regions corresponding to the candidate sky region seed points and the sky grayscale reference values; Calculate the weighted average deviation between the grayscale values ​​of the regions corresponding to the candidate sky region seed points and the sky grayscale reference values; The cluster center sky region is determined based on the ratio of the mean to the weighted average deviation.

6. The unmanned ballast inspection robot according to claim 1, characterized in that, The step of obtaining the coupler connection status information of the train ahead based on the real-time coupler image analysis includes: The real-time coupler image is subjected to a second preprocessing to obtain a preprocessed coupler image. The second preprocessing includes Laplacian sharpening and processing based on the lighting system. The preprocessed coupler image is subjected to bidirectional principal component analysis to obtain a dimension-reduced feature image; The dimensionality-reduced feature image is subjected to hybrid feature extraction using a self-attention convolutional neural network model to obtain hybrid features; The hybrid features are input into a preset neural network model to identify the vehicle connection status, thereby obtaining the coupler connection status information of the train ahead.

7. The unmanned ballast inspection robot according to claim 6, characterized in that, The second preprocessing of the real-time coupler image to obtain a preprocessed coupler image includes: Calculate the average grayscale value of the real-time coupler image to quantize the illumination value; Calculate the illumination adaptive weight matrix based on the quantized illumination value according to the average gray value; Edge enhancement is performed on the real-time coupler image based on the illumination adaptive weight matrix; The illumination level of the real-time coupler image is determined by quantizing the illumination value based on the average gray value. The structural unit size of the real-time coupler image after edge enhancement is adjusted according to the illumination level; The real-time coupler image after structural unit size adjustment is processed by closing operation to obtain the preprocessed coupler image.

8. The unmanned ballast inspection robot according to claim 6, characterized in that, The step of performing bidirectional principal component analysis on the preprocessed coupler image to obtain a dimensionality-reduced feature image includes: Calculate the first covariance matrix along the row direction of the preprocessed coupler image, and extract the first principal components of the preprocessed coupler image based on the first covariance matrix to form a principal component image; The second covariance matrix is ​​calculated along the column direction of the principal component image, and the second principal components of the principal component image are extracted based on the second covariance matrix to form the dimensionality-reduced feature image.

9. The unmanned ballast inspection robot according to claim 6, characterized in that, The step of extracting mixed features from the dimensionality-reduced feature image using a self-attention convolutional neural network model to obtain mixed features includes: Local texture features in the reduced-dimensional feature image are extracted using depthwise separable convolution in the self-attention convolutional neural network model. The global features are obtained by extracting the global spatial relationship between the locking pin and the hook tongue in the dimensionality-reduced feature image through the self-attention encoder in the self-attention convolutional neural network model; The hybrid feature is obtained by integrating the local texture features and the global features through element-wise addition.

10. The unmanned ballast inspection robot according to claim 6, characterized in that, The step of inputting the hybrid features into a preset neural network model for vehicle connection status recognition to obtain the coupler connection status information of the train ahead includes: The mixed features are weighted by the joint attention mechanism in the preset neural network model to obtain weighted features. The weighted features are input into the classification layer of the preset neural network model to obtain the coupler connection status information of the train ahead.

Images