Rail defect detection method, system and equipment based on mobile robot and medium

Abstract

The invention relates to a rail defect detection method, system and equipment based on a mobile robot and a medium, and the method comprises the steps: constructing a normal rail image data set, extracting a multi-scale feature map based on a pre-trained RepViT model and a PaDiM algorithm, building a rail normal state statistical model, exporting the pre-trained RepViT model into an ONNX format, constructing a rail defect inference engine, and carrying out the inference of the rail defect. When a rail defect inference engine is deployed to a new road section, parameter fine adjustment based on an incremental learning mechanism is carried out, the rail defect inference engine is integrated into an operating system of a mobile robot, real-time rail images are collected, abnormal scores are calculated, a double-threshold active rechecking mechanism is designed, and the mobile robot carries out defect detection result judgment and autonomous rechecking and returns defect information. And a detection closed loop is formed. According to the invention, the problems of strong dependency on defect samples, poor environmental adaptability and difficulty in deployment on a mobile platform in the prior art are effectively solved, and efficient, real-time and automatic detection of rail defects under complex terrains is realized.

Description

Technical Field

[0001] This invention relates to the intersection of robotics technology and intelligent operation and maintenance of rail transit, and particularly to a method, system, equipment and medium for detecting rail defects based on mobile robots. Background Technology

[0002] Currently, rail defect detection mainly relies on manual inspection, large specialized inspection vehicles, and fixed monitoring equipment. These methods have several prominent bottlenecks in practical applications.

[0003] First, in terms of efficiency and cost, manual inspection is inefficient, requires a large workforce, and the results are easily affected by environmental conditions and the subjective state of the personnel. Although large inspection vehicles can improve efficiency to some extent, their purchase and maintenance costs are high, and they are difficult to operate in complex terrain sections such as tunnels, bridges, and mountains, resulting in significant blind spots. Fixed monitoring equipment, due to its fixed deployment location, cannot achieve flexible and routine coverage of the entire line and has poor adaptability.

[0004] Secondly, existing detection techniques have significant limitations. Algorithms based on traditional image processing are weakly robust to changes in lighting, rain, snow, and background interference, resulting in a high false alarm rate and hindering their practical application. While supervised learning-based deep learning models have improved detection performance, they rely on a large number of precisely labeled defect samples for training. However, railway defects are typically low-probability events in actual operation, resulting in scarce defect samples and high labeling costs. This leads to insufficient generalization ability of the models when faced with unseen defect types, making it difficult to adapt to the varied environments and defect morphologies encountered in real-world applications.

[0005] Furthermore, with the development of visual models, although large-scale models such as Vision Transformer (ViT) have powerful feature extraction capabilities, their high computational complexity poses a severe challenge to embedded devices on mobile platforms, making it difficult to meet the computational requirements of real-time detection and limiting their application in lightweight deployment scenarios such as mobile robots.

[0006] In summary, existing rail defect detection technologies have shortcomings in terms of detection efficiency, environmental adaptability, sample dependence, and mobile platform deployment capabilities. It is necessary to provide an all-weather automated detection solution that can adapt to complex terrain, reduce dependence on labeled samples, have high detection accuracy, and enable real-time processing on mobile platforms. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method, system, equipment, and medium for detecting rail defects based on mobile robots, which solves the limitations of existing rail defect detection technologies in terms of detection efficiency, environmental adaptability, sample dependence, and mobile platform deployment capabilities.

[0008] To achieve the above technical objectives, the present invention provides the following technical solution: a rail defect detection method based on a mobile robot, comprising the following steps: Original railway track images and video streams from multiple scenarios were collected by a mobile robot. After preliminary inspection and preprocessing, regions of interest were extracted and their sizes were normalized to obtain normal railway track images and construct a normal railway track image dataset. Normal railway track images are input into a pre-trained RepViT model to extract multi-scale feature maps; The multi-scale feature maps include Stage2 feature maps, Stage3 feature maps, and Stage4 feature maps, which are extracted from Stage2, Stage3, and Stage4 of the RepViT model backbone network, respectively. Based on the PaDiM algorithm, the extracted multi-scale feature maps are upsampled to a uniform size and then stitched along the channel dimension to form an enhanced feature vector. The mean vector and covariance matrix of its multivariate Gaussian distribution are calculated to establish a statistical model of the normal state of the railway track. Export the pre-trained RepViT model to ONNX format to build a rail defect inference engine to adapt to edge device deployment; When the rail defect inference engine is deployed to a new section, the parameters of the rail defect inference engine are automatically fine-tuned based on the incremental learning mechanism. After the parameter fine-tuning, the statistical model of the normal state of the rail in the new section scenario is reconstructed based on the PaDiM algorithm. The rail defect inference engine with fine-tuned parameters is integrated into the operating system of the mobile robot to create a defect detection node. The defect detection node receives real-time rail images, calls a preprocessing callback function to preprocess the real-time rail images, and then sends them to the rail defect inference engine to extract enhanced feature vectors. Based on the statistical model of normal rail conditions, the Mahalanobis distance of the enhanced feature vectors is calculated as the anomaly score, and then an anomaly heatmap is generated. Design a dual-threshold active review mechanism, setting an early warning threshold and a confirmation threshold, and use a mobile robot to judge the defect detection results; When the anomaly score of a real-time railway track image frame is lower than the warning threshold, it is directly judged as defect-free; When the anomaly score of a real-time railway track image exceeds the confirmation threshold, it is directly determined as a defect and an alarm process is immediately triggered. When the anomaly score of a real-time rail image frame is between the warning threshold and the confirmation threshold, it enters the suspected defect state, triggers the review process, back-projects the maximum value point of the anomaly heatmap to determine the estimated defect location, the mobile robot plans the path and adjusts its pose to align the camera optical center with the estimated defect location, and re-acquires real-time rail images from at least two perspectives, obtains the anomaly scores of these real-time rail images and calculates the average value. If the average value is higher than the confirmation threshold, it is finally determined to be a defect; otherwise, it is determined to be without defect and exits the suspected defect state. After receiving the defect detection results, the mobile robot transmits the defect information back, completing the detection process.

[0009] Optionally, the preliminary inspection and preprocessing, extracting the region of interest and normalizing its size to obtain a normal railway track image, includes: The original railway track images and video streams are initially checked for the number, format, and integrity of the images, and corrupted files caused by equipment failure or transmission errors are removed to construct the original railway track image dataset. Image hashing algorithm, homomorphic filtering algorithm, and median filtering algorithm are applied to preprocess the original railway track image dataset for deduplication, illumination correction, and noise reduction, respectively, to generate preprocessed railway track images; The linear region of the railway track is extracted as the region of interest from the preprocessed railway track image using a method based on edge detection and Hough transform. The size of the region of interest was normalized using a bilinear interpolation algorithm and adjusted to 640×640 to form a normal railway track image.

[0010] Optionally, the step of using a method based on edge detection and Hough transform to extract the linear region of the railway track as the region of interest from the preprocessed railway track image includes: The preprocessed railway track image is converted to grayscale and Gaussian blurring is performed to suppress noise, resulting in a Gaussian railway track grayscale image. Edge detection is performed on the Gaussian rail grayscale image using the Canny operator, and then the straight line segments are detected and located using the Hough transform. The minimum bounding rectangle is calculated and constructed based on the detected and located line segments, and this rectangular region is taken as the region of interest.

[0011] Optionally, exporting the pre-trained RepViT model to ONNX format includes: The export process uses INT8 quantization.

[0012] Optionally, the INT8 quantization includes: A calibration set is constructed by randomly selecting representative railway track images from normal samples of the new scene, and quantization parameters are generated. The new scene refers to a scene that was not sampled when constructing a normal railway track image dataset; The normal sample refers to a railway track image collected in the new scenario that has been confirmed to contain no defects or abnormal features after manual review or algorithm verification. The representative railway track image is one that can cover the sources of external variation in the new scene; The quantization parameters include scaling factor and zero point; Based on the quantization parameters, the RepViT model is mapped from FP32 precision type to INT8 precision type.

[0013] Optionally, the automatic parameter fine-tuning of the rail defect inference engine based on an incremental learning mechanism when the rail defect inference engine is deployed to a new track section includes: Collect railway track images of the new section and clean and verify their normality. Construct a dataset of normal railway track images of the new section and divide it into a training set and a validation set for the new section. Incremental training of the rail defect inference engine was performed using the new track section training set. An early stop strategy is adopted. After each incremental training cycle, the performance of the rail defect inference engine is evaluated using a new track segment validation set. If the performance does not improve or begins to decline within 5 consecutive cycles, the parameter fine-tuning is immediately terminated and the parameters are rolled back to the parameters saved in the incremental training cycle with the best evaluation performance. The loss during parameter fine-tuning consists of both feature consistency loss and domain adaptation loss. The feature consistency loss is defined as the mean square error loss between the enhanced feature vector extracted by the current rail defect inference engine and the enhanced feature vector extracted by the original rail defect inference engine. The domain adaptation loss minimizes the difference between the feature distribution of the new road segment scene and the feature distribution learned by the original rail defect inference engine.

[0014] Optionally, the step of back-projecting the maximum value point of the abnormal heatmap to determine the estimated defect location, and the mobile robot planning its path and adjusting its pose to align the camera's optical center with the estimated defect location, includes: The two-dimensional pixel coordinates of the maximum value point on the abnormal heat map of the current real-time railway track image are obtained, and then converted into normalized coordinates in the camera coordinate system based on the camera intrinsic parameter matrix. Finally, they are converted into three-dimensional spatial coordinates in the motion robot coordinate system based on the extrinsic parameter matrix of the camera relative to the mobile robot base, which are used as the predicted location of the defect. Plan a route to take two separate photos of the railway tracks from at least two different perspectives; The motion control vector is calculated based on the difference between the predicted defect location and the position of the mobile robot in the global coordinate system, and the position is adjusted accordingly. The yaw angle adjustment is calculated based on the pixel deviation between the camera's optical center and the predicted defect location, and the attitude is adjusted accordingly.

[0015] The present invention also provides a rail defect detection system based on a mobile robot, for applying the aforementioned rail defect detection method based on a mobile robot, comprising: a cloud platform, an edge computing layer, and a terminal execution layer; The cloud platform includes: a defect data warehouse and intelligent analysis module, a system management and data visualization module, an alarm and report automatic generation module, and a model warehouse and incremental learning management module. The defect data warehouse and intelligent analysis module are used to store and manage all defect information returned by the edge computing layer, and to perform trend analysis and predictive maintenance judgment based on historical data. The multi-robot task scheduling and status monitoring module is responsible for assigning inspection tasks to multiple mobile robots and monitoring their location, power level, and working status in real time on the global map. The system management and data visualization module is used to provide a human-computer interaction interface, perform system configuration and user management, and intuitively display the detection results and robot status information in the form of charts. The alarm and report automatic generation module is used to receive confirmed defect alarm information and notify relevant personnel, and can also automatically generate inspection reports. The model repository and incremental learning management module are used to store the rail defect inference engine and the rail normal state statistical model under different road segment scenarios. When the system performance drops in a new road segment scenario, the module uses the new road segment normal rail image uploaded from the edge computing layer to start the incremental learning process, optimize the rail defect inference engine parameters, and push the updated rail defect inference engine to the corresponding edge computing device through the model distribution and deployment channel. The edge computing layer includes: a data acquisition and preprocessing module, a real-time inference engine, a local decision-maker, a data and alarm management module, and a robot motion control interface; The data acquisition and preprocessing module is used to acquire railway track images through a visual sensor, perform image preprocessing, and generate regions of interest as ROI image blocks. The real-time inference engine is used to load and run the rail defect inference engine, perform inference on ROI image blocks, and output anomaly scores and anomaly heatmaps. The local decision-maker is used to receive reasoning results and apply a dual-threshold active verification mechanism, set warning thresholds and confirmation thresholds, generate instructions to control the mobile robot to perform active verification and transmit defect information back, forming a detection closed loop. The data and alarm management module is used to package, compress, and encrypt defect information in preparation for transmission back to the cloud platform. The robot motion control interface is used to send motion control commands to the mobile robot according to the instructions generated by the local decision-maker. The terminal execution layer includes: sensing and perception components, motion and execution components, and communication and energy components; The sensing and perception components include a visual sensor and a positioning and attitude sensor; The visual sensor is used to acquire images of the railway track; the positioning and attitude sensor is used to acquire the pose information of the mobile robot. The motion and execution component is used to execute motion control commands from the edge computing layer, enabling the mobile robot to complete movement and pose adjustment. The communication and energy components are used to communicate data with the cloud platform and edge computing layer, and to provide energy for the entire system.

[0016] The present invention also provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the described method for detecting rail defects based on a mobile robot.

[0017] The present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute the rail defect detection method based on a mobile robot as described in any one of the claims.

[0018] By employing the above technical solutions, the present invention provides a method, system, equipment, and medium for detecting railway defects based on mobile robots, which has at least the following beneficial effects: (1) This invention has made a significant breakthrough in the paradigm of defect detection, effectively solving the inherent problem that it is difficult to obtain a large number of defect samples in industrial scenarios and that the labeling cost is high. Most existing technologies rely on supervised learning methods, which require a large number of labeled defect samples for model training. However, this invention creatively adopts an unsupervised anomaly detection path, so that only a large number of easily obtainable normal samples are needed to accurately define the health status of the rails. (2) The present invention has significant advantages in terms of model efficiency and deployment feasibility. In response to the challenge of limited onboard computing unit resources of mobile platforms such as quadruped robots, the present invention does not simply use complex models, but deeply optimizes the core detection model. By converting it to ONNX format and using TensorRT for INT8 quantization and other optimization methods, the computational load and memory usage of the model are greatly reduced. This lightweight design enables the powerful vision model to run efficiently on mobile devices, overcoming the bottleneck of deploying complex models on the edge. (3) The present invention exhibits excellent environmental adaptability and robustness. By collecting normal railway track images in multiple environments to construct a training set and introducing an incremental learning mechanism, when the system is deployed to a new section, the model can be quickly fine-tuned using a small number of normal samples from new scenes. This capability effectively overcomes interference from complex terrain, outdoor lighting changes, and cluttered backgrounds, ensuring the stability and accuracy of the detection system in different operating environments. (4) This invention introduces an active review mechanism based on dual thresholds. Its core is to set dual thresholds to achieve fine quantification of defect suspicion. Once the system determines that the defect has entered the "suspected defect" state, it immediately takes over the robot's motion control and triggers it to execute a set of active review action sequences, including autonomous path planning and precise pose adjustment, to obtain multi-view observation evidence. This multi-view verification process is essentially a Bayesian update decision, which greatly improves the confidence of confirmation. After the review is confirmed, all key information is automatically packaged and sent back, thus constructing an automated detection closed loop that integrates real-time perception, intelligent decision-making and active action. (5) The system constructed by this invention forms a complete cloud-edge-device collaborative operation system. This system uses the cloud platform as the scheduling and cognition center, responsible for macro-decision-making and resource optimization functions such as massive data management, multi-robot collaborative task scheduling, global model training and updating, alarm information processing, and historical data mining and trend prediction. The edge computing layer, as a key link in intelligent processing, is deployed on the robot's onboard computing unit and integrates ROS, defect detection nodes, real-time inference engine, and local decision-making module (including dual-threshold active verification logic). It performs real-time processing and intelligent analysis on the raw data collected by the terminal execution layer, and can perform local decision-making and closed-loop control when the network is limited or a rapid response is required, effectively sharing the cloud computing pressure and ensuring The system ensures the real-time performance and reliability of the obstacle removal system. The terminal execution layer consists of the mobile robot itself and its onboard hardware such as vision sensors, motion mechanisms, and positioning modules. It directly acts on the work site, responsible for collecting raw environmental data and image information, precise pose adjustment, and executing control commands from the edge computing layer or cloud platform. Through close collaboration among the cloud, edge, and terminal, the system achieves reasonable allocation and efficient linkage of computing tasks at different levels. The edge computing layer significantly reduces response latency and alleviates network bandwidth pressure. At the same time, sensitive terminal data can be processed locally to improve security, while the powerful computing power of the cloud platform supports complex model training and global optimization. Ultimately, the system achieves full automation and intelligence of railway inspection from perception to decision-making. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the rail defect detection method based on a mobile robot according to the present invention; Figure 2 This invention provides a schematic diagram illustrating the process of constructing a normal railway track image dataset for embodiments of the present invention. Figure 3 This is a schematic diagram of the multi-scale feature map generation process of the RepViT model provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of the process for establishing a statistical model of normal track status based on the PaDiM algorithm, provided in an embodiment of the present invention. Figure 5 This is an example of an abnormal heat map of a mobile robot transmitting data about a rail defect, provided in an embodiment of the present invention. Figure 6 This is a diagram illustrating the overall architecture of a mobile robot rail defect detection system provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0021] Those skilled in the art will understand that all or part of the steps in the implementation of the methods of the embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0022] Please refer to Figures 1-6 This illustration shows a specific implementation of this embodiment. This embodiment constructs a normal railway track image dataset, extracts multi-scale feature maps based on a pre-trained RepViT model and the PaDiM algorithm, and establishes a statistical model of normal railway track conditions. The pre-trained RepViT model is exported in ONNX format, and a railway defect inference engine is constructed. When the railway defect inference engine is deployed to a new section, parameter fine-tuning based on an incremental learning mechanism is performed. The engine is integrated into the operating system of a mobile robot, real-time railway track images are collected to calculate anomaly scores, and a dual-threshold active verification mechanism is designed. The mobile robot performs defect detection result judgment and autonomous verification, and transmits defect information back, forming a detection closed loop. This effectively solves the problems of strong dependence on defect samples, poor environmental adaptability, and difficulty in deployment on mobile platforms in existing technologies, and realizes efficient, real-time, and automated detection of railway defects in complex terrain.

[0023] Please refer to Figure 1 This embodiment proposes a rail defect detection method based on a mobile robot, which includes the following steps: S1. Raw railway track images and video streams from multiple scenarios are collected using a mobile robot. Preliminary inspection and preprocessing are performed, regions of interest are extracted and their sizes are normalized to obtain normal railway track images, thus constructing a normal railway track image dataset. This process can be found in [reference needed]. Figure 2 .

[0024] As a preferred embodiment of step S1, the specific process includes: First, using an RGB camera mounted on a mobile robot platform, raw images and video streams of the railway tracks were collected under various real-world operating environments. To ensure the constructed model possesses strong environmental robustness, the data acquisition process needs to systematically cover different lighting conditions, weather conditions, and diverse track scenarios. The core objective of this stage is to acquire a large amount of characterizing data that only includes railway tracks in normal conditions, while ensuring data diversity to simulate various complex situations that may be encountered in real-world inspections.

[0025] After acquiring the original railway track images and video streams, a rigorous quality assessment and cleaning process is required. This process includes preliminary checks on the number, format, and integrity of the images, removing corrupted files caused by equipment malfunctions or transmission errors, and constructing the original railway track image dataset.

[0026] Subsequently, in order to eliminate data redundancy, deduplication processing based on image hashing algorithm is adopted. By calculating the average hash value and comparing the Hamming distance, highly similar or duplicate image frames generated by continuous shooting are effectively identified and removed.

[0027] To address unavoidable image quality issues during acquisition, noise filtering and image inpainting are implemented. For images with uneven illumination, a homomorphic filtering algorithm is applied to separate the illumination and reflection components in the frequency domain, enhancing details and performing illumination correction. The homomorphic filtering algorithm formula is as follows: ; in, This represents the spatial coordinates of the output image after processing by the homomorphic filtering algorithm. Pixel intensity value at that location , These represent the row and column indices of a pixel in the image matrix, i.e., its spatial coordinates. It is a Gaussian high-pass filter. , These represent the frequency coordinates in the horizontal and vertical directions in the frequency domain, respectively. It is the Fast Fourier Transform. express inverse transform; Represents the natural logarithm operation. Represents the spatial coordinates of the original input image to be processed. Pixel intensity value at that location This indicates the operation of the natural exponent.

[0028] Subsequently, a medium-frequency filtering algorithm is applied to remove noise, generating a preprocessed image of the railway track.

[0029] To further improve the efficiency and targeting of subsequent defect detection, it is necessary to accurately extract the linear regions of the railway tracks as regions of interest (ROIs) from the entire scene image. This method employs a classic computer vision algorithm based on edge detection and Hough transform: the preprocessed railway track image is converted into a grayscale image and Gaussian blurred to suppress noise, resulting in a Gaussian railway track grayscale image; then, the Canny operator is used to perform edge detection on the Gaussian railway track grayscale image, and the Hough transform is used to detect the straight line segments within it. Based on the parallel line characteristics of the railway tracks in the image, the positions of the straight line segments are located; finally, the minimum bounding rectangle is calculated and constructed based on the detected and located straight line segments, thereby generating the region of interest for model training.

[0030] After extracting the regions of interest (ROIs), all ROIs are normalized to a uniform input size required by the subsequent RepViT model. In this process, bilinear interpolation is used to scale the ROIs to 640×640 pixels for use as input to the RepViT model.

[0031] This scheme did not choose the 224×224 resolution set by the traditional RepViT model because 640×640 achieves the best balance between preserving sufficient image details to detect minor defects and controlling system computational complexity to achieve real-time inference at the edge. Meanwhile, the bilinear interpolation algorithm, by performing linear interpolation calculations between adjacent pixels, can maintain image visual quality well while ensuring processing efficiency. The calculation process of the bilinear interpolation algorithm can be described as follows: ; In the above formula, This represents the spatial coordinates of the point to be interpolated in the image. , , , They are respectively The coordinates of the bottom-left, bottom-right, top-left, and top-right pixels surrounding the grid in the image, where , This represents the floor function; the "surroundings" refers to the points to be interpolated. It falls within the unit square area formed by these four pixels; This represents the bilinear interpolation operation. Indicates to The result of bilinear interpolation, i.e., the estimated pixel intensity value for that pixel; symbol This means "approximately equal to," implying that the result obtained by this method is essentially a high-quality linear approximation, rather than an exact value. , They are respectively , The interpolation weights in the direction are specifically defined as the interpolation weights of the points to be interpolated. Relative to the bottom left pixel of its grid The relative distance is calculated using the following formula: , representing the horizontal distance between the point to be interpolated and the left edge of the grid; , which represents the vertical distance between the point to be interpolated and the bottom edge of the grid.

[0032] After the above series of processing steps, a clean, non-redundant, and consistent quality normal railway track image dataset is finally obtained, providing high-quality input for the subsequent training of unsupervised anomaly detection models.

[0033] S2. Input the normal railway track image into the pre-trained RepViT model to extract multi-scale feature maps; based on the PaDiM algorithm, upsample the extracted multi-scale feature maps to a uniform size and then stitch them along the channel dimension to form an enhanced feature vector. Calculate the mean vector and covariance matrix of its multivariate Gaussian distribution to establish a statistical model of the normal state of the railway track.

[0034] As a preferred embodiment of step S2, the specific process includes: First, the normal railway track image obtained after step S1 is input into the pre-trained RepViT model for multi-scale feature extraction. RepViT is an innovative visual Transformer architecture. Its core advantage over traditional Vision Transformers or complex CNN models lies in its inherent attention mechanism, which provides global contextual information capture capabilities while achieving extremely low computational complexity and parameter count, thus meeting the real-time requirements of mobile deployments. The RepViT model used in this invention is a benchmark model obtained after pre-training on the ImageNet dataset using a supervised classification task. This benchmark model itself possesses powerful and general visual feature extraction capabilities. This pre-training stage employs the standard cross-entropy loss function. Its formula is: ; in The total number of categories, For real labels, For the RepViT model to predict belonging to the th The probability of a class This indicates logarithmic calculation.

[0035] This method does not use the final output of the network, but instead extracts feature maps from specific stages of its backbone network. The feature maps extracted in Stage 2 are selected to capture subtle texture defects on the rail surface, those in Stage 3 are used to identify medium-scale structural anomalies, and those in Stage 4 are used to detect macroscopic geometric deformations. This multi-scale fusion strategy ensures comprehensive perception of defects from the pixel level to the object level. For an input image, the set of multi-scale feature maps output by the RepViT model can be represented as: ; in , , These represent the spatial height, width, and number of channels of the feature map extracted in the first stage. Represents multi-scale feature maps, including Stage2 feature maps. Stage 3 Feature Map Stage 4 Feature Map .

[0036] The process of multi-scale feature map extraction in the RepViT model described above is as follows: Figure 3 As shown, the input normal railway track image has a spatial size of 640×640×3. The Stage1 feature map is obtained through its backbone layer and Stage1. Then, through downsampling and Stage2, the Stage2 feature map is obtained. Then, through downsampling and Stage 3, the Stage 3 feature map is obtained. Then, through downsampling and Stage 4, the Stage 4 feature map is obtained. .

[0037] Subsequently, based on the extracted multi-scale feature maps, the PaDiM algorithm first transforms them to a uniform size through bilinear upsampling, and then splices them along the channel dimension to form an enhanced feature vector, thereby modeling the normal feature distribution and establishing a statistical model of the normal state of the railway track.

[0038] PaDiM (Patch Distribution Modeling) is an unsupervised anomaly detection and localization algorithm, particularly suitable for industrial scenarios where defect samples are scarce and their types are unknown. See also... Figure 4 Given Zhang Zheng's sample image (i.e., the preprocessed railway track image in this invention) is used to extract image blocks. (that is, the normal railway track image obtained by region of interest extraction in this invention) Image embedding (i.e., the process of extracting multi-scale feature maps using the RepViT model in this invention), followed by upsampling and concatenation, generates an enhanced feature vector set. , is represented as: ; in, Indicates the first The enhanced feature vectors corresponding to each image; For each spatial location on the enhanced feature vector Construct a multivariate Gaussian distribution and perform parameter estimation to obtain the mean vector of the multivariate Gaussian distribution. Covariance Matrix This is used to describe the characteristic changes of the location under normal conditions, forming a Gaussian parameter matrix, and then establishing a statistical model of the normal state of the railway track.

[0039] Furthermore, since the RepViT model is converted to a pure CNN architecture after training, its forward inference process consists entirely of highly optimized convolutional operations. The locality of convolution ensures that its output feature map perfectly corresponds to image patches in space. This aligns perfectly with the PaDiM algorithm's assumption that it needs to build an independent Gaussian distribution model for each spatial location and perform Mahalanobis distance calculations. This architectural consistency allows the RepViT model to provide the PaDiM algorithm with the most suitable, high-quality feature representation, thereby achieving extreme acceleration on edge devices while ensuring high-precision defect detection.

[0040] During mobile robot movement, image jitter and slight blurring are inevitable due to robot body sway. The RepViT+PaDiM combination constructed in this invention ensures that the local feature representation of images acquired in a non-stationary state remains relatively stable, thereby significantly reducing false alarms and false negatives caused by robot movement and improving the practicality of mobile inspection. This is because the pure convolutional architecture after RepViT transformation has the inherent characteristics of local connectivity and translation invariance, and its convolution operation is insensitive to small changes in local image areas (such as slight motion blur).

[0041] Meanwhile, RepViT's extremely low computational load directly translates into low power consumption for the onboard computing device. This not only affects processing speed but also means a significant reduction in energy consumption for the computing unit with the same battery capacity. For mobile platforms such as mobile robots, battery life is a key bottleneck determining the range of a single inspection. Reduced energy consumption of the computing module frees up more power for the motion module, directly extending the single-operation time of the mobile robot and expanding the inspectable range. This is a significant advantage that is not considered by inspection systems deployed in fixed locations. The RepViT-PaDiM fusion scheme designed in this invention generates high-quality, pixel-level anomaly heatmaps, which are general "anomaly probability" measures independent of specific defect types. Mobile robot decision-making systems can directly utilize the intensity distribution and location information of the anomaly heatmaps, rather than specific defect classification results, to generate flexible verification strategies (such as "approaching the brightest area on the heatmap"). This spatial attention-based guidance enables robots to leap from "passive photography" to "active exploration."

[0042] Ultimately, the entire flawless rail appearance was modeled as a statistical model consisting of multivariate Gaussian distributions of all spatial locations (i.e., the constructed statistical model of the rail's normal state), which fully depicts the appearance characteristics and statistical properties of a normal rail at different scales.

[0043] Table 1 shows the test results of the RepViT-PaDiM fusion scheme designed in this invention compared with other existing schemes (AutoEncoder, PatchCore, ResNet-50). In the table, "AUROC" represents the area under the curve, serving as an evaluation metric for detection accuracy. It can be seen that the RepViT-PaDiM fusion scheme designed in this invention achieves a significant reduction of two orders of magnitude in model parameters (corresponding to "model size" in Table 1) and a more than three-fold increase in inference speed, with extremely low loss in detection accuracy. This successfully solves the industry problem of existing high-performance detection algorithms being bloated, computationally expensive, and difficult to deploy at edge computing environments. (The test environment for the test results in Table 1 is...) NVIDIA Jetson AGX Orin ("-" indicates that the solution does not have this function or the indicator is not applicable.) Model Name technical route Image-level AUROC (%) Pixel-level AUROC (%) Inference speed (FPS) Model size (MB) Do we need defect samples? AutoEncoder Unsupervised reconstruction 82.5 55.1 30 5.1 no PatchCore Unsupervised memory 96.1 95.8 8 1024 no ResNet-50 Supervised learning 97.5 - 12 90 yes This invention (RepViT-PaDiM) Unsupervised feature modeling 95.8 94.9 25 3.5 no S3. Export the pre-trained RepViT model to ONNX format and build a rail defect inference engine to adapt to edge device deployment.

[0044] As a preferred embodiment of step S3, the specific process includes: The RepViT model pre-trained with the PyTorch framework was exported to the ONNX format, and then an inference engine for rail defects was built using NVIDIA TensorRT.

[0045] In the INT8 quantization stage, the construction of the calibration set is a crucial step in ensuring the accuracy of the quantized model. The calibration set consists of a set of samples that represent the data distribution of the target scene. Its function is to estimate the dynamic range of activation values ​​for each layer in the network, thereby determining the quantization scaling factor S and the zero point Z, and realizing the accurate mapping of the RepViT model from FP32 accuracy type to INT8 accuracy type. Its quantization transformation can be expressed as: ; The scaling factor S and the zero point Z are determined using an entropy calibration algorithm. This indicates the rounding operation.

[0046] To ensure the accuracy of the quantized RepViT model in inference within the road segment, the calibration set must fully reflect the actual data distribution in that road segment scenario. Therefore, 800 representative images were randomly selected from normal samples of the new road segment scenario to form the calibration set, ensuring coverage of the main lighting conditions, background environment, and other visual variation features in that scenario.

[0047] "New scenarios" specifically refers to a set of road sections or environmental conditions that have not been sampled when constructing a normal railway track image dataset; "Normal samples" specifically refer to railway track images acquired in the new scenario that have been manually reviewed or verified by reliable benchmark algorithms and confirmed to contain no defects or abnormal features. This differs from the preliminary inspection and preprocessing methods in step S1: step S1 efficiently constructs a high-quality, non-redundant, and uniformly formatted normal railway track image dataset from a large-scale collection of raw railway track images and video streams; while the normal samples used for INT8 quantization here need to be 100% confirmed to contain no defects to prevent contamination of the calibration process (i.e., more stringent than the data screening in step S1). "Representative railway track images" means that the selected railway track images need to systematically cover the external sources of variation in the new scene, so as to fully characterize the visual diversity of the scene, such as different lighting conditions and the possible changes in perspective during the movement of the mobile robot.

[0048] This calibration set will be used to generate quantization parameters (including scaling factors and zeros), thereby enabling the model to retain its recognition performance in new scenes to the maximum extent while maintaining low-precision computational efficiency.

[0049] Unlike conventional methods that use the original training set for calibration, this invention employs an adaptive calibration method tailored to the target scene when generating quantization parameters. The conventional calibration process involves using inference optimization tools such as TensorRT and selecting the entropy calibration algorithm of trt.IInt8EntropyCalibrator2. The key difference between this invention and conventional methods lies in using normal samples from the new scene as calibration input, rather than the model's original training data. The calibration iterates through the constructed calibration set, statistically analyzing the dynamic distribution range of the RepViT model's activation values, and calculating the scaling factor (S) and zeros (Z) that optimally represent the data distribution of the new scene. This design enables the quantization parameters to accurately match the real data distribution of the new scene, mitigating quantization errors caused by domain offset at their source.

[0050] S4. When the rail defect inference engine is deployed to a new section, the parameters of the rail defect inference engine are automatically fine-tuned based on the incremental learning mechanism. After the parameter fine-tuning, the statistical model of the normal state of the rail in the new section scenario is reconstructed based on the PaDiM algorithm.

[0051] As a preferred embodiment of step S4, the specific process includes: When the rail defect inference engine is deployed to a new section of track and its performance degrades due to differences in the scenario, the system will initiate parameter fine-tuning based on an incremental learning mechanism.

[0052] First, images of the railway tracks in the new section were collected, cleaned, and their normality was confirmed. A dataset of normal railway track images of the new section was constructed and divided into a training set and a validation set for the new section.

[0053] The rail defect inference engine was incrementally trained using the new track section training set.

[0054] The validation set for new road sections is strictly independent during model fine-tuning. It is used only for performance evaluation after each training cycle and serves as the basis for judging the early stopping strategy. It does not participate in the model's parameter updates.

[0055] During incremental learning, the hyperparameter settings follow the principle of stable adaptation. The learning rate, as the core hyperparameter, is initially set to the order of 1e-4, adjusting the model parameters in a small and gradual manner to avoid disturbing the original knowledge structure, thereby effectively preserving the basic features learned in the original scene. Simultaneously, the optimizer used is stochastic gradient descent with momentum (SGD), with the momentum parameter set to 0.9.

[0056] To determine the optimal fine-tuning endpoint, the stopping condition does not depend on a fixed number of cycles, but is based on continuous monitoring of the validation set performance. This invention employs an early stopping strategy. That is, after each incremental training cycle, the performance of the rail defect inference engine is evaluated using a newly held, independently held validation set of railway segments. If the performance does not improve or begins to decline within 5 consecutive cycles, parameter fine-tuning is immediately terminated, and the parameters are rolled back to those saved in the incremental training cycle with the best evaluation performance. This ensures the generalization ability of the final model (i.e., the rail defect inference engine after parameter fine-tuning).

[0057] The core of this scheme is to employ a fine-tuning strategy based on feature distribution alignment. During fine-tuning, the loss function is composed of feature consistency loss and a domain adaptation loss. Its loss function... Represented as: ; in To balance hyperparameters, For feature consistency loss, Domain adaptation loss.

[0058] The feature consistency loss is used to ensure that the enhanced feature vector extracted by the rail defect inference engine after parameter fine-tuning for the rail image of the new section maintains similarity with the enhanced feature vector extracted by the original rail defect inference engine before parameter fine-tuning for the same rail image. Its purpose is to preserve the model's original feature extraction capability and prevent forgetting. This loss is achieved by calculating the mean squared error of the two features, as expressed in the following formula: ; in, This indicates the calculation of the mean square error. Images showing the railway tracks of the new section. , These represent the rail defect inference engine after parameter fine-tuning and the original rail defect inference engine before parameter fine-tuning, respectively, for the rail images of the new section. Extracted enhanced feature vectors; Domain adaptation loss aims to minimize the difference between the feature distribution of the new railway segment scene and the feature distribution learned by the original railway defect inference engine (i.e., the original RepViT model). In this embodiment, the maximum mean discrepancy (MMD) is used as the metric, and its calculation formula can be expressed as: ; in, This represents the calculation of the square of the maximum mean difference. The set of feature vectors learned by the original RepViT model. This is a set of feature vectors for the new road segment scenario. express The Middle 1 eigenvector express The Middle 1 eigenvector express Total number of feature vectors in the middle. express Total number of feature vectors in the middle. For feature mapping function, For the regenerative nucleus Hilbert space (RKHS). In the regenerating nucleus Hilbert space The squared norm of a vector.

[0059] After fine-tuning, the parameters of the multivariate Gaussian distribution for each spatial location are recalculated using the features of the new road segment scenario. Covariance Matrix This allows for the rapid reconstruction of a statistical model of the normal state of railway tracks in new track sections.

[0060] This process ensures that the system can quickly adapt to environmental changes with minimal additional data costs, significantly improving its practicality and robustness in real-world complex operation and maintenance scenarios.

[0061] S5. Integrate the rail defect inference engine with fine-tuned parameters into the operating system of the mobile robot and create a defect detection node. The defect detection node receives real-time rail images, calls a preprocessing callback function to preprocess the real-time rail images, and then sends them to the rail defect inference engine to extract enhanced feature vectors. Based on the statistical model of normal rail conditions, the Mahalanobis distance of the enhanced feature vectors is calculated as the anomaly score, and then an anomaly heatmap is generated.

[0062] As a preferred embodiment of step S5, the specific process includes: First, a standalone defect detection node is created in the ROS environment using Python. This node serves as the core computational unit of the entire detection system and integrates the rail defect inference engine and the PaDiM algorithm. Upon receiving each frame of real-time rail image, the defect detection node calls a preprocessing callback function to preprocess the real-time rail image. Then, the preprocessed real-time rail image tensor is fed into the rail defect inference engine for forward propagation, quickly calculating the corresponding anomaly score. And generate an anomaly heatmap. The anomaly score is... It is obtained by calculating the Mahalanobis distance of the enhanced feature vectors, as shown in the following formula: ; in, Indicates abnormal scores. This indicates the calculation of Mahalanobis distance. For real-time railway track images in spatial location Enhanced feature vector at the location, , These are the horizontal and vertical pixel indices of the image, respectively. , These are the mean vector and covariance matrix of the multivariate Gaussian distribution in the statistical model of normal railway track conditions, respectively. express The inverse matrix, Indicates matrix transpose; After obtaining the anomaly scores, the anomaly scores of each spatial location are integrated, with blue representing low anomaly scores and red representing high anomaly scores, thus obtaining a visually appealing anomaly heatmap.

[0063] S6. Design a dual-threshold active review mechanism, setting an early warning threshold and a confirmation threshold, and the mobile robot judges the defect detection results.

[0064] As a preferred embodiment of step S6, the specific process includes: This solution introduces a dual-threshold active review mechanism, which sets early warning thresholds. and confirmation threshold To optimize the logic for judging defect detection results. When the anomaly score of a certain frame of real-time railway track image... Below the warning threshold When the score is high, it is directly judged as defect-free; when the score of an anomaly in a real-time railway track image is low... Above the confirmation threshold When an anomaly score is detected, it is directly identified as a defect, and an alarm process is immediately triggered; when an anomaly score is detected in a real-time rail image frame... Given the warning threshold and confirmation threshold During this period, the system enters a suspected defect state, triggering the review process.

[0065] In cases of suspected defects, the mobile robot plans a path to take secondary photographs of the railway track from at least two different perspectives. During the review process, the estimated location of the defect is first determined based on an anomaly heatmap, specifically as follows: An anomaly heatmap of the current real-time railway track image is obtained. By finding the maximum point on the heatmap, the two-dimensional pixel coordinates of the defect in the real-time railway track image are obtained. Then, using the inverse matrix of a pre-calibrated camera intrinsic parameter matrix, the two-dimensional pixel coordinates are transformed back to normalized coordinates in the camera coordinate system. It is a 3x3 matrix, and its standard form is as follows: ; in, , These represent the camera's focal length along the x-axis and y-axis of the image, respectively. , These represent the pixel coordinates of the camera principal point on the x-axis and y-axis of the image, respectively; Then, normalization is performed to obtain the direction vector of the maximum point in the camera coordinate system, as shown in the following formula: ; in, Represents the camera intrinsic parameter matrix The inverse matrix, These are the pixel coordinates in the anomaly heatmap. This represents the direction vector of the maximum point in the camera coordinate system.

[0066] Then, an inverse transformation is performed based on the camera extrinsic parameter matrix to map the maximum point in the camera coordinate system back to the robot coordinate system. The specific calculation formula is as follows: ; Wherein, rotation matrix It is a 3x3 orthogonal matrix, derived from the three angles of roll, pitch, and yaw; the translation vector... It is a 3x1 vector representing the translation offset from the origin of the robot coordinate system to the origin of the camera coordinate system; It is a scale factor; The final obtained three-dimensional spatial coordinates are the predicted location of the defect.

[0067] After obtaining the predicted defect location, the mobile robot adjusts its pose to align the camera's optical center with the predicted defect location. Pose adjustment includes adjustments to both position and orientation: position adjustment is based on motion control vectors. ,in To determine the position of the mobile robot in the global coordinate system, For defect location estimation; simultaneously, the yaw angle adjustment of the mobile robot. Calculated based on the pixel deviation between the camera's optical center and the predicted defect location, i.e. ,in This is the proportionality coefficient. Here are the horizontal pixel coordinates of the defect in the real-time railway track image. The horizontal pixel coordinates of the center of the real-time railway track image. This is the image width. This adjustment ensures that the mobile robot can capture defect details from the optimal perspective.

[0068] After the mobile robot acquires 3-5 close-up real-time images of the railway track from multiple angles, the system obtains the anomaly scores of these real-time railway track images and calculates the average. If the average score is still higher than the target score, the system will take further action. If it is found to be defective, it will be confirmed as defective; otherwise, it will be determined as defect-free and exit the suspected defect status.

[0069] This multi-perspective verification strategy effectively integrates observational evidence, which is equivalent to a Bayesian update process, thereby improving the confidence of defect detection.

[0070] After obtaining the defect detection results, the S7 mobile robot transmits high-definition images of the defects, abnormal heat maps, GPS / BeiDou positioning information, and timestamps back to the monitoring center via the 5G network, completing the entire detection process.

[0071] The abnormal heat map transmitted back to the rail defect by the mobile robot can be referenced. Figure 5 , Figure 5 The image shows grayscale images (i.e. Gaussian rail grayscale images) of rail images after preprocessing for three types of defects: patches, scratches, and inclusions, as well as the generated abnormal heat maps.

[0072] This application also provides a rail defect detection system based on a mobile robot, used to apply the aforementioned rail defect detection method based on a mobile robot. The overall architecture diagram of this system can be found in [reference needed]. Figure 6 This includes: cloud platform, edge computing layer, and terminal execution layer; Specifically, the cloud platform, serving as the system's intelligent hub and data base, includes the following modules: (1) Defect data warehouse and intelligent analysis module, used to store and manage all defect information (such as images, heat maps, location stamps, etc.) returned by the edge computing layer, and to perform trend analysis and predictive maintenance judgment based on historical data; (2) Multi-robot task scheduling and status monitoring module, which is responsible for assigning inspection tasks to multiple mobile robots and monitoring their location, power, and working status in real time on the global map; (3) System management and data visualization module, which provides a human-computer interaction interface, performs system configuration and user management, and displays the detection results and robot status information in a visually intuitive form in the form of charts; (4) Alarm and report automatic generation module, which is used to receive confirmed defect alarm information and notify relevant personnel through various means, and can automatically generate inspection reports; (5) The model repository and incremental learning management module is the key to realizing the system's self-evolution. It is used to store the rail defect inference engine and the rail normal state statistical model under different road segment scenarios. When the system performance drops in a new road segment scenario, this module uses the new road segment normal rail image uploaded from the edge computing layer to start the incremental learning process, optimize the rail defect inference engine parameters, and push the updated rail defect inference engine to the corresponding edge computing device through the model distribution and deployment channel. Specifically, the edge computing layer, deployed on the robot's onboard computer, includes: (1) Data acquisition and preprocessing module, used to acquire railway track images through visual sensors, perform image preprocessing (such as grayscale conversion, scaling, etc.), and generate regions of interest as ROI image blocks; (2) Real-time inference engine, used to load and run the rail defect inference engine, perform inference on ROI image blocks, and output global anomaly scores and high-resolution anomaly heatmaps; the loaded rail defect inference engine is optimized by TensorRT. The key step in the optimization process is INT8 quantization, that is, the model weights and activation values ​​are converted from FP32 floating-point precision to INT8 integer precision through calibration technology, thereby greatly reducing the model size and computational latency, and meeting the stringent requirements of mobile platforms for power consumption and real-time performance.

[0073] (3) Local decision-maker, used to receive reasoning results and apply dual threshold active verification mechanism, set warning threshold and confirmation threshold to make dual threshold judgment, confirm defects or enter suspected defect state, generate instructions to control mobile robot to perform active verification and return defect information, and form detection closed loop; (4) Data and alarm management module, used to package, compress and encrypt defect information (original rail images and their corresponding abnormal heat maps, defect locations, etc.) and prepare them for transmission back to the cloud platform; (5) Robot motion control interface, which serves as a ROS Action client and is used to send motion control commands to the mobile robot based on the instructions generated by the local decision-maker; Specifically, the terminal execution layer is the mobile robot body (in this embodiment, a quadruped robot body is used as an example), including: (1) Sensing and perception components, including visual sensors such as cameras, positioning and attitude sensors such as GPS and IMU; The visual sensor is used to acquire images of the railway track as sensor data; the positioning and attitude sensor is used to acquire the pose information of the mobile robot as the robot's state. (2) Motion and execution components, used to execute motion control commands from the edge computing layer, enabling the mobile robot to complete movement and pose adjustment; (3) Communication and energy components, used for data communication with the cloud platform and edge computing layer, and to provide energy for the entire system.

[0074] Figure 6 The arrows clearly illustrate the dynamic interactions within the constructed mobile robot-based rail defect detection system, specifically: (1) Uplink data flow: Sensor data and robot status from the terminal execution layer are uploaded to the edge computing layer; The edge computing layer uploads the confirmed defect information and domain adaptive training data (i.e., data such as railway images collected in the new road section scenario) to the cloud platform. (2) Downlink control and update flow: The cloud platform deploys and updates the optimized model to the edge computing layer through the model distribution; the edge computing layer sends motion control commands to the terminal execution layer through the robot motion control interface; (3) Intra-layer closed loop: Inside the edge computing layer, from data acquisition to real-time reasoning, and then to local decision-making and control, a complete and real-time "perception-decision-action" closed loop is formed, which is the basis for achieving efficient and autonomous inspection.

[0075] This application also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the described rail defect detection method based on a mobile robot.

[0076] This application also provides a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the described rail defect detection method based on a mobile robot.

[0077] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0079] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting railway track defects based on a mobile robot, characterized in that, include: Original railway track images and video streams from multiple scenarios were collected by a mobile robot. After preliminary inspection and preprocessing, regions of interest were extracted and their sizes were normalized to obtain normal railway track images and construct a normal railway track image dataset. Normal railway track images are input into a pre-trained RepViT model to extract multi-scale feature maps; The multi-scale feature maps include Stage2 feature maps, Stage3 feature maps, and Stage4 feature maps, which are extracted from Stage2, Stage3, and Stage4 of the RepViT model backbone network, respectively. Based on the PaDiM algorithm, the extracted multi-scale feature maps are upsampled to a uniform size and then stitched along the channel dimension to form an enhanced feature vector. The mean vector and covariance matrix of its multivariate Gaussian distribution are calculated to establish a statistical model of the normal state of the railway track. Export the pre-trained RepViT model to ONNX format to build a rail defect inference engine to adapt to edge device deployment; When the rail defect inference engine is deployed to a new section, the parameters of the rail defect inference engine are automatically fine-tuned based on the incremental learning mechanism. After the parameter fine-tuning, the statistical model of the normal state of the rail in the new section scenario is reconstructed based on the PaDiM algorithm. The rail defect inference engine with fine-tuned parameters is integrated into the operating system of the mobile robot to create defect detection nodes. The defect detection node receives real-time rail images, calls a preprocessing callback function to preprocess the real-time rail images, and then sends them to the rail defect inference engine to extract enhanced feature vectors. Based on the rail normal state statistical model, the Mahalanobis distance of the enhanced feature vectors is calculated as the anomaly score, and then an anomaly heatmap is generated. Design a dual-threshold active review mechanism, setting an early warning threshold and a confirmation threshold, and use a mobile robot to judge the defect detection results; When the anomaly score of a real-time railway track image frame is lower than the warning threshold, it is directly judged as defect-free; When the anomaly score of a real-time railway track image exceeds the confirmation threshold, it is directly determined as a defect and an alarm process is immediately triggered. When the anomaly score of a real-time rail image frame is between the warning threshold and the confirmation threshold, it enters the suspected defect state, triggers the review process, back-projects the maximum value point of the anomaly heatmap to determine the estimated defect location, the mobile robot plans the path and adjusts its pose to align the camera optical center with the estimated defect location, and re-acquires real-time rail images from at least two perspectives, obtains the anomaly scores of these real-time rail images and calculates the average value. If the average value is higher than the confirmation threshold, it is finally determined to be a defect; otherwise, it is determined to be without defect and exits the suspected defect state. After receiving the defect detection results, the mobile robot transmits the defect information back, completing the detection process.

2. The rail defect detection method based on a mobile robot according to claim 1, characterized in that: The preliminary inspection and preprocessing, extraction of the region of interest and size normalization, to obtain a normal railway track image includes: The original railway track images and video streams are initially checked for the number, format, and integrity of the images, and corrupted files caused by equipment failure or transmission errors are removed to construct the original railway track image dataset. Image hashing algorithm, homomorphic filtering algorithm, and median filtering algorithm are applied to preprocess the original railway track image dataset for deduplication, illumination correction, and noise reduction, respectively, to generate preprocessed railway track images; The linear region of the railway track is extracted as the region of interest from the preprocessed railway track image using a method based on edge detection and Hough transform. The size of the region of interest was normalized using a bilinear interpolation algorithm and adjusted to 640×640 to form a normal railway track image.

3. The rail defect detection method based on a mobile robot according to claim 2, characterized in that: The method of extracting linear regions of the railway track as regions of interest from the preprocessed railway track image using edge detection and Hough transform includes: The preprocessed railway track image is converted to grayscale and Gaussian blurring is performed to suppress noise, resulting in a Gaussian railway track grayscale image. Edge detection is performed on the Gaussian rail grayscale image using the Canny operator, and then the straight line segments are detected and located using the Hough transform. The minimum bounding rectangle is calculated and constructed based on the detected and located line segments, and this rectangular region is taken as the region of interest.

4. The rail defect detection method based on a mobile robot according to claim 1, characterized in that: The step of exporting the pre-trained RepViT model to ONNX format includes: The export process uses INT8 quantization.

5. The rail defect detection method based on a mobile robot according to claim 4, characterized in that: The INT8 quantization includes: A calibration set is constructed by randomly selecting representative railway track images from normal samples of the new scene, and quantization parameters are generated. The new scene refers to a scene that was not sampled when constructing a normal railway track image dataset; The normal sample refers to a railway track image collected in the new scenario that has been confirmed to contain no defects or abnormal features after manual review or algorithm verification. The representative railway track image is one that can cover the sources of external variation in the new scene; The quantization parameters include scaling factor and zero point; Based on the quantization parameters, the RepViT model is mapped from FP32 precision type to INT8 precision type.

6. The rail defect detection method based on a mobile robot according to claim 1, characterized in that: The automatic parameter fine-tuning of the rail defect inference engine based on an incremental learning mechanism when deployed to a new track section includes: Collect railway track images of the new section and clean and verify their normality. Construct a dataset of normal railway track images of the new section and divide it into a training set and a validation set for the new section. Incremental training of the rail defect inference engine was performed using the new track section training set. An early stop strategy is adopted. After each incremental training cycle, the performance of the rail defect inference engine is evaluated using a new track segment validation set. If the performance does not improve or begins to decline within 5 consecutive cycles, the parameter fine-tuning is immediately terminated and the parameters are rolled back to the parameters saved in the incremental training cycle with the best evaluation performance. The loss during parameter fine-tuning consists of both feature consistency loss and domain adaptation loss. The feature consistency loss is defined as the mean square error loss between the enhanced feature vector extracted by the current rail defect inference engine and the enhanced feature vector extracted by the original rail defect inference engine. The domain adaptation loss minimizes the difference between the feature distribution of the new road segment scene and the feature distribution learned by the original rail defect inference engine.

7. The rail defect detection method based on a mobile robot according to claim 1, characterized in that: The process of back-projecting the maximum value point of the abnormal heatmap to determine the estimated defect location, and the mobile robot planning its path and adjusting its pose to align the camera's optical center with the estimated defect location includes: The two-dimensional pixel coordinates of the maximum value point on the abnormal heat map of the current real-time railway track image are obtained, and then converted into normalized coordinates in the camera coordinate system based on the camera intrinsic parameter matrix. Finally, they are converted into three-dimensional spatial coordinates in the motion robot coordinate system based on the extrinsic parameter matrix of the camera relative to the mobile robot base, which are used as the predicted location of the defect. Plan a route to take two separate photos of the railway tracks from at least two different perspectives; The motion control vector is calculated based on the difference between the predicted defect location and the position of the mobile robot in the global coordinate system, and the position is adjusted accordingly. The yaw angle adjustment is calculated based on the pixel deviation between the camera's optical center and the predicted defect location, and the attitude is adjusted accordingly.

8. A rail defect detection system based on a mobile robot, used to apply the rail defect detection method based on a mobile robot as described in any one of claims 1-7, characterized in that, include: Cloud platform, edge computing layer, terminal execution layer; The cloud platform includes: a defect data warehouse and intelligent analysis module, a system management and data visualization module, an alarm and report automatic generation module, and a model warehouse and incremental learning management module. The defect data warehouse and intelligent analysis module are used to store and manage all defect information returned by the edge computing layer, and to perform trend analysis and predictive maintenance judgment based on historical data. The multi-robot task scheduling and status monitoring module is responsible for assigning inspection tasks to multiple mobile robots and monitoring their location, power level, and working status in real time on the global map. The system management and data visualization module is used to provide a human-computer interaction interface, perform system configuration and user management, and intuitively display the detection results and robot status information in the form of charts. The alarm and report automatic generation module is used to receive confirmed defect alarm information and notify relevant personnel, and can also automatically generate inspection reports. The model repository and incremental learning management module are used to store the rail defect inference engine and the rail normal state statistical model under different road segment scenarios. When the system performance drops in a new road segment scenario, the module uses the new road segment normal rail image uploaded from the edge computing layer to start the incremental learning process, optimize the rail defect inference engine parameters, and push the updated rail defect inference engine to the corresponding edge computing device through the model distribution and deployment channel. The edge computing layer includes: a data acquisition and preprocessing module, a real-time inference engine, a local decision-maker, a data and alarm management module, and a robot motion control interface; The data acquisition and preprocessing module is used to acquire railway track images through a visual sensor and perform image preprocessing. The real-time inference engine is used to load and run the rail defect inference engine, perform inference on the preprocessed rail images, and output anomaly scores and anomaly heatmaps. The local decision-maker is used to receive reasoning results and apply a dual-threshold active verification mechanism, set warning thresholds and confirmation thresholds, generate instructions to control the mobile robot to perform active verification and transmit defect information back, forming a detection closed loop. The data and alarm management module is used to package, compress, and encrypt defect information in preparation for transmission back to the cloud platform. The robot motion control interface is used to send motion control commands to the mobile robot according to the instructions generated by the local decision-maker. The terminal execution layer includes: sensing and perception components, motion and execution components, and communication and energy components; The sensing and perception components include a visual sensor and a positioning and attitude sensor; The visual sensor is used to acquire images of the railway track; the positioning and attitude sensor is used to acquire the pose information of the mobile robot. The motion and execution component is used to execute motion control commands from the edge computing layer, enabling the mobile robot to complete movement and pose adjustment. The communication and energy components are used to communicate data with the cloud platform and edge computing layer, and to provide energy for the entire system.

9. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the rail defect detection method based on a mobile robot as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the rail defect detection method based on a mobile robot as described in any one of claims 1-7.

Images