A crack detection method and system for double-block sleepers

Abstract

The present application provides a crack detection method and system for a dual-block sleeper, which relates to the technical field of rail transit disease detection. The method comprises: inputting multi-view images into a pre-trained multi-view fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper; fusing ultrasonic detection data and infrared thermal imaging data, as well as the initial three-dimensional reconstruction model, to obtain a target three-dimensional reconstruction model; extracting crack features of different scales from the target three-dimensional reconstruction model, and identifying and marking suspected crack areas based on the extracted crack features of all scales; using a deep learning model to detect real cracks in the suspected crack area; in the presence of multiple real cracks, analyzing the crack distribution, evaluating the damage degree of the dual-block sleeper, and generating a crack detection report for the dual-block sleeper based on the damage degree evaluation results. The technical solution provided by the present application improves the efficiency and effectiveness of crack detection.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of rail transit defect detection, and in particular to a crack detection method and system for a double-block sleeper. Background Art

[0002] Double-block sleeper is a new type of sleeper, mainly composed of two independent concrete blocks connected by an elastic pad in the middle.

[0003] This design not only effectively disperses train loads, improves track stability and comfort, but also reduces the manufacturing and maintenance costs of sleepers.

[0004] Therefore, double-block sleepers are widely used in high-speed railways and urban rail transit systems.

[0005] However, in order to ensure the quality and performance of twin-block sleepers, timely detection of cracks is particularly important.

[0006] Currently, the more advanced detection solution uses a fusion of laser 3D scanning and ultrasonic detection data to inspect sleepers and identify sleeper cracks.

[0007] The current method, during the spatial fusion of 3D and 2D inspection data, lacks the deep feature interaction of multimodal data, making it difficult to accurately establish the 3D correspondence between surface deformation and internal cracks. This results in low detection efficiency and makes it difficult to detect subtle cracks, including cracks within the sleeper. Therefore, a highly efficient crack detection method for dual-block sleepers that can effectively identify various crack types is urgently needed. Summary of the Invention

[0008] The embodiments of the present application provide a crack detection method and system for a dual-block sleeper, so as to solve the problems of low efficiency and poor effectiveness of crack detection in the prior art.

[0009] In a first aspect, an embodiment of the present application provides a crack detection method for a dual-block sleeper, comprising:

[0010] Acquire multi-view images, ultrasonic testing data, and infrared thermal imaging data for twin-block sleepers;

[0011] Inputting the multi-view images into a pre-trained multi-view fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper;

[0012] fusing the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model;

[0013] Using a target filter, extracting crack features of different scales from the target three-dimensional reconstructed model, and identifying and marking suspected crack areas based on the extracted crack features of all scales;

[0014] Using a deep learning model to detect real cracks in the suspected crack area to obtain the crack position, crack size and shape of the real cracks;

[0015] In the presence of multiple real cracks, the crack distribution is analyzed, and the damage degree of the dual-block sleeper is evaluated based on the crack position, crack size and shape, as well as the crack distribution of each real crack. A crack detection report for the dual-block sleeper is generated based on the damage degree evaluation results.

[0016] Optionally, fusing the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model includes:

[0017] Preprocessing the ultrasonic detection data to obtain preprocessed ultrasonic detection data, calculating wavelet coefficients of each layer based on the preprocessed ultrasonic detection data using wavelet transform combined with multi-resolution analysis to obtain feature information of different scales, generating feature extraction results, and applying short-time Fourier transform based on the feature extraction results to extract time-frequency domain features of the ultrasonic detection data to obtain a time-frequency domain feature map;

[0018] Perform temperature correction on the infrared thermal imaging data to obtain corrected infrared thermal imaging data, and use cluster analysis technology to identify abnormal points in the infrared thermal imaging data to obtain abnormality detection results;

[0019] Based on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial three-dimensional reconstruction model, multimodal feature fusion is performed through feature splicing technology and attention mechanism to generate an intermediate three-dimensional reconstruction model;

[0020] According to the time-frequency domain feature maps and the anomaly detection results, decision-level fusion is performed in combination with ensemble learning and Bayesian technology to obtain a decision-level fusion result. The intermediate 3D reconstruction model is optimized according to the decision-level fusion result to obtain a target 3D reconstruction model.

[0021] Optionally, the generating an intermediate 3D reconstruction model by performing multimodal feature fusion based on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model through feature splicing technology and an attention mechanism includes:

[0022] Using an attention mechanism, combined with historical attention weights of each modal feature, the initial attention weights of the modal features corresponding to the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model are calculated;

[0023] According to the relationship between different modal features, the attention weight of each modal feature is adjusted to obtain the target attention weight of each modal feature;

[0024] A feature stitching technique is used to perform weighted stitching processing on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial three-dimensional reconstruction model according to the target attention weight of each modal feature to obtain a multi-modal weighted feature vector, and an intermediate three-dimensional reconstruction model is generated based on the multi-modal weighted feature vector.

[0025] Optionally, performing decision-level fusion based on the time-frequency domain feature map and the anomaly detection result in combination with ensemble learning and Bayesian technology to obtain a decision-level fusion result, and optimizing the intermediate 3D reconstruction model based on the decision-level fusion result to obtain a target 3D reconstruction model includes:

[0026] Based on ensemble learning, multiple base classifiers are constructed. According to the time-frequency domain feature graphs and anomaly detection results, different machine learning algorithms are used to train each base classifier separately to generate prediction results for each base classifier.

[0027] Calculate the prior probability and likelihood function of each base classifier, use the Bayesian algorithm to calculate the posterior probability, and obtain the decision-level fusion result based on the posterior probability and the prediction results of each base classifier;

[0028] The parameters of the intermediate 3D reconstruction model are adjusted according to the decision-level fusion result to obtain a target 3D reconstruction model corresponding to the adjusted parameters.

[0029] Optionally, the deep learning model includes a target detection model and a speech segmentation model. The deep learning model is used to detect real cracks in the suspected crack area to obtain the crack position, crack size, and shape of the real cracks, including:

[0030] Use the pre-trained target detection model and irregular recognition box to determine the boundary data of the real crack;

[0031] According to the boundary data of the real crack, combined with a pre-trained semantic segmentation model, the image in the suspected crack area is semantically segmented to obtain a pixel-level mask of the real crack;

[0032] Based on the pixel-level mask of the real crack, the shape of the real crack is determined, and the pixel-level length and width of the real crack in the image within the suspected crack area are calculated;

[0033] The actual length and width of the real crack are determined based on the pixel-level length and width of the real crack in the image of the suspected crack area, combined with the conversion relationship and proportional relationship between the image three-dimensional coordinate system and the spatial coordinate system.

[0034] Optionally, the method of using a pre-trained target detection model in combination with an irregular recognition frame to determine boundary data of a real crack includes:

[0035] Use the pre-trained object detection model to detect the boundaries of real cracks in the suspected crack area through irregular recognition boxes;

[0036] The position bias value of the irregular recognition frame is calculated using the regression bias function, and the position perception of the irregular recognition frame is determined using the spatial attention mechanism. The weighted sum of the position bias value and the position perception is determined as the position difference. The position difference is used to correct the boundary of the real crack and obtain the boundary data of the real crack.

[0037] Optionally, performing semantic segmentation on the image in the suspected crack area based on the boundary data of the real crack in combination with a pre-trained semantic segmentation model to obtain a pixel-level mask of the real crack includes:

[0038] Based on the boundary data of the real crack, a pre-trained semantic segmentation model is used to classify each pixel in the image of the suspected crack area to achieve semantic segmentation of the image in the suspected crack area, and a pixel-level label map is obtained. Each pixel-level mask in the pixel-level label map is used to indicate whether the corresponding pixel is a crack or not.

[0039] Adjust the pixel-level mask in the pixel-level label map to obtain an adjusted pixel-level label map so that the real cracks are continuous and complete;

[0040] The adjusted pixel-level label map is optimized by combining the prior knowledge of cracks and the context information of the real cracks to obtain the pixel-level mask of the real cracks.

[0041] In a second aspect, an embodiment of the present application provides a crack detection system for a dual-block sleeper, comprising:

[0042] An acquisition module is used to acquire multi-view images, ultrasonic detection data, and infrared thermal imaging data of the dual-block sleeper;

[0043] A generation module, configured to input the multi-view images into a pre-trained multi-view fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper;

[0044] a fusion module, configured to fuse the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model;

[0045] an extraction and recognition module, configured to extract crack features of different scales from the target three-dimensional reconstructed model using a target filter, and identify and mark suspected crack areas based on the extracted crack features of all scales;

[0046] A detection module, configured to detect real cracks in the suspected crack area using a deep learning model to obtain the crack position, crack size, and shape of the real cracks;

[0047] The assessment generation module is used to analyze the crack distribution in the presence of multiple real cracks, and to assess the damage degree of the dual-block sleeper based on the crack position, crack size and shape, as well as the crack distribution of each of the real cracks, and to generate a crack detection report for the dual-block sleeper based on the damage degree assessment results.

[0048] In a third aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a crack detection method for a dual-block sleeper as described in any one of the first aspects.

[0049] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a crack detection method for a dual-block sleeper as described in any one of the first aspects.

[0050] An embodiment of the present application provides a crack detection method for a dual-block sleeper, the method comprising: acquiring multi-perspective images, ultrasonic detection data, and infrared thermal imaging data of the dual-block sleeper; inputting the multi-perspective images into a pre-trained multi-perspective fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper; fusing the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model; using a target filter to extract crack features of different scales from the target three-dimensional reconstruction model, and identifying and marking suspected crack areas based on the extracted crack features of all scales; using a deep learning model to detect real cracks in the suspected crack area to obtain the crack position, crack size, and shape of the real cracks; in the presence of multiple real cracks, analyzing the crack distribution, and evaluating the damage degree of the dual-block sleeper based on the crack position, crack size, shape, and crack distribution of each real crack, and generating a crack detection report for the dual-block sleeper based on the damage degree evaluation result.

[0051] The embodiment of the present application achieves high-precision, all-round detection of cracks in sleepers by integrating multimodal data and combining deep learning and three-dimensional reconstruction technology. Specifically, this method can not only generate a detailed initial three-dimensional reconstruction model, but also accurately identify suspected crack areas through multi-scale fusion and feature extraction, and further accurately locate the position, size and shape of the real cracks. In addition, the embodiment of the present application can comprehensively evaluate the degree of damage to the double-block sleepers by conducting a comprehensive analysis of the distribution of multiple cracks, thereby providing a scientific basis for maintenance decisions and ensuring the safety and reliability of railway infrastructure. The crack detection report finally generated not only improves the detection efficiency and accuracy, but also enhances the robustness and adaptability of the crack detection system of the double-block sleepers, and is suitable for complex and changeable practical application scenarios.

[0052] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A flow chart of a crack detection method for a dual-block sleeper provided in an embodiment of the present application;

[0055] Figure 2 A schematic structural diagram of a crack detection system for a dual-block sleeper provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0058] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0060] The crack detection method for a dual-block sleeper provided in this embodiment can be performed under exemplary environmental conditions, equipment conditions, and detection conditions.

[0061] Among them, environmental conditions include light conditions, temperature conditions, humidity conditions and cleaning conditions. Specifically, the setting of light conditions in the embodiment of the present application can ensure that the light is sufficient and uniform when collecting multi-perspective images, and avoid strong direct light or shadows affecting the image quality of multi-perspective images. The setting of temperature conditions in the embodiment of the present application can ensure that the collection of infrared thermal imaging data is carried out under stable temperature conditions, and avoid distortion of thermal imaging data due to excessive temperature differences. The setting of humidity conditions in the embodiment of the present application can keep the detection environment dry and avoid the influence of moisture on ultrasonic detection data. The setting of cleaning conditions in the embodiment of the present application can ensure that the surface of the sleeper is clean, without obvious dust, oil and other impurities, so as to ensure the accuracy of multi-perspective images and ultrasonic detection data.

[0062] Equipment conditions include a multi-view camera, ultrasonic detection equipment, infrared thermal imager, computing equipment and three-dimensional reconstruction software. Specifically, the multi-view camera in the embodiment of the present application may refer to a high-resolution camera that can take images of the dual-block sleeper from multiple perspectives. The ultrasonic detection equipment may refer to a high-precision ultrasonic flaw detector that can accurately detect defects inside the dual-block sleeper. The infrared thermal imager may refer to a high-sensitivity infrared thermal imager that can capture the temperature distribution on the surface of the sleeper. The computing equipment may refer to a high-performance computer equipped with sufficient storage space and computing power for image processing and data analysis. The three-dimensional reconstruction software may refer to a three-dimensional reconstruction software that supports multi-view fusion, which is used to generate an initial three-dimensional reconstruction model of the dual-block sleeper.

[0063] The detection condition can be set as the detection condition of one double-block sleeper, or can be set as the detection condition of four consecutive double-block sleepers to be deployed on a track. Therefore, when the embodiment of the present application performs crack detection on the double-block sleepers, the detection can be performed under two different detection conditions. This embodiment can perform crack detection only once in one of the detection conditions, or can perform crack detection twice in both detection conditions. Among them, the first detection condition is the case where only one double-block sleeper is considered. The second detection condition is the case where four consecutive double-block sleepers on each track need to be considered. Therefore, this embodiment can set up an experimental track under the second detection condition, and perform crack detection on four double-block sleepers with four double-block sleepers as a unit, thereby improving the accuracy of double-block sleeper detection. The reasons are: (1) The four consecutive double-block sleepers consider the joint action of multiple double-block sleepers, which can provide a more comprehensive perspective to evaluate the stability of the track. (2) By checking the four consecutive double-block sleepers, it is possible to better evaluate whether the connection between multiple double-block sleepers and between the double-block sleepers and the rails is uniform, which is very important for ensuring the smooth operation of the train. (3) The condition of four consecutive bi-block sleepers as a unit directly affects the overall structural strength of the track. Damage to any one bi-block sleeper may affect the function of the entire unit, and thus affect the safety of the track. (4) Testing four consecutive bi-block sleepers can help identify potential problems in advance, such as wear, cracks or other forms of damage, so that preventive measures can be taken to avoid larger failures.

[0064] It should be noted that, in both the first and second detection situations, the embodiments of the present application can adopt the following crack detection method for a double-block sleeper.

[0065] Figure 1 A flow chart of a crack detection method for a double-block sleeper provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes:

[0066] S11. Acquire multi-view images, ultrasonic detection data, and infrared thermal imaging data of the dual-block sleeper.

[0067] Multi-view images include multiple images from different viewpoints, each containing visible light image feature data. In the first detection scenario, data is acquired for a single dual-block sleeper. In the second detection scenario, data is acquired for four dual-block sleepers at a time.

[0068] S12. Input the multi-view images into a pre-trained multi-view fusion model to generate an initial 3D reconstruction model of the dual-block sleeper.

[0069] It should be understood that to cope with different detection situations, the embodiments of the present application can provide multi-perspective fusion models tailored to the detection situation. Specifically, in the first detection situation, the embodiments of the present application use a multi-perspective fusion model corresponding to a single dual-block sleeper rail; in the second detection situation, the embodiments of the present application use a multi-perspective fusion model corresponding to four dual-block sleeper rails.

[0070] For example, the multi-view fusion model in all detection situations can include a deep learning framework and a feature pyramid network. The following analyzes the deep learning framework and the feature pyramid network respectively:

[0071] (1) The deep learning framework includes convolutional layers, residual connections, and pooling layers. For example, the expression of the convolutional layer is: ;in, It is Feature maps of images from different viewpoints.

[0072] is the convolution kernel weight matrix, size is , is the side length of the convolution kernel, is the number of input channels, is the number of output channels. It is Perspective images. It is The bias vector of the perspective image is , is an activation function, such as the ReLU function. The expression of the residual connection is: ;in, It is the feature map after adding residual connection, ResBlock is the residual block, which is used to alleviate the The gradient vanishing problem in the feature map of the image from each perspective. The expression of the pooling layer is: ;in, This is a pooled feature map obtained based on the feature map after adding residual connections. Pool is a pooling operation, such as maximum pooling or average pooling. The present embodiment uses convolutional layers to extract local features of the feature image using convolution kernels, which helps capture subtle changes in the surface of the dual-block sleeper. Residual connections can effectively solve the vanishing gradient problem in deep networks and improve the training effect of deep learning frameworks. Pooling layers can reduce the spatial dimension of the feature map, reducing the amount of computation while retaining important feature information.

[0073] (2) The feature pyramid network can set up top-down paths and lateral connections; specifically, the expression of the top-down path is: ;in, It is Feature map after layer fusion, It is the feature map of the previous layer of the pooled feature map; conv is the convolution operation, for example, the convolution operation can be used The convolution kernel performs feature fusion to ensure that feature maps at different levels can be aligned in the channel dimension. Up is an upsampling operation. For example, the upsampling operation can be implemented using bilinear interpolation or transposed convolution. The purpose is to restore the spatial resolution so that high-level features can be added to low-level features. The expression for horizontal connection is: ;in, It is the feature map after horizontal connection, conv is the convolution operation, for example, you can use Convolution kernels perform feature alignment. In this embodiment of the application, in a top-down path, high-level feature maps are fused with low-level feature maps through upsampling operations, enhancing the model's ability to perceive features at different scales. Furthermore, feature maps of the same scale are fused through horizontal connections to retain more detailed information.

[0074] S13: Fusing the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model.

[0075] The embodiment of the present application is based on a target three-dimensional reconstruction model and can combine visible light image feature data, ultrasonic detection data and infrared thermal imaging data to comprehensively reflect whether there are crack characteristics in the dual-block sleeper.

[0076] In one optional implementation, in either the first or second detection scenario, the embodiment of the present application directly executes S13 after executing S12. In another optional implementation, in the second detection scenario, after executing S12, a differential analysis can be performed on the ultrasonic detection data and infrared thermal imaging data of the four dual-block sleeper rails. If no abnormal data exists in the ultrasonic detection data of the four dual-block sleeper rails, and no abnormal data exists in the infrared thermal imaging data of the four dual-block sleeper rails, it is determined that no cracks exist in the four dual-block sleeper rails, and subsequent steps are skipped. Otherwise, execution continues with S13 to generate the target three-dimensional reconstruction model.

[0077] S14. Use a target filter to extract crack features of different scales from the target 3D reconstructed model, and identify and mark suspected crack areas based on the extracted crack features of all scales.

[0078] Wherein, in step S14, the target filter is an adaptive filter or a multi-scale Gaussian filter, wherein the adaptive filter is used to dynamically adjust the filter parameters. Specifically, the embodiment of the present application can dynamically adjust the parameters of the adaptive filter according to the material and surface characteristics of the dual-block sleeper, thereby improving the robustness of the crack feature extraction. At the same time, the filter can also extract crack features at different scales to ensure that cracks of different sizes and shapes are captured. The multi-scale Gaussian filter is used for multi-scale analysis. Specifically, the embodiment of the present application can extract crack features at different scales through a multi-scale Gaussian filter to ensure that cracks of different sizes and shapes are captured. At the same time, the multi-scale Gaussian filter can also extract the edge and texture features of the cracks, thereby improving the accuracy of crack identification.

[0079] S15. Use a deep learning model to detect real cracks in the suspected crack area to obtain the crack position, crack size and shape of the real cracks.

[0080] S16. In the presence of multiple real cracks, analyze the crack distribution, and evaluate the damage degree of the twin-block sleeper based on the crack position, crack size and shape of each real crack, as well as the crack distribution. Generate a crack detection report for the twin-block sleeper based on the damage degree evaluation results.

[0081] The embodiment of the present application can make compliance judgments according to the preset inspection standards, and then automatically issue an inspection report. Therefore, step S16 includes the following steps: Step 161, obtain the inspection standard file, and identify the inspection standards corresponding to each degree of damage based on the inspection standard file. Step 162, based on the inspection standards corresponding to each degree of damage identified from the inspection standard file, as well as the crack position, crack size and shape, and crack distribution, determine the corresponding degree of damage. Exemplarily, mild damage: fewer cracks, and the length and width are within the safe range. Moderate damage: a large number of cracks, and the length or width of some cracks is close to the critical value. Severe damage: dense cracks, long cracks or wide cracks, which may affect the structural integrity of the sleeper. Step 163, automatically generate a crack detection report. Exemplarily, the crack detection report can briefly introduce the purpose, method and standard of the detection, and can also list the number, location, length, width and other information of all detected real cracks; display the distribution of real cracks on the sleeper, and give the damage degree and treatment suggestions for the double-block sleeper. Exemplarily, the generation of the crack detection report includes two processes: damage degree assessment and automatic report generation:

[0082] The following testing standards can be used to assess the extent of damage: ;in, Is the damage degree of the double-block sleeper is the number of cracks, is the crack length, is the crack width, , They are the crack number threshold, crack length threshold, and crack width threshold corresponding to mild damage; The thresholds for the number of cracks, length of cracks, and width of cracks corresponding to severe damage are respectively. The embodiment of the present application can assess the damage degree of a bi-block sleeper by setting different thresholds and comprehensively considering the number, length, and width of cracks, thereby providing a reasonable basis for decision-making.

[0083] The report content includes at least one of the following: (1) Inspection purpose: to ensure the safety and reliability of the sleeper. (2) Methods and standards: a detailed description of the inspection methods and standards used. (3) Crack information: a list of the number of cracks detected, the location of the cracks, the length and width of the cracks, and the shape of the cracks. (4) Crack distribution: a map showing the distribution of cracks on the sleeper. (5) Damage degree and treatment suggestions: specific treatment suggestions are made based on the evaluation results. The embodiment of the present application can provide comprehensive crack detection results and treatment suggestions through detailed report content, helping maintenance personnel to take timely measures to ensure the safe operation of the track.

[0084] The above process can provide a crack detection method for dual-block sleepers, ensuring its effectiveness and reliability in practical applications. It not only improves the accuracy of crack detection, but also enhances the robustness and generalization ability of the model, making it suitable for various complex application scenarios.

[0085] By executing steps S11 to S16, this embodiment of the present application integrates multi-view images, ultrasonic inspection data, and infrared thermal imaging data, and combines them with a deep learning framework and feature pyramid network to achieve high-precision, comprehensive crack detection for dual-block sleepers. By acquiring and fusing visible light images, ultrasonic inspection data, and infrared thermal imaging data, the system captures sleeper status information from multiple dimensions, ensuring comprehensive and accurate crack detection. Adopting appropriate multi-view fusion models for different inspection scenarios (single or four sleepers) enhances the system's adaptability and flexibility. A top-down path and lateral connection mechanism enable the model to better capture features at different scales, enhancing its ability to recognize complex crack morphologies. The introduction of a top-down path and lateral connection demonstrates how to fuse features at different levels through specific mathematical expressions, improving the model's multi-scale perception capabilities. A multi-scale Gaussian filter extracts crack features at different scales, ensuring that cracks of various sizes and shapes are captured. Based on pre-set inspection criteria, the system automatically assesses damage severity and generates detailed crack inspection reports, streamlining maintenance processes and improving work efficiency. Based on preset inspection standards, it automatically assesses the damage extent and generates detailed crack inspection reports, simplifying the maintenance process and improving work efficiency.

[0086] How to accurately fuse ultrasonic detection data, infrared thermal imaging data, and the initial 3D reconstruction model is the key to obtaining an accurate target 3D reconstruction model. For example, in one possible embodiment, S13, fusing the ultrasonic detection data, infrared thermal imaging data, and the initial 3D reconstruction model to obtain the target 3D reconstruction model, includes:

[0087] Step 131: Preprocess the ultrasonic detection data to obtain preprocessed ultrasonic detection data. Based on the preprocessed ultrasonic detection data, use wavelet transform combined with multi-resolution analysis to calculate the wavelet coefficients of each layer to obtain feature information at different scales and generate feature extraction results. Based on the feature extraction results, apply short-time Fourier transform to extract the time-frequency domain features of the ultrasonic detection data to obtain a time-frequency domain feature map. It should be understood that the time-frequency domain feature map can provide information about how cracks change over time and frequency, which is very useful for understanding the development pattern of cracks. The time-frequency domain feature map can be input as an additional feature into deep learning or other machine learning models to help improve the accuracy of crack detection. In addition, the time-frequency domain feature map can also be used to assist in the qualitative and quantitative analysis of cracks, such as evaluating the length, width, and development rate of cracks.

[0088] This step is the ultrasonic detection data processing flow, which involves three steps: data preprocessing, feature extraction and short-time Fourier transform.

[0089] (1) Data preprocessing includes denoising, smoothing and other processing methods. Specifically, denoising uses the following formula: ;in, It is the ultrasonic detection data after denoising. DWT is discrete wavelet transform, which is used to remove high-frequency noise. is the ultrasonic detection data before denoising; is the smoothing coefficient, which is used to control the strength of the median filter; It is a median filter, which is used to remove impulse noise from ultrasonic detection data; is the threshold coefficient, which is used to control the intensity of wavelet threshold denoising; It is wavelet threshold denoising, using wavelet threshold Remove low-frequency noise from ultrasonic detection data. Smoothing can be done using the following formula: ;

[0090] in, is the smoothed ultrasonic detection data; GaussianFilter is a Gaussian filter used to smooth the signal; is the standard deviation of the Gaussian filter; laplacian It is the Laplace operator, which is used to enhance the edge information of ultrasonic detection data; is the sharpening factor, which is used to control the strength of the Laplace operator; is the bilateral filter coefficient, which is used to control the impact of bilateral filtering; It is a bilateral filter, using the spatial standard deviation and range standard deviation Smoothing. The present embodiment combines discrete wavelet transform, median filtering, and wavelet threshold denoising to effectively remove high-frequency and impulse noise, improving signal quality. Ultrasonic detection data is smoothed and edge-enhanced using Gaussian filters, Laplacian operators, and bilateral filters while retaining detailed features.

[0091] (2) Feature extraction includes wavelet transform and multi-resolution analysis. Specifically, feature extraction can be performed using the following formula: ;

[0092] in, It is The wavelet coefficients of the layer; It's time The pre-processed ultrasonic detection data at 1000 Hz; is the wavelet basis function, is the complex conjugate of the wavelet basis function; is the enhancement coefficient, which is used to control the intensity of the additional wavelet transform; is an additional wavelet transform using a different wavelet basis function right Perform feature extraction; is the multi-resolution analysis coefficient, which is used to control the intensity of multi-resolution analysis; Used to represent multi-resolution analysis, which can use wavelet basis functions of different scales Perform feature extraction. The embodiment of the present application combines multiple wavelet basis functions and multi-resolution analysis to extract features of different scales and improve the richness of features.

[0093] (3) Short-time Fourier transform can be expressed using the following formula: ;

[0094] in, It is the result of short-time Fourier transform, which is used to represent the time-frequency domain characteristics of ultrasonic detection data. is the frequency, It’s time; is the integration variable, used to represent time, is the ultrasonic signal in time The value at is used to represent the ultrasonic detection data; is the window function; is the enhancement coefficient, which is used to control the intensity of the additional short-time Fourier transform; is an additional short-time Fourier transform, using a different Another window function Perform feature extraction; is the short-time Fourier transform coefficient, which is used to control the intensity of the short-time Fourier transform. It is another window function; short_time_fourier_transform is the short-time Fourier transform, using different window functions Perform feature extraction. The embodiment of the present application extracts time-frequency domain features through different window functions to enhance the model's ability to perceive different frequency components.

[0095] Step 132: Temperature correction is performed on the infrared thermal imaging data to obtain corrected infrared thermal imaging data. Cluster analysis technology is used to identify abnormal points in the infrared thermal imaging data to obtain abnormality detection results. It should be understood that the abnormality detection results can help locate the specific locations where cracks or other structural defects may exist. The abnormality detection results are very important for guiding subsequent detailed inspections or maintenance work. At the same time, the abnormality detection results can also be used to train or adjust the machine learning model to make it better adapt to specific types of abnormal situations, thereby improving the generalization ability and robustness of the model. In addition, when generating the final crack detection report, the abnormality detection results can serve as an important reference to help evaluate the degree of damage to the double-block sleeper and provide corresponding treatment suggestions.

[0096] It should be understood that infrared thermal imaging data analysis includes temperature correction and anomaly detection processes. Among them, temperature correction can be achieved using the following formula: ;

[0097] in, is the corrected temperature, is the original temperature, is the ambient temperature, is the reference temperature; is the temperature compensation coefficient, is the humidity influence coefficient, humidity is the humidity correction function, which is used to consider the effect of humidity on temperature measurement; is the radiation correction coefficient, which is used to control the intensity of the radiation correction. is the radiation correction, using the emissivity and transmittance Correct the original temperature. This embodiment combines ambient and reference temperatures for correction, eliminating the effects of environmental factors. It also considers the impact of humidity on temperature measurement to improve the accuracy of temperature correction. Radiation correction using emissivity and transmittance further improves temperature measurement accuracy.

[0098] To achieve anomaly detection, this embodiment can use cluster analysis technology, which can use the following formula: ;in, It is and The comprehensive distance between samples, and They are and The first sample dimensional features, is the feature dimension, Is the similarity weight, used to control the influence of cosine similarity, cosine_similarity is the cosine similarity, which is used to measure the directional similarity between samples. is the Mahalanobis distance weight, which is used to control the influence of Mahalanobis distance. is the Mahalanobis distance, using the covariance matrix The embodiment of the present application combines Euclidean distance, cosine similarity and Mahalanobis distance, comprehensively considers the distance and direction similarity between samples, and improves the accuracy of clustering.

[0099] Step 133: Based on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model, multimodal feature fusion is performed through feature splicing technology and attention mechanism to generate an intermediate 3D reconstruction model.

[0100] As a possible implementation, step 133, based on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model, performs multimodal feature fusion using feature stitching technology and an attention mechanism to generate an intermediate 3D reconstruction model, including:

[0101] Step a1: Use the attention mechanism and combine the historical attention weights of each modal feature to calculate the initial attention weights of the modal features corresponding to the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model.

[0102] Step a2: According to the relationship between different modal features, adjust the attention weight of each modal feature to obtain the target attention weight of each modal feature.

[0103] It should be understood that multimodal feature fusion may include an attention mechanism and a feature splicing technique; wherein the attention mechanism may adopt the following formula: ;

[0104] in, It is The target attention weight of the modality feature, where , hour, represents the first modal feature corresponding to the ultrasonic detection data, hour, represents the second modal feature corresponding to the corrected infrared thermal imaging data, hour, Represents the third modality feature corresponding to the visible light image feature data in the initial 3D reconstruction model, It is modal feature scoring function, the function expression can be dot product or weighted sum, is the parameter in the modal feature scoring function, is the context attention coefficient, which is used to control the influence of context attention, is the function for calculating the relationship between different modal features, The embodiment of the present application dynamically adjusts the importance of different modal features through the attention mechanism, improves the model's sensitivity to key features, and introduces a contextual attention mechanism to further enhance the model's understanding of contextual information.

[0105] Step a3: Using feature stitching technology, according to the target attention weight of each modal feature, the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model are weighted stitched together to obtain a multimodal weighted feature vector, and an intermediate 3D reconstruction model is generated based on the multimodal weighted feature vector.

[0106] Step a3 can use the following formula: ;

[0107] in, is the multimodal weighted feature vector, , , are the target attention weights of the first modality feature, the second modality feature, and the third modality feature, respectively. , , The embodiment of the present application performs weighted concatenation processing on different modal features to form a multimodal feature vector, thereby improving the richness of the input information of the model.

[0108] By executing steps a1 to a3, the embodiment of the present application introduces an attention mechanism and a contextual attention mechanism to dynamically adjust the importance of different modal features, and generates a multimodal weighted feature vector through weighted splicing technology, which has the following advantages: by calculating the initial attention weight and adjusting the target attention weight according to the relationship between the modal features, the model can dynamically focus on the most important features, thereby improving the sensitivity to key information. The embodiment of the present application introduces a contextual attention mechanism, so that the model can not only focus on a single modal feature, but also consider the relationship between different modal features, thereby enhancing the ability to understand complex scenes. The embodiment of the present application generates a multimodal weighted feature vector through weighted splicing processing, integrates data from different modalities, provides richer input information, and helps to improve the detection accuracy of the model. The calculation method of the attention weight includes parameters in the scoring function, the contextual attention coefficient, and the relationship calculation function, making the entire process highly controllable and adjustable. The embodiment of the present application ensures the controllability and flexibility of the entire process through specific mathematical expressions and parameter settings, is suitable for more diverse application scenarios, improves the technical depth of the crack detection method, and enhances the robustness and accuracy of the system, providing more reliable technical support for practical applications.

[0109] Step 134: Based on the time-frequency domain feature graph and the anomaly detection results, decision-level fusion is performed in combination with ensemble learning and Bayesian technology to obtain a decision-level fusion result. The intermediate 3D reconstruction model is optimized based on the decision-level fusion result to obtain a target 3D reconstruction model.

[0110] As a possible implementation method, step 134 performs decision-level fusion based on the time-frequency domain feature graph and the anomaly detection result by combining ensemble learning and Bayesian technology to obtain a decision-level fusion result, and optimizes the intermediate 3D reconstruction model based on the decision-level fusion result to obtain the target 3D reconstruction model, including:

[0111] Step b1: Construct multiple base classifiers based on ensemble learning. Use different machine learning algorithms to train each base classifier according to the time-frequency domain feature graph and anomaly detection results to generate prediction results for each base classifier.

[0112] It should be understood that integrated learning can be expressed as ensemble ,in, are the prediction results of each base classifier in the m base classifiers. The detection results of multiple base classifiers are combined to improve the robustness and accuracy of the detection results through weighted voting or Bayesian fusion.

[0113] Step b2: Calculate the prior probability and likelihood function of each base classifier, calculate the posterior probability using the Bayesian algorithm, and obtain the decision-level fusion result based on the posterior probability and the prediction results of each base classifier.

[0114] It should be understood that decision-level fusion may include integrated learning technology and Bayesian fusion technology. The embodiment of the present application further enhances the model's ability to handle the uncertainty of decision-level fusion results by introducing the Bayesian algorithm.

[0115] Step b3: adjusting the parameters of the intermediate 3D reconstruction model according to the decision-level fusion result to obtain a target 3D reconstruction model corresponding to the adjusted parameters.

[0116] By executing steps b1 to b3, the embodiment of the present application provides a more specific implementation method of integrated learning and Bayesian technology. By constructing multiple base classifiers and using different machine learning algorithms for training, different types of features and patterns can be captured, thereby improving the diversity and generalization ability of the model. This section explains in detail the role of the Bayesian algorithm in calculating prior probabilities, likelihood functions, and posterior probabilities, and enhances the model's ability to handle uncertainty. This section clarifies how to adjust the parameters of the intermediate three-dimensional reconstruction model based on the decision-level fusion results, ensuring the accuracy and adaptability of the target three-dimensional reconstruction model. In summary, this process improves the technical depth of the crack detection method, enhances the robustness and accuracy of the system, and is suitable for more diverse application scenarios. In addition, by introducing specific Bayesian algorithms and parameter adjustment steps, the entire process is made more transparent and controllable, providing a solid foundation for subsequent research and applications.

[0117] By executing steps 131 to 134, the embodiment of the present application performs more in-depth preprocessing and feature extraction on the ultrasonic detection data and infrared thermal imaging data, and enhances the technical depth and practical application effect of the crack detection method through high-level multimodal feature fusion and decision-level fusion. This method not only improves the accuracy of crack detection, but also enhances the robustness and generalization ability of the system, and is suitable for more diverse application scenarios. An attention mechanism is introduced to highlight important features, making multimodal data fusion more efficient and targeted. Combining ensemble learning and Bayesian technology for decision-level fusion can comprehensively consider multiple information sources and improve the reliability and accuracy of the final decision. The intermediate three-dimensional reconstruction model is optimized according to the decision-level fusion results to obtain a more accurate target three-dimensional reconstruction model, thereby improving the accuracy and reliability of crack detection. In addition, the embodiment of the present application also provides important support for the study of crack development patterns and damage assessment by increasing the recognition of time-frequency domain features and abnormal points.

[0118] In a possible embodiment, the deep learning model includes a target detection model and a speech segmentation model. S15: Using the deep learning model to detect real cracks in the suspected crack area to obtain the crack position, crack size, and shape of the real cracks includes:

[0119] Step 151: Use the pre-trained target detection model in combination with the irregular recognition frame to determine the boundary data of the real crack.

[0120] As a possible implementation, step 151, using a pre-trained target detection model in combination with an irregular recognition frame to determine boundary data of a real crack, includes:

[0121] Step c1: Using a pre-trained object detection model, detect the boundaries of real cracks in the suspected crack area using an irregular recognition frame. It should be understood that the irregular recognition frame can be a deformed frame obtained by removing the non-crack area from the rectangular frame.

[0122] Step c2: Use the regression bias function to calculate the position bias value of the irregular recognition frame, use the spatial attention mechanism to determine the position perception of the irregular recognition frame, determine the weighted sum of the position bias value and the position perception as the position difference, and use the position difference to correct the boundary of the real crack to obtain the boundary data of the real crack.

[0123] For example, the target detection model may include the design of bounding box regression and loss function. The bounding box regression may adopt the following formula: ;

[0124] in, is the boundary data before the real crack correction, is the boundary data after the real crack correction, is the position offset value of the irregular recognition frame, is the position perception of the irregular recognition frame, is the bias coefficient, which is used to control the impact of regression bias. is the spatial attention coefficient, which is used to control the influence of spatial attention.

[0125] By executing steps c1 to c2, the embodiment of the present application uses the regression bias function and the spatial attention mechanism to correct the prediction bias and improve the accuracy of the irregular recognition frame.

[0126] The loss function of the target detection model can be expressed as follows: ;

[0127] in, is the total loss of the deep learning model. It is a classification loss, such as cross-picking loss; is the regression loss, such as smooth L1 loss, and are the classification loss weight and regression loss weight; Is the balance loss, used to balance the proportion of negative samples, Is the balance coefficient, used to control the impact of balance loss, focal_loss It is the focal loss between the real detection results and the predicted detection results, which is used to solve the problem of class imbalance. is the focus loss coefficient, which is used to control the effect of focus loss.

[0128] The design of the loss function of the deep learning model in the embodiment of the present application takes into account the balance loss and focal loss, which can ensure that the deep learning model has good generalization ability on different types of samples.

[0129] Step 152: Based on the boundary data of the real crack and in combination with a pre-trained semantic segmentation model, semantic segmentation is performed on the image in the suspected crack area to obtain a pixel-level mask of the real crack.

[0130] As a possible implementation, step 152, based on the boundary data of the real crack and in combination with a pre-trained semantic segmentation model, performs semantic segmentation on the image within the suspected crack area to obtain a pixel-level mask of the real crack, including:

[0131] Step d1: Based on the boundary data of the real crack, use the pre-trained semantic segmentation model to classify each pixel in the image in the suspected crack area to achieve semantic segmentation of the image in the suspected crack area and obtain a pixel-level label map. Each pixel-level mask in the pixel-level label map is used to indicate whether the corresponding pixel is a crack or not.

[0132] Step d2: Adjust the pixel-level mask in the pixel-level label image to obtain an adjusted pixel-level label image so that the real cracks are continuous and complete.

[0133] Step d3: Combining the prior knowledge of cracks and the contextual information of the real cracks, the adjusted pixel-level label map is optimized to obtain the pixel-level mask of the real cracks.

[0134] The semantic segmentation model includes two aspects: pixel-level classification and loss function design of the semantic segmentation model:

[0135] Exemplarily, steps d1 to d3 are used to implement pixel-level classification, and ultimately obtain a pixel-level mask of the real crack. The calculation formula of the pixel-level mask of the real crack is as follows: ;

[0136] in, is the first pixels, is the first The pixel-level mask of pixels is used to represent the The probability that a pixel belongs to the crack class or the non-crack class, is the adjusted pixel-level label map, is the prior knowledge coefficient, which is used to control the influence of prior knowledge of cracks. is a priori knowledge function, used to introduce the priori knowledge of cracks in the double-block sleeper. It is the coefficient of contextual information, which is used to control the influence of contextual information. This is context information used to enhance the semantic segmentation model's understanding of the local context. The present embodiment introduces prior knowledge functions and context information to improve the semantic segmentation model's ability to identify various types of cracks.

[0137] The loss function of the semantic segmentation model can be expressed as follows: ;

[0138] in, is the segmentation loss of the semantic segmentation model, is the total number of pixels, , It is Pixels belong to The label of the category, Indicates cracks, Indicates non-cracks, It is Pixels belong to The predicted probability of the class, is the Dice loss, is the one-hot encoding vector of the true label, indicating the The true category of pixels, is the predicted probability distribution vector, containing the The predicted probability that pixels belong to each category, is the Lovasz-Hinge loss, used to optimize the segmentation boundary, is the weight of Dice loss, is the weight of the Lovasz-Hinge loss, which is used to control the impact of the Lovasz-Hinge loss. In the embodiment of the present application, when designing the loss function, the Dice loss and the Lovasz-Hinge loss are comprehensively considered to improve the detection accuracy of the semantic segmentation model for small cracks.

[0139] By executing steps d1 to d3, the embodiment of the present application introduces a more detailed semantic segmentation process and optimization mechanism, especially in terms of pixel-level classification, label map adjustment, and the combination of prior knowledge and contextual information. Specifically, each pixel in the suspected crack area is classified by a pre-trained semantic segmentation model to generate a pixel-level label map, thereby ensuring a fine distinction between cracks and non-cracks. By adjusting the pixel-level mask in the pixel-level label map, the continuity and integrity of the real cracks are ensured, and breaks or discontinuities are avoided. The adjustment process can remove noise and small errors, making the crack boundaries smoother and more natural. By introducing crack prior knowledge functions and contextual information, the model's ability to recognize different types of cracks is improved, especially small or complex cracks. The loss function of the semantic segmentation model comprehensively considers cross entropy loss, Dice loss, and Lovasz-Hinge loss, thereby improving the detection accuracy of small cracks.

[0140] Step 153: Based on the pixel-level mask of the real crack, determine the shape of the real crack, and calculate the pixel-level length and width of the real crack in the image within the suspected crack area.

[0141] Step 154 ​​: Determine the actual length and width of the real crack based on the pixel-level length and width of the real crack in the image within the suspected crack area, combined with the conversion relationship and proportional relationship between the image three-dimensional coordinate system and the spatial coordinate system.

[0142] By executing steps 151 to 154, the embodiment of the present application introduces a more specific and detailed processing method, especially in terms of crack boundary detection, semantic segmentation, shape determination, and actual length and width calculation. Specifically, by combining a pre-trained target detection model with an irregular recognition frame, the boundary data of the real crack can be determined more accurately, avoiding the errors that may be caused by the traditional rectangular frame. Combined with the pre-trained semantic segmentation model, the image in the suspected crack area is semantically segmented to obtain a pixel-level mask of the real crack, ensuring the refined expression of the crack position and shape. Based on the pixel-level mask of the real crack, the shape of the crack can be determined more accurately, providing reliable geometric information for subsequent analysis. According to the conversion relationship and proportional relationship between the three-dimensional coordinate system of the image and the spatial coordinate system, the pixel-level size is converted into the actual length and width, ensuring the accuracy of the measurement results. Ultimately, the technical depth of the crack detection method is improved, and the robustness and accuracy of the system are enhanced, making it suitable for more diverse application scenarios. In addition, by introducing specific processing steps and technical details, the entire process is made more transparent and controllable, providing solid technical support for practical applications.

[0143] Figure 2 A structural diagram of a crack detection system for a double-block sleeper provided in an embodiment of the present application is shown in FIG. Figure 2As shown, the system includes:

[0144] The acquisition module 21 is used to acquire multi-view images, ultrasonic detection data and infrared thermal imaging data of the dual-block sleeper.

[0145] The generation module 22 is used to input the multi-view images into a pre-trained multi-view fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper.

[0146] The fusion module 23 is used to fuse the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model.

[0147] The extraction and identification module 24 is used to extract crack features of different scales from the target 3D reconstructed model using a target filter, and identify and mark suspected crack areas based on the extracted crack features of all scales.

[0148] The detection module 25 is used to detect real cracks in the suspected crack area using a deep learning model to obtain the crack position, crack size and shape of the real cracks.

[0149] The evaluation generation module 26 is used to analyze the crack distribution in the presence of multiple real cracks, and evaluate the damage degree of the double-block sleeper based on the crack position, crack size and shape of each real crack, as well as the crack distribution, and generate a crack detection report for the double-block sleeper based on the damage degree evaluation result.

[0150] Figure 2 The crack detection system for the dual-block sleeper can be performed Figure 1 The implementation principle and technical effects of the crack detection method for a dual-block sleeper described in the illustrated embodiment will not be elaborated upon. The specific manner in which the various modules and units of the crack detection system for a dual-block sleeper in the aforementioned embodiment perform their operations has been described in detail in the embodiments of the method and will not be elaborated upon here.

[0151] In one possible design, Figure 2 The crack detection system for the dual-block sleeper of the illustrated embodiment may be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32 .

[0152] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0153] The processing component 32 is used to: obtain multi-view images, ultrasonic detection data and infrared thermal imaging data of the dual-block sleeper; input the multi-view images into a pre-trained multi-view fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper; fuse the ultrasonic detection data and infrared thermal imaging data, as well as the initial three-dimensional reconstruction model, to obtain a target three-dimensional reconstruction model; use a target filter to extract crack features of different scales from the target three-dimensional reconstruction model, and identify and mark suspected crack areas based on the extracted crack features of all scales; use a deep learning model to detect real cracks in the suspected crack area to obtain the crack position, crack size and shape of the real cracks; in the case of multiple real cracks, analyze the crack distribution, and evaluate the damage degree of the dual-block sleeper based on the crack position, crack size and shape of each real crack, as well as the crack distribution, and generate a crack detection report for the dual-block sleeper based on the damage degree evaluation result.

[0154] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0155] The storage component 31 is configured to store various types of data to support operations on the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0156] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0157] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0158] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0159] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0160] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 A crack detection method for a dual-block sleeper according to the illustrated embodiment.

[0161] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0163] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A crack detection method for a double-block sleeper, characterized in that: include: An experimental track was set up, and using four bi-block sleepers as a unit, multi-view images, ultrasonic test data, and infrared thermal imaging data were acquired from the bi-block sleepers. Inputting the multi-view images into a pre-trained multi-view fusion model to generate an initial three-dimensional reconstruction model of the dual-block sleeper, the multi-view fusion model including a deep learning framework and a feature pyramid network having a top-down path and lateral connections; performing difference analysis on the ultrasonic detection data and the infrared thermal imaging data of the four dual-block sleeper rails, respectively; if abnormal data exists in the ultrasonic detection data of the four dual-block sleeper rails, or if abnormal data does not exist in the infrared thermal imaging data of the four dual-block sleeper rails, fusing the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model; A target filter is used to extract crack features of different scales from the target 3D reconstructed model, and based on the extracted crack features of all scales, suspected crack areas are identified and marked. The target filter is an adaptive filter or a multi-scale Gaussian filter. The adaptive filter is used to dynamically adjust the filter parameters according to the material and surface characteristics of the bi-block sleeper. The target filter extracts crack features at different scales. Using a deep learning model to detect real cracks in the suspected crack area to obtain the crack position, crack size and shape of the real cracks; In the presence of multiple real cracks, the crack distribution is analyzed, and the damage degree of the dual-block sleeper is evaluated based on the crack position, crack size and shape, as well as the crack distribution of each of the real cracks. A crack detection report for the dual-block sleeper is generated based on the damage degree evaluation results.

2. The method according to claim 1, characterized in that The step of fusing the ultrasonic detection data, the infrared thermal imaging data, and the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model includes: Preprocessing the ultrasonic detection data to obtain preprocessed ultrasonic detection data, calculating wavelet coefficients of each layer based on the preprocessed ultrasonic detection data using wavelet transform combined with multi-resolution analysis to obtain feature information of different scales, generating feature extraction results, and applying short-time Fourier transform based on the feature extraction results to extract time-frequency domain features of the ultrasonic detection data to obtain a time-frequency domain feature map; Perform temperature correction on the infrared thermal imaging data to obtain corrected infrared thermal imaging data, and use cluster analysis technology to identify abnormal points in the infrared thermal imaging data to obtain abnormality detection results; Based on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial three-dimensional reconstruction model, multimodal feature fusion is performed through feature splicing technology and attention mechanism to generate an intermediate three-dimensional reconstruction model; According to the time-frequency domain feature maps and the anomaly detection results, decision-level fusion is performed in combination with ensemble learning and Bayesian technology to obtain a decision-level fusion result. The intermediate 3D reconstruction model is optimized according to the decision-level fusion result to obtain a target 3D reconstruction model.

3. The method according to claim 2, characterized in that The method generates an intermediate 3D reconstruction model by performing multimodal feature fusion based on the pre-processed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model through feature splicing technology and attention mechanism, including: Using an attention mechanism, combined with historical attention weights of each modal feature, the initial attention weights of the modal features corresponding to the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial 3D reconstruction model are calculated; According to the relationship between different modal features, the attention weight of each modal feature is adjusted to obtain the target attention weight of each modal feature; A feature stitching technique is used to perform weighted stitching processing on the preprocessed ultrasonic detection data, the corrected infrared thermal imaging data, and the visible light image feature data in the initial three-dimensional reconstruction model according to the target attention weight of each modal feature to obtain a multi-modal weighted feature vector, and an intermediate three-dimensional reconstruction model is generated based on the multi-modal weighted feature vector.

4. The method according to claim 2, characterized in that The method of performing decision-level fusion based on the time-frequency domain feature graph and the anomaly detection result in combination with ensemble learning and Bayesian technology to obtain a decision-level fusion result, and optimizing the intermediate 3D reconstruction model based on the decision-level fusion result to obtain a target 3D reconstruction model includes: Based on ensemble learning, multiple base classifiers are constructed. According to the time-frequency domain feature graphs and anomaly detection results, different machine learning algorithms are used to train each base classifier separately to generate prediction results for each base classifier. Calculate the prior probability and likelihood function of each base classifier, use the Bayesian algorithm to calculate the posterior probability, and obtain the decision-level fusion result based on the posterior probability and the prediction results of each base classifier; The parameters of the intermediate 3D reconstruction model are adjusted according to the decision-level fusion result to obtain a target 3D reconstruction model corresponding to the adjusted parameters.

5. The method according to claim 1, wherein The deep learning model includes a target detection model and a speech segmentation model. The deep learning model is used to detect the real cracks in the suspected crack area to obtain the crack position, crack size and shape of the real cracks, including: Use the pre-trained target detection model and irregular recognition box to determine the boundary data of the real crack; According to the boundary data of the real crack, combined with a pre-trained semantic segmentation model, the image in the suspected crack area is semantically segmented to obtain a pixel-level mask of the real crack; Based on the pixel-level mask of the real crack, the shape of the real crack is determined, and the pixel-level length and width of the real crack in the image within the suspected crack area are calculated; The actual length and width of the real crack are determined based on the pixel-level length and width of the real crack in the image of the suspected crack area, combined with the conversion relationship and proportional relationship between the image three-dimensional coordinate system and the spatial coordinate system.

6. The method according to claim 5, characterized in that The method uses a pre-trained target detection model and an irregular recognition frame to determine the boundary data of the real crack, including: Use the pre-trained object detection model to detect the boundaries of real cracks in the suspected crack area through irregular recognition boxes; The position bias value of the irregular recognition frame is calculated using the regression bias function, and the position perception of the irregular recognition frame is determined using the spatial attention mechanism. The weighted sum of the position bias value and the position perception is determined as the position difference. The position difference is used to correct the boundary of the real crack and obtain the boundary data of the real crack.

7. The method according to claim 5, characterized in that The method of performing semantic segmentation on the image in the suspected crack area based on the boundary data of the real crack and combining it with a pre-trained semantic segmentation model to obtain a pixel-level mask of the real crack includes: Based on the boundary data of the real crack, a pre-trained semantic segmentation model is used to classify each pixel in the image of the suspected crack area to achieve semantic segmentation of the image in the suspected crack area, and a pixel-level label map is obtained. Each pixel-level mask in the pixel-level label map is used to indicate whether the corresponding pixel is a crack or not. Adjust the pixel-level mask in the pixel-level label map to obtain an adjusted pixel-level label map so that the real cracks are continuous and complete; The adjusted pixel-level label map is optimized by combining the prior knowledge of cracks and the context information of the real cracks to obtain the pixel-level mask of the real cracks.

8. A crack detection system for a double-block sleeper, characterized in that: include: An acquisition module is used to set up an experimental track and acquire multi-view images, ultrasonic detection data, and infrared thermal imaging data of four bi-block sleepers as a unit; a generation module, configured to input the multi-view images into a pre-trained multi-view fusion model to generate an initial 3D reconstruction model of the dual-block sleeper, wherein the multi-view fusion model includes a deep learning framework and a feature pyramid network, wherein the feature pyramid network is provided with a top-down path and lateral connections; a fusion module for performing difference analysis on the ultrasonic detection data and the infrared thermal imaging data of the four dual-block sleeper rails, and if abnormal data exists in the ultrasonic detection data of the four dual-block sleeper rails, or if abnormal data does not exist in the infrared thermal imaging data of the four dual-block sleeper rails, fusing the ultrasonic detection data and the infrared thermal imaging data with the initial three-dimensional reconstruction model to obtain a target three-dimensional reconstruction model; an extraction and recognition module, configured to extract crack features of different scales from the target 3D reconstructed model using a target filter, and identify and mark suspected crack areas based on the extracted crack features of all scales. The target filter is an adaptive filter or a multi-scale Gaussian filter. The adaptive filter is configured to dynamically adjust filter parameters based on the material and surface characteristics of the bi-block sleeper. The target filter extracts crack features at different scales. A detection module, configured to detect real cracks in the suspected crack area using a deep learning model to obtain the crack position, crack size, and shape of the real cracks; The assessment generation module is used to analyze the crack distribution in the presence of multiple real cracks, and to assess the damage degree of the dual-block sleeper based on the crack position, crack size and shape, as well as the crack distribution of each of the real cracks, and to generate a crack detection report for the dual-block sleeper based on the damage degree assessment results.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a crack detection method for a dual-block sleeper as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, a crack detection method for a double-block sleeper as claimed in any one of claims 1 to 7 is implemented.

Images