Method and system for accurately detecting fine defects of surface coating putty of high iron structural member

Abstract

The application belongs to the technical field of robot grinding and polishing, and particularly discloses a method and system for precisely detecting fine defects of putty on a surface of a high-speed rail structural member. The method comprises the following steps: obtaining three-dimensional shape information of the surface of the high-speed rail structural member in a synchronous state, and depth information and luminosity information of the surface of the high-speed rail structural member, fusing the three-dimensional shape information, the depth information and the luminosity information to obtain surface topography data; fusing influences of multiple single modes on different defects to obtain nonlinear relationships between different modes and different defects, and performing defect recognition and defect feature extraction; based on the recognized defects, the defect features and a robot grinding and polishing pose, performing trajectory planning on grinding and polishing points to perform self-adaptive grinding and polishing repair on defect positions of the putty coating on the surface of the high-speed rail structural member. The application can realize precise recognition of defects of the putty coating on the surface of the high-speed rail structural member and efficient extraction of defect features, and provides support for subsequent self-adaptive grinding and polishing repair of the defects.

Description

Technical Field

[0001] This invention belongs to the field of robotic polishing technology, and more specifically, relates to a method and system for accurately detecting minute defects in the surface coating putty of high-speed rail structural components. Background Technology

[0002] High-speed rail structural components mainly include the train head and body, both characterized by their large size, complex shape, and localized structural rigidity variations. They are the most important large and complex components of the entire high-speed rail train. Due to the harsh operating environment, the train head and body are frequently exposed to various climatic conditions such as acids, alkalis, and moisture. Therefore, their surfaces require topcoat spraying to improve their corrosion resistance, weather resistance, wear resistance, and shock resistance. To ensure the adhesion of the paint material to the train head and body surfaces, putty application and sanding are usually required. Currently, putty application on high-speed rail structural components is still done manually, resulting in significant environmental pollution, long construction periods, and serious material waste. The uniformity and consistency of the putty application depend entirely on the skills, operational standards, and mood of the workers on site, making it difficult to guarantee the thickness and uniformity of the putty coating. This leads to numerous defects in the putty application, such as pitting, raised edges, and wavy stripes. Currently, defects in putty are mainly identified through manual visual inspection, followed by manual grinding and polishing for repair. However, this method suffers from low defect identification efficiency and incomplete identification, which seriously affects the continuity and effectiveness of subsequent vehicle operations. Therefore, a method that can automatically and accurately identify defects in the putty coating on the surface of high-speed rail structural components is needed.

[0003] To address the aforementioned issues, Chinese patent application CN114720476A discloses a method for detecting and identifying defects in the vehicle's exterior by modeling the vehicle to be repaired in a custom spatial coordinate system, increasing the robot's effective workspace through a coordinated layout of linear tracks and the robot, and combining camera capture, imaging, and modeling with data comparison using AI self-recognition engine software. The method also allows online control of the robot to execute repair procedures on the vehicle's defective areas. This method is characterized by its simplicity, accuracy, and efficiency. Chinese patent CN212622274U discloses a defect detection device for sanding putty on high-speed rail aluminum alloy painted vehicle bodies. By using a defect detection device and a floating sanding device at the end of a robot, including a protective cover, a smart camera, and a light source, it can detect defects on the vehicle body surface after putty application and facilitate subsequent repairs.

[0004] However, Chinese patent application CN114720476A focuses on detecting and repairing appearance defects in small cars, which is insufficient for large components like high-speed trains. Chinese patent CN212622274U uses camera photography and image processing for defect detection, but this method cannot obtain precise three-dimensional morphology of the putty coating on the surface of high-speed train structural components, nor can it completely identify all defects in the putty coating. Neither of these methods achieves high-precision and efficient global measurement of the three-dimensional morphology of the putty coating on the surface of high-speed train structural components, nor does it accurately process modal data of the putty coating. They cannot identify minute surface defects from multiple perspectives, and cannot effectively handle the accurate identification of subtle defects in the putty coating, such as pitting, raised ridges, and wavy stripes, or efficiently acquire defect geometric features.

[0005] To address the issues of weak surface texture, small light intensity differences, smooth surface, and indistinct putty defect characteristics on the surface of large high-speed rail structural components, there is an urgent need in this field to propose a new sensing and measurement system that can accurately measure the three-dimensional morphology of the surface of high-speed rail structural components under low light intensity differences, and accurately identify and classify defect areas based on its point cloud data, thus preparing for subsequent adaptive grinding and polishing repair. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for accurately detecting minute defects in the putty coating of high-speed railway structural components. This method, combining the characteristics of high-speed railway structural component grinding and its defect identification and repair processes, designs a precise identification method and an efficient extraction method for defect features in the putty coating of high-speed railway structural components. This method integrates structured light measurement and photometric stereo vision technology, enabling accurate measurement of the three-dimensional morphology of the putty coating on the high-speed railway surface even under low light intensity differences. By studying the defect detection network model under single modes and performing data fusion, the nonlinear relationship between different modes and different defects is fitted. Simultaneously, a random forest method is used to augment the sample, enriching the defect sample data. Anomaly detection is performed on the augmented data to eliminate abnormal data. Defect regions are divided, and different types of defects are labeled. The distribution and geometric information of defects are also acquired, thus preparing for subsequent adaptive grinding and polishing repair, ensuring the surface smoothness of the entire high-speed railway, reducing wind resistance and air resistance, improving the operational safety and stability of the train, and enhancing the train's operating efficiency and energy.

[0007] To achieve the above objectives, according to one aspect of the present invention, a method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components is proposed, comprising the following steps:

[0008] Step 1: Obtain the three-dimensional shape information, depth information, and photometric information of the surface of the high-speed rail structural component under synchronous conditions; then fuse the three-dimensional shape information, depth information, and photometric information to obtain surface morphology data.

[0009] Step 2: The effects of multiple single modes on different defects are fused to obtain the nonlinear relationship between different modes and different defects. Based on this nonlinear relationship and surface morphology data, defect identification and defect feature extraction are performed.

[0010] Step 3: Based on the identified defects, defect features, and robot polishing pose, trajectory planning is performed on the polishing points to adaptively polish and repair the defective putty coating on the surface of high-speed rail structural components.

[0011] As a further preferred option, step one includes the following steps:

[0012] S111 uses a structured light projection device to acquire the three-dimensional shape information of the surface of high-speed rail structural components, and a photometric stereo vision camera device to acquire the depth and photometric information of the surface of high-speed rail structural components. At the same time, it ensures that the structured light projection device and the photometric stereo vision camera device work synchronously to acquire information at the same time.

[0013] S112 fuses the data from the structured light projection device and the photometric stereo vision camera device, and then registers the data acquired by the two to ensure their consistency in space. Based on this, it fuses the three-dimensional shape information, depth information and photometric information to obtain accurate and complete surface topography data.

[0014] As a further preferred embodiment, step two, which involves fusing the influence of multiple single modes on different defects to obtain the nonlinear relationship between different modes and different defects, includes the following steps:

[0015] S211 Based on the surface topography data obtained in step one, a feature-based alignment method is used to align the data;

[0016] S212 constructs a single-modal defect detection network. One mode is selected, and its corresponding aligned data is used as input. The data is then imported into the single-modal defect detection network for processing. The nonlinear relationship between the single mode and different defects is fitted using deep learning methods.

[0017] S213 Construct an attention mechanism model, calculate attention weights using the output of the single-modal defect detection network in step S22, and train the attention mechanism model to fit the nonlinear relationship between different modalities and different defects.

[0018] As a further preferred embodiment, in step S212, the single-modal defect detection network uses a convolutional neural network as a deep learning model and performs residual connections, data annotation, and data augmentation to fit the nonlinear relationship between the modality and different defects. Finally, the single-modal defect detection network is trained and optimized using the labeled dataset obtained from the data annotation.

[0019] As a further preferred embodiment, step S213 specifically includes the following steps:

[0020] S2131 introduces an attention mechanism, designs an attention module, and calculates attention weights by combining the output of a single-modal defect detection network. The mechanism for calculating attention weights is achieved by using dot product attention to calculate the similarity between query items and key items.

[0021] S2132 Based on the similarity value calculated in step S2131, a normalization operation is performed to obtain the attention distribution, which represents the importance of each modality in the current defect or other defects;

[0022] S2133 uses attention weights to perform a weighted summation of the value terms, resulting in a weighted weight representation;

[0023] S2134 combines attention weights and obtains nonlinear relationships between different modalities and defects through model training.

[0024] As a further preferred option, the method also includes: augmenting the defect data of the surface morphology data using a random forest method to increase the number of defect samples, as detailed below:

[0025] First, the data processed by the attention mechanism model is used as the training dataset and labeled. The training dataset is then randomly sampled to generate multiple sub-datasets, each of which is the same size as the original dataset.

[0026] Next, a decision tree model was constructed using the CHAID algorithm on the subset dataset. This decision tree model used the chi-square test to evaluate the correlation between each attribute and the target variable, and selected the most significant attribute for splitting. The prediction results of each decision tree were then integrated using a random forest model to make the final prediction.

[0027] Finally, cross-validation was used to evaluate the random forest model, and the model was tuned and optimized based on the evaluation results.

[0028] As a further preferred embodiment, step two, which involves defect identification and defect feature extraction based on the nonlinear relationship and surface morphology data, includes the following steps:

[0029] S221 constructs a feature extraction branch, using a convolutional neural network to learn high-level features in various modal data and extract rich feature representations from them;

[0030] S222 constructs an anomaly detection branch, using a generative adversarial network anomaly detection algorithm to learn from the data of each modality, in order to extract regions that may have defects and remove bad data from the database;

[0031] S223 constructs a fusion mechanism to fuse the outputs of the feature extraction branch and the anomaly detection branch;

[0032] S224 uses the U-Net model as the base network for semantic segmentation. The fused data is imported into the U-Net model for semantic segmentation to extract features of different defects. At the same time, the putty coating area on the surface of high-speed rail structural components is divided into defect areas and non-defect areas, and the defect type of the defect area is identified.

[0033] As a further preferred option, in step S224, in order to optimize the model's feature extraction capability for the putty coating data on the surface of high-speed rail structural components, a feature extraction loss function is introduced:

[0034]

[0035] In the formula, N is the number of samples, and C is the number of categories. Let be the true label of the i-th sample belonging to the j-th class. It is the model's predicted probability that the i-th sample belongs to the j-th class.

[0036] Then, the alignment loss and feature extraction loss are fused to construct the overall loss function:

[0037]

[0038] in, and All are weighting coefficients. For alignment loss.

[0039] According to another aspect of the present invention, a precise detection system for minute defects in the surface coating putty of high-speed railway structural components is also provided, comprising:

[0040] The end-point measurement module is used to acquire the three-dimensional shape information, depth information, and photometric information of the surface of the high-speed rail structural components.

[0041] A multi-functional sensor fusion and data processing module is used to fuse the three-dimensional shape information, depth information and photometric information to obtain surface morphology data.

[0042] The defect identification and defect feature extraction module is used to fuse the influence of multiple single modes on different defects to obtain the nonlinear relationship between different modes and different defects. Based on this nonlinear relationship and surface morphology data, defect identification and defect feature extraction are performed.

[0043] The adaptive grinding and polishing repair module is used to perform trajectory planning for robot grinding and polishing processing points based on the identified defects, defect features, and robot grinding and polishing pose, so as to adaptively grind and polish defective areas of putty coating on the surface of high-speed rail structural components.

[0044] As a further preferred embodiment, the end measurement module includes:

[0045] A structured light projection device for acquiring three-dimensional shape information of the surface of high-speed rail structural components, and a photometric stereo vision camera device for acquiring depth and photometric information of the surface of high-speed rail structural components.

[0046] The structured light projection device and the photometric stereo vision camera device work synchronously to acquire information at the same time;

[0047] The system also includes a measurement robot equipped with the end-effector measurement module, and a moving guide rail for the movement of the measurement robot.

[0048] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0049] 1. This invention combines structured light measurement and photometric stereo vision technology to accurately measure the three-dimensional morphology of putty coatings on high-speed railway surfaces under low light intensity differences. By studying defect detection network models under single modes and fusing data, the nonlinear relationships between different modes and different defects are fitted. Simultaneously, random forest is used to augment the sample data, enriching the defect sample data. Anomaly detection is performed on the augmented data to eliminate outliers, defect regions are divided, and different types of defects are labeled. The distribution and geometric information of defects are also acquired, thus achieving accurate identification of defects in the putty coatings on high-speed railway structural components and efficient extraction of defect features, providing support for subsequent adaptive grinding and polishing repair.

[0050] 2. This invention integrates structured light measurement and photometric stereo vision into a multifunctional sensing measurement method to achieve precise three-dimensional morphology measurement of putty coating on the surface of high-speed rail structural components and comprehensive acquisition of subtle putty defect features. At the same time, it uses a joint calibration between a moving guide rail, robot, and end-effector based on pose graph optimization to achieve precise global error compensation of the measurement system.

[0051] 3. This invention proposes an end-to-end network modeling method based on attention mechanism for multi-branch, multi-modal data alignment. To simplify the problem and delve deeper into defect detection under different modalities, a single-modal defect detection network is first established. Then, data from different modalities are aligned and imported into the single-modal defect detection network for further processing. An attention mechanism is introduced to obtain the proportion of defects corresponding to different modalities, and finally, the nonlinear relationship between different modalities and different defects is fitted.

[0052] 4. This invention uses the random forest method for data augmentation to enrich the number of defect samples. It studies semantic segmentation theory and proposes a deep learning semantic segmentation modeling method that integrates anomaly detection and feature extraction branches. This method can eliminate bad data, extract defect features, divide defective regions into normal regions, and accurately identify defects such as pockmarks, ridges, and wavy stripes, as well as efficiently acquire defect geometric features.

[0053] 5. To ensure the accuracy of multimodal data alignment and improve the extraction effect of key features, this invention introduces an alignment loss function and a feature extraction loss function. By minimizing the loss function, the model parameters are updated, enabling the model to achieve the best fitting effect when performing data extraction and feature extraction, thus helping to complete the high-precision extraction of putty defects in high-speed rail structural components. Attached Figure Description

[0054] Figure 1 This is a flowchart of a method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components according to the present invention;

[0055] Figure 2 This is a flowchart illustrating a method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components, as described in an embodiment of the present invention.

[0056] Figure 3 This is a schematic diagram of a system for accurately detecting minute defects in the surface coating putty of high-speed rail structural components, according to an embodiment of the present invention.

[0057] Figure 4 This is a flowchart of multi-branch data processing based on an attention mechanism, as described in an embodiment of the present invention.

[0058] Figure 5 This is a flowchart of the defect sample data augmentation steps involved in an embodiment of the present invention;

[0059] Figure 6 This is a flowchart of defect classification and feature extraction based on semantic segmentation method involved in the embodiments of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0061] like Figures 1 to 6 As shown in the figure, the present invention provides a method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components, comprising the following steps:

[0062] Step 1: Obtain the three-dimensional shape information, depth information, and photometric information of the surface of the high-speed rail structural component under synchronous conditions. Then, fuse the three-dimensional shape information, depth information, and photometric information to obtain surface morphology data.

[0063] Step 2: The effects of multiple single modes on different defects are fused to obtain the nonlinear relationship between different modes and different defects. Based on this nonlinear relationship and surface morphology data, defect identification and defect feature extraction are performed.

[0064] Step 3: Based on the identified defects, defect features, and robot polishing pose, trajectory planning is performed on the polishing points to adaptively polish and repair the defective putty coating on the surface of high-speed rail structural components.

[0065] like Figure 3 As shown, the detection and subsequent adaptive grinding and polishing repair rely on the following system:

[0066] The end-effector measurement module is used to acquire the three-dimensional shape information, depth information, and photometric information of the surface of high-speed rail structural components. Specifically, the end-effector measurement module integrates a structured light projection device to acquire the three-dimensional shape information of the high-speed rail structural component surface. It also integrates a photometric stereo vision camera device to acquire the depth and photometric information of the high-speed rail structural component surface. A synchronization device is also included to ensure that the structured light projection device and the photometric stereo vision camera device can work synchronously to acquire information at the same time.

[0067] After collecting the three-dimensional shape, depth, and photometric information of the high-speed rail surface, the data is transmitted to the multi-functional sensor fusion and data processing module. First, sensor fusion is performed, fusing data from the structured light measurement system and the photometric stereo vision system. This takes into account the advantages of high accuracy in structured light measurement and the strong ability of photometric stereo vision to depict subtle defects. The data acquired by the two sensors are registered to ensure their consistency in space. Then, the three-dimensional topography information reconstructed by structured light is fused with the depth information estimated by stereo vision to obtain more accurate and complete surface topography data.

[0068] The defect identification and defect feature extraction module is used to fuse the influence of multiple single modes on different defects, obtaining the nonlinear relationship between different modes and different defects. Based on this nonlinear relationship and surface morphology data, defect identification and defect feature extraction are performed. Feature extraction and analysis are then performed on the fused data. System calibration and adjustment are conducted to ensure the accurate correspondence between the structured light projection and the stereo vision system, as well as the correct alignment of photometric and depth information.

[0069] The adaptive grinding and polishing repair module is used to perform trajectory planning for grinding and polishing processing points based on the identified defects, defect features, and robot grinding and polishing pose, so as to adaptively grind and polish defective areas of putty coating on the surface of high-speed rail structural components.

[0070] To address the issue of large error accumulation caused by the large number of devices in a robotic grinding and polishing repair system, a joint calibration method based on pose graph optimization is proposed, which integrates the robot, the moving guide rail, the end-efficiency measuring equipment, and the high-speed rail. This method enables precise compensation for global errors in the measurement system and achieves high-precision and efficient global measurement of the three-dimensional morphology of the putty coating on the surface of high-speed rail structural components.

[0071] Based on any or a combination of the above embodiments, in this embodiment, in step one, the three-dimensional shape information of the surface of the high-speed rail structural component is acquired using a structured light projection device, and the depth and photometric information of the surface of the high-speed rail structural component are acquired using a photometric stereoscopic vision camera device. Simultaneously, the structured light projection device and the photometric stereoscopic vision camera device are ensured to operate synchronously to acquire information at the same time. The data from the structured light projection device and the photometric stereoscopic vision camera device are fused, and then the data acquired by both are registered to ensure their consistency in space. Based on this, the three-dimensional shape information, depth information, and photometric information are fused to obtain accurate and complete surface topography data.

[0072] Based on any of the above embodiments or combinations of embodiments, in this embodiment, in step two, based on the surface topography data obtained in step one, a feature-based alignment method is used to align the data; a single-modal defect detection network is constructed, one modality is selected, and its corresponding aligned data is used as input and imported into the single-modal defect detection network for processing. The nonlinear relationship between the single modality and different defects is fitted using a deep learning method; an attention mechanism model is constructed, and the attention weights are calculated using the output results of the single-modal defect detection network in step S22. The attention mechanism model is trained to fit the nonlinear relationship between different modalities and different defects.

[0073] Specifically, this embodiment acquires the point cloud model, gradient model, photometric information, and depth information of the putty layer on the surface of high-speed rail structural components using a measurement device. Because the putty layer on the surface of high-speed rail structural components has many defect types, and the measurement data is large and diverse, a single-modal defect detection network model is first studied, and then an attention mechanism is introduced to obtain the proportion of defects corresponding to different modes. This ultimately reveals the nonlinear relationship between different modes and different defects, facilitating subsequent processing. Therefore, a multi-branch data alignment and single-modal data processing network is designed. To simplify the problem and to further study defect detection under this mode, research on the single mode begins. The following data has been measured using a robot measurement system, including gradient, point cloud, photometric information, and depth information. A feature-based alignment method can be used to ensure that the data is acquired in the same coordinate system. After alignment, one mode is selected as input data and imported into the single-modal defect detection network for processing.

[0074] The single-modal defect detection network employs a convolutional neural network as the deep learning model, and performs steps such as residual connections, data annotation, and data augmentation. It uses deep learning methods to fit the nonlinear relationship between the modality and different defects. Data annotation involves preparing a labeled dataset for the single-modal data, annotating the defect regions on the coating surface. Data augmentation involves augmenting the training data to improve the model's generalization ability. Finally, the labeled dataset is used to continuously train the single-modal defect detection network, monitoring its performance and optimizing it.

[0075] After processing each modality's data individually, an attention mechanism is introduced, an attention module is designed, and the attention weights are calculated based on the output of the single-modality defect detection network. These weights can also represent the weights of each modality's influence on different defects. A mechanism for calculating attention weights is designed, which is implemented by calculating the similarity between query terms and key terms using dot-product attention. Then, using the calculated similarity values, a normalization operation is performed to obtain the attention distribution, representing the importance of each modality in the current defect or other defects. Subsequently, the attention weights are used to perform a weighted summation of the value terms to obtain a weighted weight representation, which facilitates subsequent processing and use. Finally, the nonlinear relationships between different modalities and different defects are obtained by combining the attention weights and training the model.

[0076] For multimodal data, alignment between different modalities is crucial. To ensure that the model aligns different modalities in the feature space, an alignment loss function is introduced. By minimizing the alignment loss, the representations between modalities become more consistent, which helps improve the model's generalization ability and robustness, enabling it to better handle multimodal data. The alignment loss can be represented by the Euclidean distance loss:

[0077]

[0078] in, and These are two feature representations, express Norm.

[0079] Based on any of the above embodiments or combinations of embodiments, this embodiment addresses the characteristic that the measurement data of the putty layer on the surface of high-speed railway structural components has abundant normal data but limited defect data, making it impossible to fully understand all defect data. Therefore, a random forest method is proposed to augment the data and increase the number of defect samples. First, the training dataset is prepared using a multimodal data processing network based on an attention mechanism and labeled. The training dataset is then randomly sampled to generate multiple subsets, each the same size as the original dataset. However, due to the random sampling, each subset may contain duplicate samples or unselected samples. Next, a base learner is constructed. For each subset, a decision tree model is built using the CHAID algorithm. This model uses a chi-square test to evaluate the correlation between each attribute and the target variable and selects the most significant attribute for splitting. When constructing the decision tree, features selected during node splitting and hyperparameters such as tree depth are typically considered. After all decision trees are constructed, the random forest integrates the prediction results of each decision tree to make the final prediction. Finally, cross-validation was used to evaluate the random forest model, and based on the evaluation results, the random forest model was tuned and optimized, including adjusting hyperparameters, optimizing feature selection, and increasing the number of trees, in order to improve the model's performance.

[0080] Based on any of the above embodiments or combinations of embodiments, in this embodiment, step two, which involves defect identification and defect feature extraction based on the nonlinear relationship and surface morphology data, further includes: after data augmentation using the random forest method, the amount of data increases significantly, and abnormal data may also appear. In order to accurately remove bad data from the database and accurately identify subtle defects in the putty coating on the surface of the high-speed rail car body, a deep learning semantic segmentation model that integrates anomaly detection and feature extraction branches is designed based on semantic segmentation theory.

[0081] Feature extraction branch: This branch uses convolutional neural networks to learn high-level features from various modalities, extracting rich feature representations. Its main function is to capture key information from each modality, which helps identify different types of defects.

[0082] Anomaly Detection Branch: This branch uses generative adversarial network anomaly detection algorithms to learn from the data of each modality. It helps to extract potentially defective regions and remove bad data from the database, so as to perform feature extraction and semantic segmentation tasks more accurately.

[0083] Simultaneously, a fusion mechanism is constructed to combine the outputs of the feature extraction branch and the anomaly detection branch. This mechanism can also incorporate an attention mechanism, effectively fusing the output data of the two branches by calculating their attention weights. The fused data is then re-introduced into the semantic segmentation network for re-segmentation, improving the accuracy of feature acquisition.

[0084] The U-Net model was chosen as the foundational network for semantic segmentation, responsible for the overall semantic segmentation task. This network extracts features of different defects, such as surface contours, surface textures, illumination intensity, and surface height. Simultaneously, the high-speed rail structural components are divided into defective and non-defective regions. For defective regions, the type of defect is identified, such as pitting, raised ridges, or wavy stripes.

[0085] To optimize the model's feature extraction capability from the putty coating data on the surface of high-speed rail structural components, enabling it to better capture key information from the data, a feature extraction loss function is introduced. This feature extraction loss can be represented using classification loss.

[0086]

[0087] Taking into account both the accuracy of multimodal data alignment and the improvement of key feature extraction, the alignment loss and feature extraction loss can be fused together and weighted into a single overall loss function:

[0088]

[0089] in, and These are the weighting coefficients for alignment loss and feature extraction loss, which need to be adjusted according to the actual situation.

[0090] Then, the model needs to be optimized, which may require adjusting hyperparameters and weights of the loss function, so that the model performs well on both tasks and can accurately identify and efficiently acquire geometric information of minor defects in the putty coating on the surface of high-speed rail structural components, such as pitted, raised, and wavy striped defects.

[0091] Based on any of the above embodiments or combinations of embodiments, in this embodiment, based on the identified defects, defect features and robot grinding and polishing pose, trajectory planning is performed on the grinding and polishing processing points to adaptively grind and polish the defective parts of the putty coating on the surface of high-speed rail structural components.

[0092] In summary, the defect repair method of this invention can ensure the surface smoothness of the entire high-speed train body, reduce wind resistance and air resistance, improve the operational safety and stability of the train, and enhance the train's operational efficiency and energy efficiency. Specifically, this invention achieves accurate identification and efficient extraction of defect features in the putty coating of high-speed train structural components, thus providing a theoretical basis for subsequent adaptive grinding and polishing repair. To address the problems of weak surface texture, small light intensity differences, smooth surface, and indistinct putty defect features in the putty coating of large high-speed train surfaces, a novel sensing and measurement system was designed. This system combines structured light measurement and photometric stereo vision technology, enabling accurate measurement of the three-dimensional morphology of the putty coating on high-speed train surfaces under low light intensity differences. By studying the defect detection network model under single-modality conditions and performing data fusion, the nonlinear relationship between different modes and different defects was fitted. Simultaneously, random forest method was used for sample augmentation to enrich the defect sample data. Anomaly detection was performed on the augmented data to eliminate abnormal data, defect areas were divided, and different types of defects were labeled. The distribution and geometric information of defects were also acquired, preparing for subsequent adaptive grinding and polishing repair.

[0093] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components, characterized in that, Includes the following steps: Step 1: Obtain the three-dimensional shape information, depth information, and photometric information of the surface of the high-speed rail structural component under synchronous conditions. Then, fuse the three-dimensional shape information, depth information, and photometric information to obtain surface morphology data. Step one includes the following steps: S111 uses a structured light projection device to acquire the three-dimensional shape information of the surface of high-speed rail structural components, and a photometric stereo vision camera device to acquire the depth and photometric information of the surface of high-speed rail structural components. At the same time, it ensures that the structured light projection device and the photometric stereo vision camera device work synchronously to acquire information at the same time. S112 fuses the data from the structured light projection device and the photometric stereo vision camera device, and then registers the data acquired by the two to ensure their consistency in space. Based on this, it fuses the three-dimensional shape information, depth information and photometric information to obtain accurate and complete surface topography data. Step 2: The effects of multiple single modes on different defects are fused to obtain the nonlinear relationship between different modes and different defects. Based on this nonlinear relationship and surface morphology data, defect identification and defect feature extraction are performed. Step two, the defect identification and defect feature extraction based on the nonlinear relationship and surface morphology data, includes the following steps: S221 constructs a feature extraction branch, using a convolutional neural network to learn high-level features in various modal data and extract rich feature representations from them; S222 constructs an anomaly detection branch, using a generative adversarial network anomaly detection algorithm to learn from the data of each modality, in order to extract regions that may have defects and remove bad data from the database; S223 constructs a fusion mechanism to fuse the outputs of the feature extraction branch and the anomaly detection branch; S224 uses the U-Net model as the basic network for semantic segmentation. The fused data is imported into the U-Net model for semantic segmentation to extract the features of different defects. At the same time, the putty coating area on the surface of the high-speed rail structural component is divided into defect area and non-defect area, and the defect type of the defect area is identified. In step S224, to optimize the model's feature extraction capability for the putty coating data on the surface of high-speed rail structural components, a feature extraction loss function is introduced: ， In the formula, N is the number of samples, and C is the number of categories. Let i be the true label of the i-th sample belonging to the j-th class. It is the model's predicted probability that the i-th sample belongs to the j-th class; Then, the alignment loss and feature extraction loss are fused to construct the overall loss function: ， in, and All are weighting coefficients. For alignment loss; Step 3: Based on the identified defects, defect features, and robot polishing pose, trajectory planning is performed on the polishing points to adaptively polish and repair the defective putty coating on the surface of high-speed rail structural components.

2. The method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components according to claim 1, characterized in that, Step two, which involves fusing the influence of multiple single modes on different defects to obtain the nonlinear relationship between different modes and different defects, includes the following steps: S211 Based on the surface topography data obtained in step one, a feature-based alignment method is used to align the data; S212 constructs a single-modal defect detection network. One mode is selected, and its corresponding aligned data is used as input. The data is then imported into the single-modal defect detection network for processing. The nonlinear relationship between the single mode and different defects is fitted using deep learning methods. S213 Construct an attention mechanism model, calculate attention weights using the output of the single-modal defect detection network in step S22, and train the attention mechanism model to fit the nonlinear relationship between different modalities and different defects.

3. The method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components according to claim 2, characterized in that, In step S212, the single-modal defect detection network uses a convolutional neural network as a deep learning model and performs residual connections, data annotation, and data augmentation to fit the nonlinear relationship between the modality and different defects. Finally, the labeled dataset obtained from the data annotation is used to train and optimize the single-modal defect detection network.

4. The method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components according to claim 2, characterized in that, Step S213 specifically includes the following steps: S2131 introduces an attention mechanism, designs an attention module, and calculates attention weights by combining the output of a single-modal defect detection network. The mechanism for calculating attention weights is achieved by using dot product attention to calculate the similarity between query items and key items. S2132 Based on the similarity value calculated in step S2131, a normalization operation is performed to obtain the attention distribution, which represents the importance of each modality in the current defect or other defects; S2133 uses attention weights to perform a weighted summation of the value terms, resulting in a weighted weight representation; S2134 combines attention weights and obtains nonlinear relationships between different modalities and defects through model training.

5. The method for accurately detecting minute defects in the surface coating putty of high-speed railway structural components according to claim 2, characterized in that, Also includes: Random forest was used to augment the defect data in the surface morphology data, increasing the number of defect samples, as detailed below: First, the data processed by the attention mechanism model is used as the training dataset and labeled. The training dataset is then randomly sampled to generate multiple sub-datasets, each of which is the same size as the original dataset. Next, a decision tree model was constructed using the CHAID algorithm on the subset dataset. This decision tree model used the chi-square test to evaluate the correlation between each attribute and the target variable, and selected the most significant attribute for splitting. The prediction results of each decision tree were then integrated using a random forest model to make the final prediction. Finally, cross-validation was used to evaluate the random forest model, and the model was tuned and optimized based on the evaluation results.

6. A precise detection system for minute defects in the surface coating putty of high-speed railway structural components, used to implement the precise detection method for minute defects in the surface coating putty of high-speed railway structural components as described in any one of claims 1-5, characterized in that, include: The end-point measurement module is used to acquire the three-dimensional shape information, depth information, and photometric information of the surface of the high-speed rail structural components. A multi-functional sensor fusion and data processing module is used to fuse the three-dimensional shape information, depth information and photometric information to obtain surface morphology data. The defect identification and defect feature extraction module is used to fuse the influence of multiple single modes on different defects to obtain the nonlinear relationship between different modes and different defects. Based on this nonlinear relationship and surface morphology data, defect identification and defect feature extraction are performed. The adaptive grinding and polishing repair module is used to perform trajectory planning for robot grinding and polishing processing points based on the identified defects, defect features, and robot grinding and polishing pose, so as to adaptively grind and polish defective areas of putty coating on the surface of high-speed rail structural components.

7. The precise detection system for minute defects in the surface coating putty of high-speed railway structural components according to claim 6, characterized in that, The end-point measurement module includes: A structured light projection device for acquiring three-dimensional shape information of the surface of high-speed rail structural components, and a photometric stereo vision camera device for acquiring depth and photometric information of the surface of high-speed rail structural components. The structured light projection device and the photometric stereo vision camera device work synchronously to acquire information at the same time; The system also includes a measurement robot equipped with the end-effector measurement module, and a moving guide rail for the movement of the measurement robot.

Images