Railway sound barrier bolt loosening intelligent identification method

Abstract

The invention discloses a railway sound barrier bolt loosening intelligent identification method, which is used for intelligent detection and identification of a railway sound barrier bolt loosening state, and comprises the steps: obtaining a railway sound barrier image, and carrying out the identification of the railway sound barrier image based on deep learning, determining a sound barrier structured component to achieve a substantial positioning of the position of the railway sound barrier bolt to obtain position information, and calibrating the pixel accuracy and the position information of the railway sound barrier image; the sound barrier structural component comprises a stand column, a unit panel and a bolt. Identifying the type of the bolt and the position of the component based on a bolt structure refined identification method so as to carry out refined identification on the state of the bolt; outlier detection, clustering and interpolation are carried out on the recognition result based on a DBSCAN algorithm, so that automatic filtering of clustering is realized to improve the accuracy of the recognition result; and carrying out disease identification on the railway sound barrier bolt based on the processed identification result. The invention further discloses an identification system, electronic equipment and a computer readable storage medium.

Description

Technical Field

[0001] The invention relates to the technical field of railway sound barrier appearance bolt status detection, and in particular to a railway sound barrier bolt loosening intelligent identification method. Background Art

[0002] Bolts are indispensable components in the machinery industry, and their preload ensures the firmness of the structure. During the operation of high-speed trains, vibrations and impacts will occur due to the movement of the train itself and the influence of external factors. These vibrations and impacts will be transmitted to each bolt connection structure through the car body. When the amplitude, frequency, direction and other parameters of these vibrations and impacts exceed the bearing capacity of the bolt connection structure, it may cause the bolts to loosen or the connection structure to be damaged. In addition, during the operation of high-speed trains, due to factors such as acceleration, deceleration, and turning of the car body, additional dynamic loads will be generated, which may also cause the bolt connection structure to loosen or be damaged.

[0003] In order to prevent similar incidents from happening again, it is particularly important to improve fault diagnosis technology. On the one hand, this technology can locate and predict potential problems before a fault occurs, so that corresponding measures can be taken to avoid accidents. On the other hand, after a fault occurs, analysis and judgment can quickly locate the problem and reduce maintenance costs and time. Therefore, bolt loosening detection is a very important technical work that can ensure the safety and stability of train operation.

[0004] In view of the practical significance of machine vision inspection of bolts, domestic and foreign scholars have conducted relevant research and achieved certain practical application results. Park et al. combined Hough transform and Canny edge detection algorithm to estimate bolt looseness by tracking the rotation trajectory of the bolt boundary. Cha et al. used Hough transform and support vector machine (SVM) to detect the tightening state of bolts. With the rapid development of target detection algorithms based on deep learning, they are widely used in bolt detection. Wang Xiangzhou et al. realized the real-time detection and positioning of angle steel tower burst bolts based on YOLOv5s-T and RGB-D camera. Zhu Jun used improved ResNeXt to extract bolt features based on YOLOv4 to locate bolts in key parts of the car body and effectively detected the looseness of bolts. Zhao et al. used SSD to train a dual-target detection model to detect the bolt head and the digital label on it respectively, and judged the tightening state of the bolt by detecting whether the relative position of the two changed. However, the above algorithms are generally aimed at general scenarios. The current research has failed to effectively target and utilize the significant features of the positioning paint on the bolts of railway locomotive bogies to assist in identifying the tightening state of the bolts.

[0005] In short, there are still some defects in bolt loosening detection in the current rail transit industry. Due to the complexity of the bolt connection structure and the difficulty of the detection process, there is no unified, convenient and effective detection method. Therefore, providing timely and effective intelligent railway sound barrier bolt loosening detection means, accurately reporting fault diagnosis information, and timely discovering and solving potential problems are of far-reaching significance to the development of my country's rail transit. Summary of the invention

[0006] The purpose of the present invention is to design a method for detecting and identifying the loose state of bolts of railway sound barriers that comprehensively utilizes deep learning technology and image processing technology to address the defects of the prior art, so as to realize automatic recognition and judgment of the loose state of bolts. The overall technical route is mainly divided into three parts: a method for identifying structural components of sound barriers, a method for identifying bolt structure refinement, and a method for filtering outliers and filling missing values ​​based on statistics. Among them, the method for identifying structural components of sound barriers first identifies the iconic structural components of the sound barrier to achieve the approximate positioning of the bolt position and the calibration of pixel accuracy and position information; the method for identifying the detailed structure of the bolts can identify the type and component position of the bolts, etc., to achieve fine recognition of the bolt state; the method for missing and filling out outliers based on statistics combines artificial experience information and statistical information, filters out outliers in the recognition results through statistical methods, fills in the missing recognition results, and further improves the accuracy of the recognition results based on the results of the first two modules.

[0007] The first aspect of the present invention is to provide a method for intelligently identifying the loosening of railway sound barrier bolts, which is used for intelligent detection and identification of the loosening state of railway sound barrier bolts, comprising:

[0008] S1, obtaining a railway sound barrier image, and identifying the railway sound barrier image based on deep learning, determining the sound barrier structural components to achieve the approximate positioning of the railway sound barrier bolts to obtain position information, and calibrating the pixel accuracy of the railway sound barrier image and the position information; wherein the sound barrier structural components include columns, unit panels and bolts;

[0009] S2, based on the bolt structure refinement identification method, identifies the type and component position of the bolt to finely identify the bolt status;

[0010] S3, performing outlier detection, clustering and interpolation on the recognition results based on the DBSCAN algorithm, thereby realizing automatic filtering of clusters to improve the accuracy of the recognition results;

[0011] S4, identifying defects of the railway sound barrier bolts based on the processed identification results.

[0012] Preferably, the S1 comprises:

[0013] S11, establishing a deep learning model for identifying the sound barrier structural components, inputting a plurality of historical sound barrier images with marked sound barrier structural components into the deep learning model and training the deep learning model to obtain a sound barrier structural component recognition model; completing automatic sound barrier structural component recognition based on the sound barrier structural component recognition model to obtain the recognition result of the sound barrier structural component;

[0014] S12, aggregating the characteristics of different structural components of the railway sound barrier into common common characteristics, and extracting attribute information based on the common characteristics; the attribute information includes pixel accuracy calibration information, entrance and exit identification information, sound barrier height calculation information and sound barrier length calculation information;

[0015] S13, calibrating the recognition result of the sound barrier structural component based on the attribute information.

[0016] Preferably, the deep learning model is a YOLO V10 model, including:

[0017] (1) Backbone network: A deep convolutional neural network is used, including multiple convolutional layers, batch normalization layers and activation functions; the convolutional layers are used to extract local features of the image; the batch normalization layers are used to accelerate the training process and reduce the model's sensitivity to initial weights; the activation function is used to increase the nonlinearity of the network and help the network learn complex features;

[0018] (2) Neck network: It includes multiple feature pyramid network layers or path aggregation network layers, which are used to fuse features of different scales; the feature pyramid network layer is used to construct a feature pyramid from top to bottom to enhance the model's detection capability for multi-scale targets; the path aggregation network layer is used to enhance feature transfer from bottom to top to further improve the feature fusion effect;

[0019] (3) Head network: used to generate the final detection result; the head network includes a convolution layer and a detection layer, the convolution layer is used to further process the feature map; the detection layer is used to predict the offset, width, height, confidence and category probability of the bounding box at each feature map position;

[0020] (4) Loss function: used to guide model training, including positioning loss, classification loss, object confidence loss and non-object confidence loss; wherein the positioning loss is used to calculate the position error between the predicted bounding box and the true bounding box; the classification loss is used to calculate the error between the predicted category and the true category; the object confidence loss is used to calculate the error between the predicted confidence and the true confidence; the non-object confidence loss is used to penalize the background area to reduce false detection.

[0021] Preferably, S2 includes:

[0022] S21, based on the deep learning model, refine and identify the bolt as a whole and the positions of various components of the bolt structure and calculate attribute information, wherein the positions of various components include the nut and the screw;

[0023] S22, determining whether there is an abnormal problem of bolt missing, a problem of screw loosening, and an abnormal problem of not reserving a screw surplus as required according to the positional relationship between the nut and the screw;

[0024] S23, determining whether there is an abnormal bolt falling-off problem according to the number of identified nuts, including: if the number of nuts on each screw rod is lower than the standard number, confirming that there is an abnormal bolt falling-off problem.

[0025] Preferably, the S21 includes:

[0026] (1) Identify the entire bolt, including: extracting the overall position of the column bolt based on the YOLO V10 model, extracting the maximum height of the bolt based on the upper edge of the bolt identification box, expanding the identification box and limiting the area range; if there is any unidentified situation, fill it in based on the linear interpolation method to obtain the unidentified bolt;

[0027] (2) Identifying nuts, including: identifying the position of the column nuts based on the YOLO V10 model and calculating the number of nuts on each screw;

[0028] (3) Identifying the screw, including: identifying the position of the column screw based on the YOLO V10 model and calculating the exposed screw length, wherein the calculation of the exposed screw length includes: calculating the exposed screw length based on the exposed screw identification frame and pixel accuracy identified by the YOLO V10 model; and determining the number of threads of the exposed screw portion based on the screw parameters and the exposed screw length.

[0029] Preferably, the S22 includes:

[0030] If it is determined that the bolt missing abnormal problem exists, the bolt missing parts will be screened out and not included in the bolt loosening statistics;

[0031] If the number of deductions is less than the specified number, it is determined that there is a problem of screw loosening;

[0032] If the screw is not identified, the position of the top nut in the bolt is determined. If the top nut position exceeds the normal position of the screw, it is considered that serious looseness has occurred, the nut has covered the screw, and there is an abnormal problem that the screw margin required by regulations is not reserved.

[0033] Preferably, S3 includes:

[0034] S31, screening the recognition results for outliers or noise points based on the DBSCAN method, including: treating the recognition results that are significantly different from the attribute information of most sound barrier structural components as misjudgment results, including:

[0035] Determine the neighborhood: For each point in the data set, determine its neighborhood;

[0036] Calculate density: If the number of points in a point's neighborhood is greater than or equal to a threshold MinPts, the point is considered a core point;

[0037] Expanding clusters: Starting from the core point, all points that can be reached by density are grouped into one cluster;

[0038] Marking outliers: Points that are not in any cluster are considered outliers or noise points;

[0039] S32, interpolating the recognition result based on the DBSCAN method, including: based on the periodic characteristics of the typical structural components of the sound barrier, extracting the distance between every two typical structural components of the sound barrier, and obtaining the mode value of the distance based on the DBSCAN method, if two typical structural components of the sound barrier are too close to each other, it is considered to be a misjudgment and is screened out, if they are too far apart, it is considered to be a missed judgment, and the number of missed targets is calculated according to the distance between them, and the targets are evenly inserted therein to achieve interpolation;

[0040] S33, performing correlation calibration on the recognition result based on the DBSCAN method, including:

[0041] Calculate the confidence of associated components: each component provides a confidence less than 1, and the sum of the confidences of the components is 1. The more sound barrier structure components extracted at the associated positions, the higher the confidence;

[0042] Screening false positives: Set a confidence threshold L. If a group of components has a confidence level lower than the threshold, it is considered a false positive and is filtered out.

[0043] Missed interpolation: If the confidence of a group of components is higher than the confidence threshold L but less than 1, it is considered that some structural components are missed and they are supplemented at the corresponding positions.

[0044] The second aspect of the present invention is to provide a railway sound barrier bolt loosening intelligent identification system, which is used to implement the method of the first aspect, so as to intelligently detect and identify the railway sound barrier bolt loosening state, including:

[0045] A bolt foreground segmentation module (101) is used to obtain a railway sound barrier image, identify the railway sound barrier image based on deep learning, determine the sound barrier structural components to achieve the approximate positioning of the railway sound barrier bolts to obtain position information, and calibrate the pixel accuracy of the railway sound barrier image and the position information; wherein the sound barrier structural components include columns, unit panels and bolts;

[0046] A bolt shape recognition module (102) is used to identify the type and component position of the bolt based on a bolt structure refinement recognition method so as to perform fine recognition of the bolt state;

[0047] A bolt post-processing module (103) is used to perform outlier detection, clustering and interpolation on the recognition results based on the DBSCAN algorithm, thereby realizing automatic filtering of clusters to improve the accuracy of the recognition results;

[0048] An intelligent detection and identification module (104) is used to identify defects in the railway sound barrier bolts based on the processed identification results.

[0049] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method described in the first aspect.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor and execute the method described in the first aspect.

[0051] Beneficial effects of the method and system of the present invention:

[0052] The present invention uses a combination of large-model, small-sample object detection, bolt foreground segmentation, and bolt morphology recognition techniques to solve a variety of technical problems in traditional methods, bringing new technological breakthroughs and progress to the field of railway sound barrier appearance status detection. Compared with the prior art, the railway sound barrier appearance status detection method disclosed in this patent brings many beneficial effects:

[0053] (1) Improve detection effect: The solution combines a large-model and small-sample object detection method, a bolt foreground segmentation module, and a bolt morphology recognition module. To address the challenge of scarce training samples, the solution can achieve better segmentation effects when facing challenges such as noise, blur, and background interference in the bolt area. The solution also combines the defined shape pattern for waveform calculation and morphology judgment to achieve classification and detection of the bolt status, thereby improving the accuracy and robustness of detection.

[0054] (2) Improve production efficiency and quality: Through automated detection and identification methods, loose bolts in railway sound barriers can be discovered in a timely manner, avoiding the risk of accidents caused by loose bolts, improving the safety and reliability of railway transportation, and also reducing the time and cost of manual inspections, thereby improving production efficiency and quality.

[0055] (3) Easy operation: The use of automated detection methods reduces manual intervention, simplifies the operation process, reduces the impact of human factors on the results, and improves the consistency and repeatability of the test.

[0056] The present invention makes full use of deep learning technology to process images, combines morphological patterns and waveform analysis methods, and realizes accurate detection of the loose state of bolts, providing important technical support for the maintenance and management of railway sound barriers. Through automated detection and identification, the problem of loose bolts can be discovered and handled in a timely manner, which not only improves the accuracy and comprehensiveness of detection, but also improves production efficiency and quality, simplifies the operation process, improves railway safety and operation efficiency, and brings important technological progress and practical value to the field of railway sound barrier appearance status detection, with broad application prospects and social significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A flow chart of a method for intelligently identifying loosened bolts of railway sound barriers provided according to an embodiment of the present invention;

[0059] Figure 2 This is a flow chart of step S1 of the intelligent identification method for loosening of railway sound barrier bolts provided according to an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of typical structural components of a sound barrier provided according to an embodiment of the present invention;

[0061] Figure 4 This is a flow chart of step S2 of the intelligent identification method for loosening of railway sound barrier bolts provided according to an embodiment of the present invention;

[0062] Figure 5 A schematic diagram of an abnormal situation of a bolt provided according to an embodiment of the present invention;

[0063] Figure 6This is a flow chart of step S3 of the method for intelligently identifying loose bolts of railway sound barriers provided according to an embodiment of the present invention;

[0064] Figure 7 This is an architecture diagram of a railway sound barrier bolt loosening intelligent identification system provided according to an embodiment of the present invention;

[0065] Figure 8 A structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0068] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0069] Embodiment 1

[0070] like Figure 1 As shown, this embodiment provides a method for intelligently identifying the loosening of railway sound barrier bolts, which is used for intelligent detection and identification of the loosening state of railway sound barrier bolts, including:

[0071] S1, obtaining a railway sound barrier image, and identifying the railway sound barrier image based on deep learning, determining the sound barrier structural components to achieve the approximate positioning of the railway sound barrier bolts to obtain position information, and calibrating the pixel accuracy of the railway sound barrier image and the position information; wherein the sound barrier structural components include columns, unit panels and bolts;

[0072] As a preferred embodiment, the S1 sound barrier typical structural component identification mainly adopts deep learning methods to detect the positions of typical structural components, and then based on the characteristics of different structural components, a variety of machine learning and statistical methods are used to correct misjudgment and missed targets, wherein the position of the bolt is located by the index of the column.

[0073] like Figure 2 As shown, as a preferred embodiment, the S1 includes:

[0074] S11, establishing a deep learning model for identifying the sound barrier structural components, inputting a plurality of historical sound barrier images with marked sound barrier structural components into the deep learning model and training the deep learning model to obtain a sound barrier structural component recognition model; completing automatic sound barrier structural component recognition based on the sound barrier structural component recognition model to obtain the recognition result of the sound barrier structural component;

[0075] In this embodiment, the deep learning model is a YOLO V10 model. The YOLO V10 method is used to identify structural components in the sound barrier image, including columns, unit panels and bolts, and the bolt positions are located by indexing the columns.

[0076] In order to solve the problem of low efficiency and low accuracy in manually identifying sound barrier structural components, the YOLO V10 model is introduced to realize automatic identification of sound barrier structural components. The YOLO (You On ly Look Once) series algorithm is a popular real-time target detection model. YOLO V10 is a version of the YOLO series. Its main structure is as follows:

[0077] (1) Backbone

[0078] The backbone network of YOLO V10 usually uses a deep convolutional neural network (CNN) to extract image features. This part usually includes multiple convolutional layers, batch normalization layers, and activation functions (such as LeakyReLU or SiLU).

[0079] Convolutional layer: used to extract local features of the image.

[0080] Batch Normalization Layer: Speeds up the training process and reduces the model’s sensitivity to initial weights.

[0081] Activation function: Increase the nonlinearity of the network and help the network learn complex features.

[0082] (2) Neck (neck network)

[0083] The neck network of YOLO V10 usually contains several Feature Pyramid Network (FPN) or Path Aggregation Network (PAN) layers to fuse features of different scales.

[0084] FPN layer: constructs a feature pyramid from top to bottom to enhance the model's ability to detect multi-scale targets.

[0085] PAN layer: Enhances feature transfer from bottom to top and further improves feature fusion effect.

[0086] (3) Head (Head Network)

[0087] The head network is the last few layers of the YOLO model and is responsible for generating the final detection results, including the location, category, and confidence of the bounding box.

[0088] Convolutional layer: further processes the feature map.

[0089] Detection layer: predicts the offset, width, height, and confidence of the bounding box at each feature map location, as well as the category probability.

[0090] (4) Loss Function

[0091] YOLO V10 uses a loss function to guide model training, which usually includes the following parts:

[0092] Localization loss: Calculates the position error between the predicted bounding box and the true bounding box.

[0093] Classification loss: Calculates the error between the predicted category and the true category.

[0094] Object confidence loss: Calculates the error between the predicted confidence and the true confidence.

[0095] Non-object confidence loss: Penalizes background areas to reduce false detections.

[0096] After the sound barrier image of the marked sound barrier structure components is input into the YOLO model, the model training is completed. After the training, the model can automatically identify the sound barrier structure components.

[0097] S12, aggregating the characteristics of different structural components of the railway sound barrier into common common characteristics, and extracting attribute information based on the common characteristics; the attribute information includes pixel accuracy calibration information, entrance and exit identification information, sound barrier height calculation information and sound barrier length calculation information;

[0098] like Figure 3 Shown is a schematic diagram of typical structural components of a sound barrier, which illustrates the characteristics of the typical structural components of a sound barrier.

[0099] S13, calibrating the recognition result of the sound barrier structural component based on the attribute information.

[0100] S2, based on the bolt structure refinement identification method, identifies the type and component position of the bolt to finely identify the bolt status;

[0101] like Figure 4 As shown, as a preferred embodiment, the S2 includes:

[0102] S21, based on the deep learning model, refine and identify the bolt as a whole and the positions of various components of the bolt structure and calculate attribute information, wherein the positions of various components include the nut and the screw;

[0103] In this embodiment, the S21 includes:

[0104] (1) Identify the entire bolt, including: extracting the overall position of the column bolt based on the YOLO V10 model, extracting the maximum height of the bolt based on the upper edge of the bolt identification box, and appropriately expanding the identification box to limit its area range; if there is an unidentified situation (there is a long distance without sound barrier components, such as screws and columns), fill in the unidentified bolts based on linear interpolation to reduce misidentification and missed identification;

[0105] (2) Identifying nuts, including: identifying the position of the column nuts based on the YOLO V10 model and calculating the number of nuts on each screw;

[0106] (3) Identifying the screw, including: identifying the position of the column screw based on the YOLO V10 model and calculating the exposed screw length, wherein the calculation of the exposed screw length includes: calculating the exposed screw length based on the exposed screw identification frame and pixel accuracy identified by the YOLO V10 model; and determining the number of threads of the exposed screw portion based on the screw parameters and the exposed screw length.

[0107] S22, determining whether there is an abnormal problem of bolt missing, a problem of screw loosening, and an abnormal problem of not reserving a screw surplus as required according to the positional relationship between the nut and the screw;

[0108] In this embodiment, the S22 includes:

[0109] If it is determined that the bolt missing abnormal problem exists, the bolt missing parts will be screened out and not included in the bolt loosening statistics;

[0110] If the number of deductions is less than the specified number, it is determined that there is a problem of screw loosening;

[0111] If the screw is not identified, the position of the top nut in the bolt is determined. If the top nut position exceeds the normal position of the screw, it is considered that serious looseness has occurred, the nut has covered the screw, and there is an abnormal problem that the screw margin required by regulations is not reserved.

[0112] S23, determining whether there is an abnormal bolt falling-off problem according to the number of identified nuts, including: if the number of nuts on each screw rod is lower than the standard number, confirming that there is an abnormal bolt falling-off problem.

[0113] As a preferred embodiment, column bolts are important sound barrier structural components that maintain the stability of the columns. Usually, a column contains a group of bolts, and each group of bolts contains 3 horizontally arranged bolts. Each bolt includes a screw rod and 2 nuts. The nuts are tightly screwed on the screw rod, and the screw rod has 2-3 threads exposed. However, due to problems such as installation and aging of components, some bolts may have abnormal conditions such as missing bolts and loose bolts. In order to effectively detect bolt anomalies, the YOLO V10 method is used to extract the position of the bolts, and complete the attribute information calculation to determine whether there is an abnormality. Specifically, Figure 5 shown.

[0114] S3, performing outlier detection, clustering and interpolation on the recognition results based on the DBSCAN algorithm, thereby realizing automatic filtering of clusters to improve the accuracy of the recognition results;

[0115] As a preferred embodiment, the implementation principle of S3 is: based on the statistical outlier missing and filling method, combined with artificial experience information and statistical information, the finely identified outliers are filtered out by statistical methods and the missing identification results are filled. Specifically, the column spacing and height are usually regular, the arrangement of the screws is usually evenly spaced, and the screw height, width, and nut height and width are basically similar. In combination with this feature, the position of different components of the bolt are obtained by clustering based on the DBSCAN method, and the reference values ​​of length and width. For those that deviate from the reference value by more than N times (a settable threshold), they are automatically filtered out. At the same time, if there is a long distance without sound barrier components (such as screws, columns), they are automatically filled by linear interpolation to reduce misidentification and missed identification.

[0116] Based on the common characteristics of sound barrier structural components, and using statistical and machine learning methods to implement post-processing of identification structures, wrong judgments and misjudgments of targets can be screened out, and missed judgments can be eliminated, which can greatly improve the recognition accuracy of typical structural components of sound barriers.

[0117] like Figure 6 As shown, as a preferred implementation, S3 includes:

[0118] S31, screening outliers or noise points from the recognition results based on the DBSCAN method;

[0119] Based on the similarity characteristics of typical structural components of sound barriers, the recognition results that are significantly different from the attribute information such as height, width, shape, upper and lower edge positions of most sound barrier structural components can be regarded as wrong judgment results. For example, the height of most sound barrier columns is 2m, so the recognition results of sound barrier columns higher than 3m or lower than 1m can be regarded as wrong judgments. The attribute information extraction method of typical structural components of sound barriers is the DBSCAN method (Density-Based Spatial Clustering of Applications with Noise). DBSCAN is a density-based clustering algorithm that can find clusters of arbitrary shapes in a spatial database with noise. The following are the steps for outlier screening of the recognition results based on the DBSCAN method:

[0120] Determine the neighborhood: For each point in the data set, determine its neighborhood (i.e., the area centered at the point with a radius of ε);

[0121] Calculate density: If the number of points in a point's neighborhood is greater than or equal to a threshold MinPts, the point is considered a core point.

[0122] Expanding clusters: Starting from the core point, all its density-reachable points (including core points and boundary points) are grouped into one cluster;

[0123] Mark outliers: Points that are not in any cluster are considered outliers or noise points.

[0124] For the attribute data of typical structural components of sound barriers, such as bolt height, ε can be set to 25% of the bolt height, and all similar heights are found to form a cluster. The largest cluster is considered to be the cluster where normal values ​​are located, and other data are identified as outliers (outliers or noise points); finally, the center point of the largest cluster is found, which is the "mode value" of the height; any value that differs too much from the mode value is screened out to achieve missed and wrong judgment screening.

[0125] S32, interpolating the recognition result based on the DBSCAN method;

[0126] Based on the periodic characteristics of the typical structural components of the sound barrier, the distance between every two typical structural components of the sound barrier can be extracted, and the mode value of the distance can be obtained based on the DBSCAN method. If the distance between two typical structural components of the sound barrier is too close, it is considered a misjudgment and is filtered out. If the distance is too far, it is considered a missed judgment. The number of missed targets is calculated based on the相聚 distance, and targets are evenly inserted therein to achieve interpolation. The specific process of interpolating the recognition result based on the DBSCAN method is as follows:

[0127] Calculate the period: Set the structural component position array as A, where Ai represents the position of the i-th structural component, and D represents the array composed of the distances between two sound barrier structural components, Di = Ai+1 - Ai;

[0128] Calculate the mode value: Extract the mode value K of the array D through the DBSCAN method;

[0129] Filter misjudgments: Set a minimum period threshold Cmin, traverse D, if Di < Cmin, it is considered that the recognition of the structural component i is incorrect, and the recognition result i is discarded;

[0130] Missed judgment interpolation: Set a maximum period threshold Cmax, traverse D, if Di > Cmax, it is considered that there are missed structural components between the structural component i and the structural component i+1. Calculate the interpolation number n = round(Di / K) - 1, where round represents the rounding function; finally, n similar structural components are inserted between the structural component i and the structural component i+1.

[0131] S33, perform correlation calibration on the recognition result based on the DBSCAN method;

[0132] Based on the correlation characteristics of the typical structural components of the sound barrier, for example, a group of unit panels must be sandwiched between columns, and a group of bolts must be sandwiched under the columns, missed judgment supplementation and misjudgment filtering are realized. The specific process of performing correlation calibration on the recognition result based on the DBSCAN method is as follows:

[0133] Calculate the confidence of associated components: Each component provides a certain confidence (less than 1), and the sum of the confidences of the components is 1. The more sound barrier structural components are extracted at related positions, the higher the confidence.

[0134] Filter misjudgments: Set a confidence threshold L. If the confidence of a group of components is lower than this threshold, it is considered a misjudgment and is filtered out.

[0135] Missed judgment interpolation: If the confidence of a group of components is higher than the confidence threshold L but less than 1, it is considered that some structural components are missed, and they are supplemented at the corresponding positions.

[0136] S4, identifying defects of the railway sound barrier bolts based on the processed identification results.

[0137] Based on the automatic identification of bolts, nuts and screws, based on the identification results and combined with the ledger information, the bolt property information calculation and disease identification work are completed, and finally the automatic screening of screw abnormalities is realized.

[0138] Embodiment 2

[0139] like Figure 7 As shown, this embodiment provides a railway sound barrier bolt loosening intelligent identification system, which is used to implement the method of embodiment 1, so as to intelligently detect and identify the railway sound barrier bolt loosening state, including:

[0140] The bolt foreground segmentation module 101 is used to obtain a railway sound barrier image, identify the railway sound barrier image based on deep learning, determine the sound barrier structural components to achieve the approximate positioning of the railway sound barrier bolts to obtain position information, and calibrate the pixel accuracy of the railway sound barrier image and the position information; wherein the sound barrier structural components include columns, unit panels and bolts;

[0141] A bolt shape recognition module 102 is used to recognize the type and component position of the bolt based on a bolt structure refinement recognition method so as to perform fine recognition of the bolt state;

[0142] The bolt post-processing module 103 is used to perform outlier detection, clustering and interpolation on the recognition results based on the DBSCAN algorithm, thereby realizing automatic filtering of clusters to improve the accuracy of the recognition results;

[0143] The intelligent detection and identification module 104 is used to identify defects of the railway sound barrier bolts based on the processed identification results.

[0144] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the method as in the first embodiment.

[0145] like Figure 8 As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301, the memory 302 stores a plurality of instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the method as in the first embodiment.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligently identifying loose bolts of railway sound barriers, characterized in that: Intelligent detection and identification of the loosening status of railway sound barrier bolts, including: S1, obtaining a railway sound barrier image, and identifying the railway sound barrier image based on deep learning, determining the sound barrier structural components to achieve the approximate positioning of the railway sound barrier bolts to obtain position information, and calibrating the pixel accuracy of the railway sound barrier image and the position information; wherein the sound barrier structural components include columns, unit panels and bolts; S2, based on the bolt structure refinement identification method, identifies the type and component position of the bolt to finely identify the bolt status; S3, performing outlier detection, clustering and interpolation on the recognition results based on the DBSCAN algorithm, thereby realizing automatic filtering of clusters to improve the accuracy of the recognition results; S4, identifying defects of the railway sound barrier bolts based on the processed identification results.

2. According to claim 1, a method for intelligently identifying loosened bolts of railway sound barriers is characterized in that: The S1 includes: S11, establishing a deep learning model for identifying the sound barrier structural components, inputting a plurality of historical sound barrier images with marked sound barrier structural components into the deep learning model and training the deep learning model to obtain a sound barrier structural component recognition model; completing automatic sound barrier structural component recognition based on the sound barrier structural component recognition model to obtain the recognition result of the sound barrier structural component; S12, aggregating the characteristics of different structural components of the railway sound barrier into common common characteristics, and extracting attribute information based on the common characteristics; the attribute information includes pixel accuracy calibration information, entrance and exit identification information, sound barrier height calculation information and sound barrier length calculation information; S13, calibrating the recognition result of the sound barrier structural component based on the attribute information.

3. The method for intelligently identifying loosened bolts of railway sound barriers according to claim 2 is characterized in that: The deep learning model is a YOLO V10 model, including: (1) Backbone network: A deep convolutional neural network is used, including multiple convolutional layers, batch normalization layers and activation functions; the convolutional layers are used to extract local features of the image; the batch normalization layers are used to accelerate the training process and reduce the model's sensitivity to initial weights; the activation function is used to increase the nonlinearity of the network and help the network learn complex features; (2) Neck network: It includes multiple feature pyramid network layers or path aggregation network layers, which are used to fuse features of different scales; the feature pyramid network layer is used to construct a feature pyramid from top to bottom to enhance the model's detection capability for multi-scale targets; the path aggregation network layer is used to enhance feature transfer from bottom to top to further improve the feature fusion effect; (3) Head network: used to generate the final detection result; the head network includes a convolution layer and a detection layer, the convolution layer is used to further process the feature map; the detection layer is used to predict the offset, width, height, confidence and category probability of the bounding box at each feature map position; (4) Loss function: used to guide model training, including positioning loss, classification loss, object confidence loss and non-object confidence loss; wherein the positioning loss is used to calculate the position error between the predicted bounding box and the true bounding box; the classification loss is used to calculate the error between the predicted category and the true category; the object confidence loss is used to calculate the error between the predicted confidence and the true confidence; the non-object confidence loss is used to penalize the background area to reduce false detection.

4. The method for intelligently identifying loosened bolts of railway sound barriers according to claim 3 is characterized in that: The S2 includes: S21, based on the deep learning model, refine and identify the bolt as a whole and the positions of various components of the bolt structure and calculate attribute information, wherein the positions of various components include the nut and the screw; S22, determining whether there is an abnormal problem of bolt missing, a problem of screw loosening, and an abnormal problem of not reserving a screw surplus as required according to the positional relationship between the nut and the screw; S23, determining whether there is an abnormal bolt falling-off problem according to the number of identified nuts, including: if the number of nuts on each screw rod is lower than the standard number, confirming that there is an abnormal bolt falling-off problem.

5. The method for intelligently identifying loosened bolts of railway sound barriers according to claim 4 is characterized in that: The S21 includes: (1) Identify the entire bolt, including: extracting the overall position of the column bolt based on the YOLO V10 model, extracting the maximum height of the bolt based on the upper edge of the bolt identification box, expanding the identification box and limiting the area range; if there is any unidentified situation, fill it in based on the linear interpolation method to obtain the unidentified bolt; (2) Identifying nuts, including: identifying the position of the column nuts based on the YOLO V10 model and calculating the number of nuts on each screw; (3) Identifying the screw, including: identifying the position of the column screw based on the YOLO V10 model and calculating the exposed screw length, wherein the calculation of the exposed screw length includes: calculating the exposed screw length based on the exposed screw identification frame and pixel accuracy identified by the YOLO V10 model; and determining the number of threads of the exposed screw portion based on the screw parameters and the exposed screw length.

6. The method for intelligently identifying loosened bolts of railway sound barriers according to claim 5 is characterized in that: The S22 includes: If it is determined that the bolt missing abnormal problem exists, the bolt missing parts will be screened out and not included in the bolt loosening statistics; If the number of deductions is less than the specified number, it is determined that there is a problem of screw loosening; If the screw is not identified, the position of the top nut in the bolt is determined. If the top nut position exceeds the normal position of the screw, it is considered that serious looseness has occurred, the nut has covered the screw, and there is an abnormal problem that the screw margin required by regulations is not reserved.

7. The method for intelligently identifying loosened bolts of railway sound barriers according to claim 6 is characterized in that: The S3 includes: S31, screening the recognition results for outliers or noise points based on the DBSCAN method, including: treating the recognition results that are significantly different from the attribute information of most sound barrier structural components as misjudgment results, including: Determine the neighborhood: For each point in the data set, determine its neighborhood; Calculate density: If the number of points in a point's neighborhood is greater than or equal to a threshold MinPts, the point is considered a core point; Expanding clusters: Starting from the core point, all points that can be reached by density are grouped into one cluster; Marking outliers: Points that are not in any cluster are considered outliers or noise points; S32, interpolating the recognition result based on the DBSCAN method, including: based on the periodic characteristics of the typical structural components of the sound barrier, extracting the distance between every two typical structural components of the sound barrier, and obtaining the mode value of the distance based on the DBSCAN method, if the two sound barrier structural components are too close to each other, it is considered to be a misjudgment and is screened out, if the two sound barrier structural components are too far apart, it is considered to be a missed judgment, the number of missed targets is calculated according to the distance between them, and the targets are evenly inserted therein to achieve interpolation; S33, performing correlation calibration on the recognition result based on the DBSCAN method, including: Calculate the confidence of associated components: each component provides a confidence less than 1, and the sum of the confidences of the components is 1. The more sound barrier structure components extracted at the associated positions, the higher the confidence; Screening false positives: Set a confidence threshold L. If the confidence of a group of components is lower than the threshold L, it is considered a false positive and is filtered out. Missed interpolation: If the confidence of a group of components is higher than the confidence threshold L but less than 1, it is considered that some structural parts are missed and will be supplemented at the corresponding positions of the components.

8. A railway sound barrier bolt loosening intelligent identification system, used to implement any of the methods described in claims 1-7, so as to intelligently detect and identify the railway sound barrier bolt loosening state, characterized in that: include: A bolt foreground segmentation module (101) is used to obtain a railway sound barrier image, identify the railway sound barrier image based on deep learning, determine the sound barrier structural components to achieve the approximate positioning of the railway sound barrier bolts to obtain position information, and calibrate the pixel accuracy of the railway sound barrier image and the position information; wherein the sound barrier structural components include columns, unit panels and bolts; A bolt shape recognition module (102) is used to identify the type and component position of the bolt based on a bolt structure refinement recognition method so as to perform fine recognition of the bolt state; A bolt post-processing module (103) is used to perform outlier detection, clustering and interpolation on the recognition results based on the DBSCAN algorithm, thereby realizing automatic filtering of clusters to improve the accuracy of the recognition results; An intelligent detection and identification module (104) is used to identify defects in the railway sound barrier bolts based on the processed identification results.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method according to any one of claims 1 to 7.

Images