X-ray image defect detection method and device, equipment and storage medium

Abstract

The invention discloses a defect detection positioning method, device and equipment based on an X-ray image and a storage medium, and relates to the field of image detection.An X-ray image of a conveying belt is obtained through an X-ray scanning camera, and a conveying belt joint detection model and target detection are trained based on the X-ray image; when a conveyor belt joint is identified in the X-ray image, determining the number of a target joint and position coordinates on the conveyor belt according to the joint image; and when the target defect on the conveying belt is detected, positioning the defect coordinate of the defect on the conveying belt according to the position coordinate of the target joint and the time interval between the target defect and the target joint. According to the scheme, the conveyor belt joint is detected through X rays, the conveyor belt section is divided through the joint, when the defect point is detected, defect positioning is achieved according to the distance value of the defect point in the joint section, and non-contact and abrasion-free defect detection, speed measurement and positioning are achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image detection, and in particular to a defect detection and positioning method, device, equipment and storage medium based on X-ray images. Background Art

[0002] In the industrial field, the speed measurement and positioning of conveyor belts mostly rely on devices such as encoders, Hall sensors, and RFID technology. Although these technologies have shown high accuracy in industrial field applications, they face many challenges and limitations during long-term operation. For example, in underground working areas, especially near mining surfaces, the content of harmful gases such as carbon monoxide and carbon dioxide is generally high. These gases not only pose a threat to human health, but also easily corrode and damage electronic equipment, affecting the long-term stable operation of speed measuring devices. The environment is usually humid and the air is humid, which can easily cause rust and aging of mechanical equipment and electronic components. For encoders that rely on mechanical contact for counting, the long-term friction between the belt and the roller and the humid environment can easily cause the transmission joint to get stuck or recover in time, thereby affecting the speed measurement accuracy. In a humid environment, even if the Hall sensor is used to detect the rotation of the roller, it is difficult to avoid the situation where the roller rotates poorly or stops rotating due to rust, which directly leads to the accumulation of errors and reduced accuracy of the speed measurement data.

[0003] In related technologies, RFID technology is used to achieve non-contact speed measurement and positioning. However, the need to pre-embed RFID tags during the belt production process increases production costs and limits its flexible application on existing belts. Summary of the invention

[0004] The embodiments of the present application provide a defect detection and positioning method, device, equipment and storage medium based on X-ray images to solve the problem of inaccurate conveyor belt defect detection and positioning.

[0005] In one aspect, the present application provides a defect detection and positioning method based on X-ray images, the method comprising:

[0006] Acquire an X-ray image of the conveyor belt through an X-ray scanning camera, and train a conveyor belt joint detection model and target detection based on the X-ray image; the joint detection model is used to identify the conveyor belt joint in the X-ray image;

[0007] In response to identifying a conveyor belt joint in the X-ray image, determining a number of a target joint and a position coordinate on the conveyor belt according to the joint image;

[0008] In response to detecting a target defect on the conveyor belt, the defect coordinates on the conveyor belt where the defect is located are located according to the position coordinates of the target joint and the time interval between the target defect and the target joint.

[0009] Specifically, the training conveyor belt joint detection model includes:

[0010] Classify and annotate X-ray images containing complete conveyor belt joints to build a training data set;

[0011] The CSPDarknet neural network is used as the feature extraction backbone network to build the joint detection model architecture, and the bounding box regression loss L1, confidence loss L2, and category loss L3 are constructed according to the bounding box regression, confidence, and category recognition results of the target detection results; they are expressed as follows:

[0012]

[0013]

[0014]

[0015] A and B are the predicted box and the real box area of ​​the conveyor belt joint respectively, and C is the smallest closed area package that can contain A and B; and p obj are the confidence of the predicted inclusion and exclusion of the target, respectively. and p noobj is the corresponding true label; BCE(·) represents the confidence calculation; is the predicted category probability distribution, c is the true category label, and CE(·) represents the cross entropy loss calculation;

[0016] The total loss of the model is calculated based on the bounding box regression loss L1, the confidence loss L2, and the category loss L3, which is expressed as follows:

[0017] Loss = λ box L1+λ conf L2+λ cls L3

[0018] The λ box , conf and λ cls They are the weight parameters for bounding box loss, confidence loss, and category loss, respectively.

[0019] Specifically, determining the number of the target joint and the position coordinates on the conveyor belt according to the joint image includes:

[0020] intercepting a joint image containing a complete target joint from the X-ray image, and extracting a joint feature vector through a trained feature extraction model;

[0021] Based on the extracted feature vector, the historical joint images in the joint image library are matched, and the joint ID number corresponding to the target joint is determined according to the matching result; the joint image library stores the joint images and number labels of all the joints of the conveyor belt;

[0022] The position coordinates of the target joint corresponding to the target joint on the conveyor belt are determined according to the ID number size of the target joint and the distance between adjacent joints.

[0023] Specifically, the current average conveyor belt speed v is obtained avg ,include:

[0024] When the iIDth joint is detected, the time difference Δt until the complete joint image is obtained and the joint offset distance d(t) when the complete joint image is obtained are determined. The initial average speed v is calculated based on the time difference Δt and the offset distance d(t). avg0 (t); is represented as follows:

[0025]

[0026] Where Δx represents the pixel displacement difference of the joint in the joint image, Δt1 represents the frame rate, n represents the number of images detected, and l p Indicates the length of the unit pixel mapped to the conveyor belt;

[0027] When the k+1th ID joint image is detected, the actual displacement value D and time interval ΔT between the adjacent kth ID joint are calculated, and the average distance v of the kth ID joint is updated according to the actual displacement value D and time interval ΔT between the two joints. avg , the formula is as follows:

[0028]

[0029] The D value is obtained based on historical location distance data.

[0030] On the other hand, the present application provides a defect detection and positioning device based on X-ray images, the device comprising:

[0031] A detection module, used for acquiring an X-ray image of the conveyor belt through an X-ray scanning camera, and training a conveyor belt joint detection model and target detection based on the X-ray image; the joint detection model is used for identifying the conveyor belt joint in the X-ray image;

[0032] a determination module, for determining the number of a target joint and the position coordinates on the conveyor belt according to the joint image in response to identifying the conveyor belt joint in the X-ray image;

[0033] The positioning module is used to locate the defect coordinates on the conveyor belt where the defect is located according to the position coordinates of the target joint and the time interval between the target defect and the target joint in response to detecting the target defect on the conveyor belt.

[0034] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the X-ray image-based defect detection and positioning method described in the above aspects.

[0035] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the X-ray image-based defect detection and positioning method described in the above aspects.

[0036] The beneficial effects of the technical solution provided by the embodiment of the present application include at least: the present application detects the conveyor belt joints by X-rays, uses the joints to indirectly divide the conveyor belt into sections, and then when the defect point is detected, the defect location can be achieved according to the distance value of the defect point in the joint section, realizing non-contact, wear-free defect detection, speed measurement and positioning, which has important practical application value and broad market prospects. This technology can not only effectively overcome the negative impact of special environments in different industries on speed measuring devices, but also improve the speed measurement accuracy and the overall stability of the system, providing strong guarantees for safe production and efficient operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of a defect detection and positioning method based on X-ray images provided in an embodiment of the present application;

[0038] Figure 2 It is a schematic diagram of distance measurement based on the conveyor belt joint;

[0039] Figure 3 This is a schematic diagram of the traditional yolo v8 model structure;

[0040] Figure 4 It is a structural schematic diagram of the improved conveyor belt joint detection model;

[0041] Figure 5 It is a schematic diagram of the structure of the phantom convolution module;

[0042] Figure 6 This is a schematic diagram of the improved BiFormer structure in backbone;

[0043] Figure 7 It is the loss curve diagram of iterative training with different loss functions;

[0044] Figure 8 , a schematic diagram showing the detection timing with the edge of the joint as the pixel;

[0045] Fig. 9 A schematic diagram of defect recognition and positioning using the edge of the joint as the pixel detection standard is shown;

[0046] Fig.10 It is an algorithm flow chart of a feasible defect detection and positioning method based on X-ray images;

[0047] Fig.11 The structural block diagram of the defect detection and positioning device based on X-ray images is shown;

[0048] Fig.12 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0050] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0051] Figure 1 : is a flow chart of a defect detection and positioning method based on X-ray images provided in an embodiment of the present application, comprising the following steps:

[0052] S1, obtain the X-ray image of the conveyor belt through the X-ray scanning camera, and train the conveyor belt joint detection model and target detection based on the X-ray image;

[0053] Because traditional encoders are not good at distance measurement and positioning, this application chooses to use the joints on the conveyor belt to assist in positioning. The length of the conveyor belt is usually several hundred meters. During the reproduction process, multiple independent conveyor belts are often used to splice into a complete conveyor belt, so joints are provided between adjacent belt bodies. For a certain conveyor belt, the number of joints must be fixed. Therefore, this application uses the joint as an auxiliary positioning component to replace the encoder distance measurement (length) positioning.

[0054] Considering the special processing technology of the joint, the conveyor belts at both ends are stretched by steel wire or belt wire bite, and its internal structure cannot be directly identified by color camera, so this application uses X-ray scanning camera instead of traditional camera. Defects on the conveyor belt can also be presented by X-ray images. Of course, in order to improve the accuracy of detection, CMOS cameras and other equipment can also be installed to detect defects simultaneously.

[0055] Because the premise of defect detection and positioning is to first identify the conveyor belt joint, this application needs to separately design a conveyor belt joint detection model to complete the target (defect) detection.

[0056] It is particularly important to note that the X-ray images used in the training model must contain complete joint images to complete data classification annotation and test training. Specifically, advanced detection algorithms such as YOLOv5 can be used to train the annotated data, continuously optimize model parameters, and improve recognition accuracy and speed. A specially designed X-ray scanning camera is used to ensure that the resolution and clarity of the image meet the requirements so as to accurately capture every detail on the belt. The camera works synchronously with the belt motion system to ensure that each pixel or line segment in the image can accurately correspond to the corresponding length of the actual belt, avoiding image overlap or misalignment problems that may exist in traditional methods.

[0057] S2, in response to identifying a conveyor belt joint in the X-ray image, determining a number of a target joint and a position coordinate on the conveyor belt according to the joint image;

[0058] Because the joints and quantity of the conveyor belt are fixed, you only need to adjust the algorithm and assign a unique ID number to each joint. The first ID joint is the default initial starting point, and all subsequent length calculations are accumulated and summed on this basis. Figure 2 This is a schematic diagram of distance measurement based on the conveyor belt joints. Assuming that there is a defect between the second and third joints, the coordinates of the defect point can be expressed as L2+l2, where L2 represents the distance from the origin (the first joint) to the second joint, and l2 represents the distance from the defect point to the second joint. Because the conveyor belt is cyclical, and the position of the joints can be calculated after the initial calculation of the position coordinates of each joint such as L1, L2, and L3, this calculation method actually only needs to directly calculate the position from the defect point to the previous nearest joint after the defect point is detected. This will not produce too much cumulative error and the positioning will be more accurate.

[0059] When the belt is running, the conveyor belt can be set to update the position coordinates of each joint once every N complete cycles to prevent the belt from expanding or contracting or pulling and deforming and affecting the measurement results. In each cycle of detection, whenever the joint detection model detects a joint from the X-ray image, it will perform position calculations based on the identified joint number. Optionally, an image recognition library can be established to determine the ID number through image recognition and feature matching, and then calculate the position coordinates. The calculation of joint coordinate update is determined based on the running speed of the conveyor belt and the time of detecting the joint.

[0060] S3, in response to detecting a target defect on the conveyor belt, locating the defect coordinates on the conveyor belt where the defect is located according to the position coordinates of the target joint and the time interval between the target defect and the target joint.

[0061] Defect detection is synchronized with joint detection and recognition. When a certain defect point is identified from the image, the position of the target joint must be determined first. The target joint can be any joint. The following text uses adjacent joints for example. Then, the absolute distance value of the defect can be determined by counting the time interval between the target defect and the target joint through the clock and multiplying it by the average speed of the conveyor belt, which is equivalent to locating the defect coordinates on the conveyor belt where the defect is located.

[0062] In summary, this application detects the conveyor belt joints through X-rays, uses the joints to indirectly divide the conveyor belt into sections, and then when the defect point is detected, the defect location can be achieved based on the distance value of the defect point in the joint section, realizing non-contact, wear-free defect detection, speed measurement and positioning, which has important practical application value and broad market prospects. This technology can not only effectively overcome the negative impact of special environments in different industries on speed measuring devices, but also improve the speed measurement accuracy and the overall stability of the system, providing strong guarantees for safe production and efficient operation.

[0063] In some embodiments, for the training of the conveyor belt joint detection model, a large amount of X-ray image data containing features such as joints and defects must first be collected to construct a data set for training deep learning models. Then the data set is manually annotated to distinguish target categories such as joints and defects, and necessary image preprocessing operations are performed, such as denoising and contrast enhancement. In practical applications, according to the actual situation of the conveyor belt, the types that need to be identified are defined, such as joints, embeddings, broken wires, twitching, etc., and professionals annotate the collected X-ray images one by one. Draw the bounding box of the defective area on the image and assign the corresponding defect type label. During the annotation process, attention should be paid to accuracy to ensure that each defect is correctly identified. The annotated data is preprocessed, including image size normalization and data enhancement (such as rotation, flipping, scaling, etc.) to increase the robustness of the model. The data should be divided into training set, validation set and test set.

[0064] When building the model, you can use advanced detection algorithms such as YOLOv8 to train the labeled data, use the CSPDarknet neural network as the feature extraction backbone network, and build the joint detection model architecture. Figure 3 This is a schematic diagram of the traditional yolov8 model structure, including:

[0065] 1) Input

[0066] The original image (such as a wire rope image) is input into the network for processing. Usually, this includes preprocessing steps such as image resizing and normalization. For YOLOv8, the size of the input image is generally fixed to 640x640x3 (width x height x number of channels) to accommodate the requirements of subsequent convolution operations.

[0067] 2) Backbone

[0068] Darknet53 structure: The backbone network of YOLOv8 adopts the improved Darknet53 structure. Darknet53 is a deep convolutional neural network originally used in YOLOv3. It has multiple layers of convolution and residual connections (ResNet-like connections) and can effectively extract low-level to high-level features of images.

[0069] Conv unit: The convolution unit (Conv) in the backbone network gradually downsamples the input image through a series of convolution operations to extract feature maps. Each convolution operation is usually followed by the application of batch normalization and activation functions (such as SiLU) to improve the training efficiency and stability of the network.

[0070] SPPF module: The Spatial Pyramid Pooling Fast (SPPF) module is used to fuse local features with global features. SPPF performs pooling operations on feature maps at multiple scales and concatenates these pooling results, thereby retaining multi-scale information and helping to identify objects of different scales.

[0071] C2f module: The C2f module is an important module introduced in YOLOv8, which combines the features of the C3 module in YOLOv5 and the efficient aggregation network in YOLOv7. The C2f module enhances the feature expression capability by adaptively selecting and fusing input features. This modular design allows the network to improve the feature aggregation capability while maintaining computational efficiency.

[0072] 3) Neck network

[0073] CSP2 structure: The Neck part adopts the CSP2 (Cross Stage Partial) structure designed by CSPNet. CSP2 reduces information loss and enhances feature fusion capabilities by separating and fusing different parts of the input feature map. This structure can effectively capture detailed information and global semantic information in the image, especially in multi-scale detection scenarios.

[0074] Path Aggregation Network (PANet): In the subsequent stages of the Neck part, the network further fuses features through the Path Aggregation Network (PANet). PANet fuses features from different scales with sufficient contextual information through bottom-up path aggregation and top-down path enhancement, thereby improving the accuracy of small object detection.

[0075] 4) Detection output terminal (Head)

[0076] Prediction head: The detection output is mainly composed of multiple prediction heads, each of which is responsible for detecting objects of different scales. The prediction head predicts the bounding box, category probability, and target confidence of each anchor point through specific regression and classification branches. The Head part in YOLOv8 is designed to be simple and efficient, and can output high-precision target detection results in real time.

[0077] Multi-scale prediction: Since YOLOv8 makes predictions on feature maps of different resolutions, it can detect objects of different sizes. This multi-scale prediction design makes the network perform better when dealing with diverse objects in complex scenes.

[0078] For the X-ray defect detection task, this paper specifically improves the YOLOv8 network structure to cope with several key challenges in this environment. First, there are many small X-ray targets, which increases the risk of missed detection during the detection process. Second, the conveyor belt skeleton structure is complex and the background interference is serious. Finally, since the detection model needs to run on an embedded device, strict restrictions are imposed on the network parameters and computing resources, and it is necessary to ensure that the algorithm can run smoothly on the device. Directly applying the original YOLOv8 to the actual production environment may result in missed detection and false detection problems. This is because its feature recognition module does not perform well in complex working environments such as mines. Therefore, this paper makes the following important improvements to the network structure of YOLOv8, see Figure 4 It is a schematic diagram of the structure of the improved conveyor belt joint detection model, including the following:

[0079] 1. Use GhostConv to replace traditional convolution: In order to reduce the complexity of the model and adapt it to the computing power of embedded devices, this paper replaces traditional convolution with GhostConv. GhostConv can significantly reduce the number of parameters and computing costs while maintaining the accuracy of the model, thereby ensuring the smooth operation of the model on embedded devices.

[0080] 2. Introducing the improved two-layer link attention mechanism BiFormer: BiFormer (improved two-layer link attention mechanism) is introduced after the SPPF module in the backbone network to enhance the model's image analysis and information fusion capabilities. BiFormer improves the model's detection accuracy in complex environments by better capturing the association between global and local features, significantly reducing the possibility of missed detection and false detection.

[0081] 3. Based on BiFPN, the weighted IoU (WIoU) loss function is introduced to speed up the convergence of the model. WIoU optimizes the accuracy of bounding box regression by weighting IoU, which is particularly suitable for tasks that require precise positioning, such as detecting conveyor belt tears.

[0082] Figure 5 It is a schematic diagram of the structure of the ghost convolution module. The convolution structure of the GhostConv module reduces feature redundancy and parameter quantity by decomposing the conventional convolution operation into two parts. This module uses point-to-point convolution operations to generate some feature maps, and uses these generated feature maps to construct the remaining feature maps. In this way, GhostConv convolution greatly reduces the parameters and calculation amount without reducing the performance of the model. Suppose the height, width and number of channels of the input features are h, w and c, the height and width of the output features are H and W, the number of convolution kernels is n, the convolution kernel size is k, the linear transformation convolution kernel size is d, and the number of transformations is s. rs and rc represent the ratio of the amount of calculation and the amount of parameters of ordinary convolution and GhostConv convolution, respectively. The calculation formula is as follows:

[0083]

[0084] In order to enhance the attention to the details of X-ray images, the YOLOv8 model architecture was improved, and a dynamic sparse attention module called BiFormer was integrated between Backbone and Neck. The introduction of BiFormer enables the model to process features related to target detection more centrally through an adaptive query mechanism, filtering out interference information. This not only effectively enhances the detection performance of the model, but also improves computational efficiency. Specifically, YOLO_BiFormer( YOLO_BF模型)The model structure integrates the BiFormer module after the spatial pyramid feature fusion process (SPFF), which effectively reduces unnecessary background information interference and focuses on the detection area of ​​belt tearing, thereby improving the detection accuracy in complex environments.

[0085] Figure 6 This is a schematic diagram of the improved structure of BiFormer in the backbone. The architecture of BiFormer revolves around the bidirectional relative attention (BRA) module and designs a four-layer pyramid layout. In the first stage, the overlapping image block embedding technology is used to ensure the locality and globality of feature extraction. Then, in the second to fourth stages, the image merging module is used to gradually extract higher-level semantic features by reducing the image resolution and increasing the channel capacity. Finally, the features of the input data are transformed and fused through the series of BiFormer modules.

[0086] In the overall architecture of BiFormer, the beginning of each BiFormer block uses 3x3 depth-separable convolution to implicitly encode the location information of joints and defects. Next, the BRA module is used to capture bidirectional relative attention to achieve effective focus and integration of features. Finally, the relational model is constructed through a multi-layer perceptron (MLP), and the location information is further embedded to make the feature expression more accurate and rich. These improvements enable the YOLO_BiFormer model structure to show higher robustness and accuracy when dealing with conveyor belt damage detection tasks in complex environments in coal mines, significantly improving the detection effect in practical applications.

[0087] According to the bounding box regression, confidence and category recognition results of the target detection results, the bounding box regression loss (Bounding Box Regression Loss) L1, confidence loss (Confidence Loss) L2, and category loss (Classification Loss) L3 are constructed respectively. In order to reduce the impact of strong background interference in coal mining operations, this paper uses WIoU to replace the traditional CIoU of the YOLO algorithm. In the CIoU loss function, all targets are treated equally, and the quality of the labeled examples of the dataset itself is rarely considered. The WIoU loss function is designed to solve the problems of category imbalance and target size imbalance in target detection tasks. This gives it better performance when facing targets with different geometric factors and improves the generalization performance of the model.

[0088] They are respectively expressed as follows:

[0089]

[0090]

[0091]

[0092]

[0093] The bounding box regression loss is used to calculate the difference between the predicted bounding box and the true bounding box, the confidence loss is used to calculate the confidence error of whether the predicted bounding box contains the target; the category loss is used to calculate the error between the predicted category and the true category.

[0094] In the above formula, R WIOU is the gradient gain, L IOU is the sliding average of momentum, β describes the outlier degree of the anchor box quality, and the mapping relationship between r and β is determined by the hyperparameters α and δ; Wg and Hg are the sizes of the minimum bounding box, (x, y) and (x gt ,y gt ) are the coordinates of the center points of the anchor box and the target box respectively, * indicates to prevent R WIOU Operations that produce gradients that hinder convergence, separating Wg and Hg from the computation graph. and p obj are the confidence of the predicted inclusion and exclusion of the target, respectively. and p noobj is the corresponding true label; BCE(·) represents the confidence calculation. is the predicted category probability distribution, c is the true category label, and CE(·) represents the cross entropy loss calculation.

[0095] The total loss of the model is calculated based on the bounding box regression loss L1, the confidence loss L2, and the category loss L3, which is expressed as follows:

[0096] Loss = λ box L1+λ conf L2+λ cls L3

[0097] The λ box , conf and λ cls They are the weight parameters of bounding box loss, confidence loss and category loss, which are used to adjust the impact of each part of the loss on the total loss.

[0098] In order to further improve the performance of the target detection model, this application deeply explores the impact of different bounding box regression loss functions on model performance under the YOLOv8 architecture. The original YOLOv8 model uses the CIoU (Complete IoU) loss function, which shows good fitting performance during model training. However, since the aspect ratio of the predicted box is calculated as a relative quantity, the inherent uncertainty of this calculation method, coupled with the different effects on high-quality and low-quality anchor boxes, may cause obstacles in the optimization process of regression loss.

[0099] To address this problem, this study proposed and analyzed the YOLO_BF model, focusing on comparing the effects of different loss functions such as WIoU (Weighted IoU), CIoU, EIoU (Extended IoU) and SIoU (Smooth IoU) on the convergence and final performance of the model, aiming to find a better loss function solution to improve the overall performance of the model.

[0100] Through a series of detailed experimental comparisons, such as Figure 7 As shown in the figure, the experimental results show that the convergence speed and final loss value of the model show significant differences under different loss functions. Specifically, the model using the EIoU loss function has the slowest convergence speed and the highest final loss value, showing low training efficiency. SIoU and CIoU are slightly better than EIoU in terms of convergence speed and final loss value, showing faster convergence speed and lower loss value.

[0101] It is particularly noteworthy that after the introduction of the WIoU loss function, the gradient descent speed of the model in the early stage of training is significantly accelerated, especially in the first 20 iterations, and convergence is quickly reached. At the same time, the final loss value of WIoU is also significantly lower than that of the other three loss functions, showing better regression performance. Further analysis shows that the use of the WIoU loss function not only accelerates the convergence speed of the model, but also achieves significant improvements in key performance indicators such as accuracy, recall, and mean average precision (mAP).

[0102] In summary, the application of WIoU loss function in YOLO_BF model not only optimizes the convergence efficiency during training, but also improves the overall performance of the model in actual detection tasks. This study shows that choosing a suitable bounding box regression loss function is of great significance for improving the accuracy and efficiency of the target detection model, and provides new ideas for further optimizing YOLOv8 and its variants.

[0103] The training process can be summarized as follows:

[0104] Initialization: Load the pre-trained model weights, initialize the optimizer (such as SGD or Adam) and the learning rate scheduler.

[0105] Forward propagation: Pass the input image through the model and calculate the output bounding box coordinates, confidence, and class probabilities.

[0106] Calculate loss: Use the above comprehensive loss function to calculate the error between the predicted value and the true value.

[0107] Back propagation: Update the model weights through the error back propagation algorithm.

[0108] Iterative training: Repeat the above process until the loss converges or the preset number of training times is reached.

[0109] Verification and adjustment: Evaluate model performance on the validation set, adjust model parameters and hyperparameters, and perform model tuning.

[0110] In some embodiments, determining the number of the target joint and the position coordinates on the conveyor belt may specifically include the following steps:

[0111] 1. Capture a joint image containing a complete target joint from the X-ray image, and extract the joint feature vector through the trained feature extraction model;

[0112] The X-ray image can be obtained by fusing the X-ray scanning sequence images. According to the device frame rate, n consecutive sequence images containing complete joint images are extracted for image stitching and fusion to obtain the output X-ray image. The present application obtains it with 2 to 3 stitched images.

[0113] Locate the complete joint rectangular coordinates. If the joint is identified, extract the coordinates. Capture the complete joint image based on the obtained coordinates to prepare for subsequent feature extraction and vector comparison.

[0114] 2. Match the extracted feature vector with the historical joint images in the joint image library, and determine the joint ID number corresponding to the target joint according to the matching result;

[0115] In this step, the feature extraction model needs to be trained in advance. The model architecture selects ResNet or DenseNet feature extraction network as the backbone network to build a feature extraction model; use the preprocessed data set to train the model. During the training process, adjust the model's hyperparameters (such as learning rate, batch size, etc.), and use the validation set to monitor the model's performance to prevent overfitting. Use the test set to evaluate the trained model, calculate indicators such as accuracy, recall rate, F1 score, and determine the actual effect of the model. Based on the evaluation results, further model tuning may be required. Deploy the trained model to the on-site X-ray detection system to ensure that it can run stably on the edge device. During deployment, ensure that the model can process images in real time and output the feature vector of the joint.

[0116] In the process of preparing the data set, the joint images with the same ID number acquired at different times are grouped into training group samples, and the joint images of all training group samples are grouped together into a feature image training set;

[0117] Constructing a triplet loss function based on the feature image training set It is expressed as follows:

[0118]

[0119] Where a is the reference point, p is the positive sample, n is the negative sample, d(x, y) represents the distance between two samples, and αα is a hyperparameter, which represents the minimum difference between the distance between ap and the distance between an.

[0120] Triplet loss is used for metric learning, and the goal is to make the joint feature extraction matches of the same ID number closer and the joint features of different IDs farther.

[0121] During the matching process, it is necessary to establish a joint image library in advance, which stores the joint images and number labels of all joints of the conveyor belt. During extraction, a screenshot is taken based on the obtained joint target frame result, and model inference is performed on the complete joint to obtain 512-dimensional vector information. The current vector result is compared with the joint vector value in the database (joint image library).

[0122] 3. According to the ID number of the target joint and the distance between adjacent joints, determine the position coordinates of the target joint on the conveyor belt.

[0123] If the match is achieved, the current joint number is obtained. At the same time, according to the previous description, the distance of each joint on the belt is fixed, so the distance of the current joint from the origin is also obtained.

[0124] In some embodiments, locating the defect coordinates on the conveyor belt where the defect is located according to the position coordinates of the target joint and the time interval between the target defect and the target joint comprises the following steps:

[0125] A. When the target defect on the conveyor belt is detected, obtain the current average speed v of the conveyor belt avg , and record the defect time point T1 of the target defect;

[0126] B. Obtain the joint time point Ts, ID number and position coordinates of the adjacent joint determined before T1, and determine the offset time difference t between the adjacent joint and the target defect point d ;

[0127] C. According to the position coordinates of the adjacent joints and the average speed v of the conveyor belt avg and the offset time difference t d, calculate the defect position L of the target defect d , which is expressed as follows:

[0128] L d =L k +v avg (t)×t d

[0129] Among them, L k Indicates the position of adjacent joints with ID numbers on the conveyor belt, with the joint with the first ID number of the conveyor belt as the starting position.

[0130] The core step of positioning is the calculation of conveyor belt speed, because the distance length calculation must rely on the speed value parameter. Therefore, this application uses the historical (last cycle) average speed to calculate the displacement value in real time.

[0131] In order to accurately calculate the average speed of the conveyor belt, the offset information between the joint ID and the image can be used. First, prepare the required initial data and prior knowledge: such as joint ID position data, pixel length conversion rate and image scanning interval. During the system initialization operation phase, when the first joint is identified (assuming the ID is k), initialize the following parameters:

[0132] Initial position Pk (e.g., the kth joint is at the 100th meter of the belt).

[0133] Initial time T0 (for example, the time of image 1 is 12:00:00).

[0134] When the iID joint is detected, the time difference Δt until the complete joint image is obtained and the joint offset distance d(t) when the complete joint image is obtained are determined. The initial average speed v is calculated based on the time difference Δt and the offset distance d(t). avg0 (t); is represented as follows:

[0135]

[0136] where Δx represents the joint in the joint image (e.g. Figure 8 The pixel displacement difference is the pixel detection standard (with the edge of the joint as the pixel detection standard), Δt1 represents the frame rate, n represents the number of images detected, and l p Indicates the length of the unit pixel mapped to the conveyor belt. Initial average speed v avg0 (t) can be used as the detection speed test result of each joint, and the actual distance to the defect is calculated based on the initial average speed of the adjacent joints, that is, v avg0 (t) = v avg (t).

[0137] Of course, the average value calculated in the above process is calculated based on the detection time of a conveyor belt joint, but this solution requires real-time frame-by-frame calculation, which is accurate but not efficient enough. For this reason, this application can use it as the initial speed and adopt the following update method on this basis:

[0138] When the k+1th ID joint image is detected, the actual displacement value D and time interval ΔT between the adjacent kth ID joint are calculated, and the average distance v of the kth ID joint is updated according to the actual displacement value D and time interval ΔT between the two joints. avg , the formula is as follows:

[0139]

[0140] The D value is obtained based on historical location distance data.

[0141] The above calculation method is based on the distance and time between the two joints. The defect distance value can also be calculated using this value. In addition, considering that the conveyor belt between the joints may deform and cause the distance value to change, the average distance value of the two joints can be used after the update cycle is reached. To update the distance value D between the joints, synchronize it to the database (joint image library) to facilitate subsequent direct calculation.

[0142] In particular, when calculating defects, it may happen that the defect is located on the joint, because the length of the joint is about 2-3m. When the defect is located in this area, directly use the output joint position coordinates, or locate the detailed coordinates based on the length of the defect from the edge, see Fig. 9 .

[0143] For the case where the defect is located at the joint, each joint image and feature vector in the original joint image library are obtained according to the initial defect-free and wear-free state. The joint will change due to wear and defects, etc., which will cause the subsequent X-ray images to change, so this application also sets up a database update mechanism.

[0144] When the conveyor belt runs to a set time or a certain length, or when the update cycle is reached, the image update is started and replaced with the latest joint images. In particular, in the case of target defects in the joint image, the joint image with the defect is immediately captured and replaced in the joint image library to prevent the joint ID number from being unable to be matched due to defects in the future.

[0145] Fig.10This is an algorithm flow chart of a feasible defect detection and positioning method based on X-ray images. First, the joint detection model is used to detect the target of all spliced ​​and fused X-ray images. The first identified joint is used as the first joint ID by default, and the initial average speed at the joint is calculated by fully detecting the pixel offset of the joint. Defect detection and joint identification are carried out simultaneously. At the same time, the complete joint image is intercepted from the X-ray image and sent to the feature extraction model to extract the joint feature vector. The feature vector is extracted to match it with the joint image (feature vector) in the database, identify the current joint ID, and calculate the joint position coordinates. In the case of detected defects, the defect coordinates are determined by the adjacent joint positions, average speed, and the time interval between the defect and the joint detection.

[0146] In particular, when the conveyor belt is cycled and detected again, the initial average speed is calculated here and updated immediately afterwards. That is, when the next joint is detected, the average speed is calculated based on the distance between the two joints and the time interval between the detections. When there is a defect later, the defect coordinates are calculated based on this average speed to achieve positioning.

[0147] In particular, defect detection and positioning on the joint will affect the feature matching results, so it is necessary to judge each captured joint image. When the defect happens to be located on the joint, the feature vector and joint image in the database are replaced according to the matching joint ID to achieve real-time update of the system and improve the accuracy of subsequent matching.

[0148] In summary, this application has the following effects:

[0149] Feature extraction and matching: Each joint in the database is assigned a unique ID. For the detected joint, its unique features (such as shape, texture, etc.) are extracted and matched with the known joint features in the database to determine the joint ID.

[0150] Speed ​​calculation: Record the time required for the joint to move from the starting position to the ending position in the image, and calculate the actual moving distance of the joint in combination with the pixel-length mapping algorithm, so as to calculate the running speed of the belt.

[0151] Position calibration: Using the ID and position information of the joint, the defect position on the belt is calibrated. This method is not only used for speed measurement (by measuring the time and distance the joint moves in the image), but also for locating defects (calculating the actual position of the defect on the belt through the ID and position information of the joint).

[0152] Real-time speed update: During the continuous operation of the belt, the real-time speed of the belt is constantly updated according to the moving speed and distance of the joint, and the moving speed and distance of the joint are calculated in real time.

[0153] Defect location prediction and correction: Between joints, the initial location of the defect is predicted using the speed and time calculated last time. When a new joint is detected, the defect location is corrected by matching the joint ID and recalculating the distance to ensure accuracy.

[0154] Abnormal detection and alarm: Set up abnormal detection mechanism. When abnormal situations such as sudden speed change and defect concentration are detected, alarm signals will be issued in time to remind operators to handle them.

[0155] Fig.11 The structure block diagram of the defect detection and positioning device based on X-ray images provided in an embodiment of the present application is shown, and the device includes:

[0156] The detection module 1110 is used to obtain an X-ray image of the conveyor belt through an X-ray scanning camera, and train a conveyor belt joint detection model and target detection based on the X-ray image; the joint detection model is used to identify the conveyor belt joint in the X-ray image;

[0157] A determination module 1120, configured to determine the number of a target joint and the position coordinates on the conveyor belt according to the joint image in response to identifying the conveyor belt joint in the X-ray image;

[0158] The positioning module 1130 is used to locate the defect coordinates on the conveyor belt where the defect is located according to the position coordinates of the target joint and the time interval between the target defect and the target joint in response to detecting the target defect on the conveyor belt.

[0159] The X-ray image-based defect detection and positioning device provided in the embodiment of the present application can be applied to the X-ray image-based defect detection and positioning method provided in the above-mentioned embodiment. For relevant details, refer to the above-mentioned method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0160] It should be noted that the X-ray image-based defect detection and positioning device provided in the embodiment of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the X-ray image-based defect detection and positioning device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the X-ray image-based defect detection and positioning method provided in the above method embodiment and the implementation method of the X-ray image-based defect detection and positioning device provided in this embodiment belong to the same concept. The specific implementation process of the X-ray image-based defect detection and positioning device provided in this embodiment is detailed in the above method embodiment and will not be repeated here.

[0161] Fig.12The block diagram of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory may be connected by a bus or in other ways. Among them, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0162] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0163] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0164] In some embodiments, the computer device may further optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit, a display screen and a keyboard.

[0165] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0166] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen is a touch screen, the display screen also has the ability to collect touch signals on the surface or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in other embodiments, the display screen can be at least two, respectively arranged on different surfaces of the computer device or in a folding design; in other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0167] The power supply is used to power various components in the computer device. The power supply can be AC, DC, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0168] Those skilled in the art will appreciate that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0169] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0170] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A defect detection and positioning method based on X-ray images, characterized in that: The method comprises: Acquire an X-ray image of the conveyor belt through an X-ray scanning camera, and train a conveyor belt joint detection model and target detection based on the X-ray image; the joint detection model is used to identify the conveyor belt joint in the X-ray image; In response to identifying a conveyor belt joint in the X-ray image, determining a number of a target joint and a position coordinate on the conveyor belt according to the joint image; In response to detecting a target defect on the conveyor belt, the defect coordinates on the conveyor belt where the defect is located are located according to the position coordinates of the target joint and the time interval between the target defect and the target joint.

2. The method according to claim 1, characterized in that The training conveyor belt joint detection model comprises: Classify and annotate X-ray images containing complete conveyor belt joints to build a training data set; The CSPDarknet neural network is used as the feature extraction backbone network to build the joint detection model architecture, and the bounding box regression loss L1, confidence loss L2, and category loss L3 are constructed according to the bounding box regression, confidence, and category recognition results of the target detection results; they are expressed as follows: A and B are the predicted box and the real box area of ​​the conveyor belt joint respectively, and C is the smallest closed area package that can contain A and B; and p obj are the confidence of the predicted inclusion and exclusion of the target, respectively. and p noobj is the corresponding true label; BCE(·) represents the confidence calculation; is the predicted category probability distribution, c is the true category label, and CE(·) represents the cross entropy loss calculation; The total loss of the model is calculated based on the bounding box regression loss L1, the confidence loss L2, and the category loss L3, which is expressed as follows: Loss=λ box ·L1+λ conf ·L2+λ cls ·L3 The λ box , conf and λ cls They are the weight parameters for bounding box loss, confidence loss, and category loss, respectively.

3. The method according to claim 2, characterized in that Determining the number of the target joint and the position coordinates on the conveyor belt according to the joint image includes: intercepting a joint image containing a complete target joint from the X-ray image, and extracting a joint feature vector through a trained feature extraction model; Based on the extracted feature vector, the historical joint images in the joint image library are matched, and the joint ID number corresponding to the target joint is determined according to the matching result; the joint image library stores the joint images and number labels of all the joints of the conveyor belt; The position coordinates of the target joint corresponding to the target joint on the conveyor belt are determined according to the ID number size of the target joint and the distance between adjacent joints.

4. The method according to claim 3, characterized in that: The method of obtaining an X-ray image of the conveyor belt comprises: Obtain the X-ray scanning sequence diagram, extract n consecutive sequence diagrams containing complete joint images according to the device frame rate, perform image stitching and fusion, and obtain the output X-ray image.

5. The method according to claim 4, characterized in that The training process of the feature extraction model includes: selecting a ResNet or DenseNet feature extraction network as a backbone network to construct a feature extraction model; The joint images with the same ID number acquired at different times are grouped into training group samples, and the joint images of all training group samples are grouped into a feature image training set; Constructing triplet loss function based on feature image training set It is expressed as follows: Where a is the reference point, p is the positive sample, n is the negative sample, d(x,y) represents the distance between two samples, and α is a hyperparameter, which represents the minimum difference between the distance between ap and the distance between an.

6. The method according to claim 5, characterized in that The method of locating the defect coordinates on the conveyor belt where the defect is located according to the position coordinates of the target joint and the time interval between the target defect and the target joint includes: When the target defect on the conveyor belt is detected, the current average speed v of the conveyor belt is obtained avg , and record the defect time point T1 of the target defect; Obtain the joint time point Ts, ID number and position coordinates of the adjacent joint determined before T1, and determine the offset time difference t between the adjacent joint and the target defect point d ; According to the position coordinates of the adjacent joints, the average conveyor belt speed v avg and the offset time difference t d , calculate the defect position L of the target defect d , which is expressed as follows: L d =L k +v avg (t)×t d Among them, L k Indicates the position of adjacent joints with ID numbers on the conveyor belt, with the joint with the first ID number of the conveyor belt as the starting position.

7. The method according to claim 6, characterized in that The current average conveyor belt speed v is obtained avg ,include: When the i-th ID joint is detected, the time difference Δt until the complete joint image is obtained and the joint offset distance d(t) when the complete joint image is obtained are determined. The initial average speed v is calculated based on the time difference Δt and the offset distance d(t). avg0 (t); is represented as follows: Where Δx represents the pixel displacement difference of the joint in the joint image, Δt1 represents the frame rate, n represents the number of images detected, and l p Indicates the length of the unit pixel mapped to the conveyor belt; When the k+1th ID joint image is detected, the actual displacement value D and time interval ΔT between the adjacent kth ID joint are calculated, and the average distance v of the kth ID joint is updated according to the actual displacement value D and time interval ΔT between the two joints. avg , the formula is as follows: The D value is obtained based on historical location distance data.

8. A defect detection and positioning device based on X-ray images, characterized in that: The device comprises: A detection module, used for acquiring an X-ray image of the conveyor belt through an X-ray scanning camera, and training a conveyor belt joint detection model and target detection based on the X-ray image; the joint detection model is used for identifying the conveyor belt joint in the X-ray image; a determination module, for determining the number of a target joint and the position coordinates on the conveyor belt according to the joint image in response to identifying the conveyor belt joint in the X-ray image; The positioning module is used to locate the defect coordinates on the conveyor belt where the defect is located according to the position coordinates of the target joint and the time interval between the target defect and the target joint in response to detecting the target defect on the conveyor belt.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the X-ray image-based defect detection and positioning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the X-ray image-based defect detection and positioning method as described in any one of claims 1 to 7.

Images