Conveyor belt skeleton joint defect detection method, device and equipment and storage medium

Abstract

The invention discloses a conveyor belt skeleton joint defect detection method, device and equipment and a storage medium, and relates to the field of image detection, an X-ray image of a conveyor belt is acquired, and a conveyor belt skeleton joint and conveyor belt defects are identified; when a skeleton joint is identified, extracting a joint image, and detecting an RFID tag of a joint area where the joint is located; when the conveyor belt defect is identified, outputting a defect label of the conveyor belt defect, and performing position and label matching on a defect area where the conveyor belt defect is located and a joint area; and outputting a defect detection result of the skeleton joint according to the position matching result and the label matching result. The joint and defect positions are intelligently detected and recognized through the X-ray image, various defect problems inside and on the surface of the belt body can be analyzed, and the recognition accuracy of joint target detection is improved. RFID information is fused to adaptively number joints, joint defects and non-joint defects are distinguished, the accuracy of a joint detection report is improved in an auxiliary mode, and conveying belt monitoring data are provided for field management personnel in real time.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image detection, and in particular to a conveyor belt skeleton joint defect detection method, device, equipment and storage medium. Background Art

[0002] As a key equipment for material transportation, conveyor belts often break at the joints due to long-term high-load operation. Therefore, conveyor belt joint identification and belt core defect detection at the joints are of great significance to ensure production safety and improve production efficiency in bulk material transportation.

[0003] In the related art, the identification of conveyor belt joints is mainly achieved by the glue marks on the belt surface or by the identification of RFID chips embedded in the belt. Many on-site conveyor belts do not have RFID chips pre-embedded or unconditionally embedded, which makes it difficult to effectively identify the location of the conveyor belt joints. On the other hand, the working conditions of on-site transportation of bulk materials are harsh, and materials or dust often adhere to the belts, making it difficult for ordinary CMOS cameras to identify glue marks on the belt surface, especially small targets such as rivets and steel needles stuck in the belt at the joints. Traditional image recognition cannot detect them, and it is also impossible to monitor surface and internal defects at the joints in real time. Summary of the invention

[0004] The embodiments of the present application provide a conveyor belt skeleton joint defect detection method, device, equipment and storage medium to solve the problem of being unable to accurately identify conveyor belt skeleton joints, joint surface and internal defects.

[0005] On the one hand, the present application provides a conveyor belt skeleton joint defect detection method, the method comprising:

[0006] Acquire an X-ray image of the conveyor belt in real time through an X-ray scanning device, and detect and identify conveyor belt skeleton joints and conveyor belt defects in the X-ray image;

[0007] In response to identifying a skeleton joint, extracting a joint image from the X-ray image, and detecting an RFID tag in a joint area where the skeleton joint is located; the RFID tag records skeleton joint information;

[0008] In response to identifying a conveyor belt defect, a defect label of the conveyor belt defect is output, the defect area where the conveyor belt defect is located is positionally and label matched with the joint area, and a defect detection result of the skeleton joint is output based on the position matching result and the label matching result.

[0009] Specifically, the detecting and identifying conveyor belt skeleton joints and conveyor belt defects in the X-ray image includes:

[0010] Sending the collected X-ray image to the joint detection model to perform conveyor belt identification and joint identification, and outputting the confidence information of the conveyor belt type and the joint;

[0011] Confirm the confidence information of the joint through the CV algorithm and output the joint information;

[0012] The collected X-ray image is sent to a defect detection model, and defect detection is performed according to the confirmed conveyor belt type; the defects include at least one of rivets, tears, foreign matter and wear.

[0013] Specifically, the joint detection model is a yolo detection model, including a first backbone network, a first neck structure and a first head structure;

[0014] The X-ray image input is 640*640 in size, and the feature map is extracted through the first backbone, wherein the convolution step size of the convolution Conv module is 2, downsampling is performed to extract features, and the spatial pyramid pooling SPPF module is used for upsampling to fuse multi-scale features; the feature map output by the first backbone is sent to the first neck structure, and shallow features and deep semantic features are fused through the feature pyramid FPN structure; the output of the first neck structure is connected to the first head structure, and the recognition result is output through convolution;

[0015] The loss function L of the joint detection model is expressed as follows:

[0016] L=1.0*lob j +0.05*lbo x +0.0*lcl s

[0017] Among them, lob j is the target confidence loss; lbo x is the prediction box loss; lcl s is the category judgment loss.

[0018] Specifically, conveyor belt types are divided into aramid conveyor belts and steel wire conveyor belts;

[0019] The method of confirming the confidence information of the joint by the CV algorithm and outputting the joint information includes:

[0020] When the output detection result is an aramid conveyor belt, the conveyor belt image is binarized and eroded to extract foreground and background elements;

[0021] Performing connectivity analysis and connected domain filtering on the binary image to obtain a filtered morphological image;

[0022] When the morphological image matches the joint shape of the aramid conveyor belt joint, outputting corresponding joint information and a joint image;

[0023] When the output detection result is a steel conveyor belt, the conveyor belt image is binarized to identify the steel wire rope features;

[0024] The image is divided into small area graphics, and the number of wire ropes in different areas is calculated according to the pixel density and the joint position and non-joint position are determined;

[0025] When the number of steel wire ropes at the joint and non-joint meets the proportional relationship, the corresponding joint information and joint image are output.

[0026] Specifically, the step of detecting the RFID tag in the joint area where the skeleton joint is located includes:

[0027] In response to detecting an RFID tag in the joint area, matching the RFID tag with a historical RFID tag in a linked list; the linked list stores RFID tag information of all identified joint nodes and associated historical joint images;

[0028] In response to matching the same RFID tag information, the joint image is updated to the joint node of the link list.

[0029] Specifically, in response to not detecting an RFID tag in the joint area, or not matching the same RFID tag information, feature matching is performed between the joint image and the historical joint images of each joint node in the linked list;

[0030] In response to matching a corresponding historical joint image, replacing the historical joint image of the corresponding joint node with the joint image;

[0031] In response to not matching a corresponding historical joint image, a new joint node is inserted into the linked list based on the detected RFID tag and / or the joint image.

[0032] Specifically, the outputting of the defect detection result of the skeleton joint according to the position matching result and the label matching result includes:

[0033] In response to the defect area being located inside the joint area, acquiring a joint defect label library of the joint area according to the conveyor belt type, and matching the library with the defect label of the conveyor belt defect;

[0034] In response to matching the same defect label, the defect position and defect information at the joint node are output;

[0035] In response to the defect area being located outside the joint area, acquiring a defect label library of a non-joint area according to the type of the conveyor belt, and matching the defect label with the defect of the conveyor belt;

[0036] In response to matching the same defect label, the defect position and defect information at the non-joint are output.

[0037] Specifically, the X-ray image is input to a defect detection model for detection, and the defect detection model includes a second backbone network, a second neck structure, and a decoupling head; the second neck includes four-way feature upper fusion and corresponding feature lower fusion branches, and the decoupling head is located at and includes four shallow feature extraction units, and the four shallow feature extraction units are respectively connected to the outputs of the four-way feature lower fusion branches;

[0038] The X-ray image input is 1280*1280 in size, and the feature map is extracted through the second backbone. At least one hole convolution is added to the first C3 layer structure in the second backbone to extract the small target defect features in the image.

[0039] On the other hand, the present application provides a conveyor belt skeleton joint defect detection device, the device comprising:

[0040] A first detection module is used to obtain an X-ray image of the conveyor belt in real time through an X-ray scanning device, and detect and identify conveyor belt skeleton joints and conveyor belt defects in the X-ray image;

[0041] A second detection module is used to extract a joint image from the X-ray image in response to identifying a skeleton joint, and detect an RFID tag in a joint area where the skeleton joint is located; the RFID tag has skeleton joint information recorded therein;

[0042] A matching output module is used to output a defect label of the conveyor belt defect in response to identifying a conveyor belt defect, match the position and label of the defect area where the conveyor belt defect is located with the joint area, and output the defect detection result of the skeleton joint according to the position matching result and the label matching result.

[0043] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the conveyor belt skeleton joint defect detection method described in the above aspects.

[0044] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the conveyor belt skeleton joint defect detection method described in the above aspects.

[0045] The beneficial effects of the technical solution provided by the embodiment of the present application include at least: through X-ray image recognition, intelligent detection and identification of joints and defect locations, various defects inside and on the surface of the belt can be analyzed, and the recognition accuracy of joint target detection can be improved. Image recognition is integrated with RFID information, and joints are numbered adaptively, as well as the identification and location of various defects, namely joint defects and non-joint defects, to assist in improving the accuracy of joint detection reports and provide comprehensive and detailed conveyor belt health monitoring result data to on-site managers in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of a conveyor belt skeleton joint defect detection method provided in an embodiment of the present application;

[0047] Figure 2 The figure shows the effect of the joint of aramid belt and wire rope belt in X-ray image;

[0048] Figure 3 is a flow chart of an algorithm for identifying joint targets using a joint detection model;

[0049] Figure 4 This is the neural network structure diagram of the joint detection model designed based on the yolo v5 detection model;

[0050] Figure 5 This is an example of the detection results of the joint detection model training;

[0051] Figure 6 Schematic diagram of image processing using CV algorithm to confirm aramid belt joints and wire rope belt joints;

[0052] Figure 7 It is a flow chart of the algorithm for numbering the detected joint images in combination with the linked list;

[0053] Figure 8 It is a schematic diagram of defect identification based on piercing rivets and fracture defects of aramid tape;

[0054] Fig. 9 It is a flow chart of the algorithm for joint numbering through RFID tags;

[0055] Fig.10 It is a flow chart of the algorithm for numbering the detected joint images in combination with the linked list;

[0056] Fig.11 It is a structural block diagram of a conveyor belt skeleton joint defect detection device provided in an embodiment of the present application;

[0057] Fig.12 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0059] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0060] Figure 1 : is a flow chart of a conveyor belt skeleton joint defect detection method provided in an embodiment of the present application, comprising the following steps:

[0061] S1, obtaining an X-ray image of the conveyor belt in real time through an X-ray scanning device, detecting and identifying conveyor belt skeleton joints and conveyor belt defects in the X-ray image;

[0062] This application is specifically used for defect detection of conveyor belt skeleton joints. Because joints are more prone to wear and breakage than non-joints, this application focuses on monitoring belt defects at joints. Traditional CMOS cameras or color line scan cameras can only detect obvious large target defects, such as epidermal shedding, adhesion of foreign matter, etc., and cannot detect tears and small target foreign matter, especially internal tears, breaks and rivet punctures, which can easily cause belt breakage. This application abandons the use of traditional cameras and uses X-ray scanning equipment to obtain X-ray images of conveyor belts in real time. Subsequent detection and identification of skeleton joints and defects are based on X-ray images. X-ray images will not affect the recognition results and accuracy due to coverings such as surface glue marks and dust, especially for steel rope conveyor belts, whose internal steel wire structure can be clearly presented in the X-ray image, and other belt bodies such as aramid belts will also present different image qualities due to thickness and material, and piercing rivets can also be identified.

[0063] In a possible implementation, different detection models can be designed and trained for detection and identification, that is, the X-ray image is input into the joint detection model to identify the joint (image), and the X-ray image is input into the defect detection model to identify the conveyor belt defect. In particular, the defect detection model should have the ability to identify defects at joints and defects at non-joints.

[0064] S2, in response to identifying the skeleton joint, extracting the joint image from the X-ray image, and detecting the RFID tag in the joint area where the skeleton joint is located;

[0065] Because of the penetrating effect of X-ray images on conveyor belts, it can be adapted to various types of conveyor belt inspections. This application uses typical aramid belts and steel cord belts as examples for illustration. Figure 2 The figure shows the effect of the joint of the aramid belt and the wire rope belt in the X-ray image. The joint of the upper aramid belt has an obvious boundary outline in the image, and the joint of the lower wire rope belt has obvious wire rope crossing. Theoretically, the number of wire ropes at the intersection is twice the number at the non-joint. As long as the corresponding joint detection model is designed and trained, the corresponding conveyor belt skeleton joint can be identified.

[0066] Because the skeleton joints are confirmed by image recognition, and the conveyor belt may misidentify the joint numbers (different numbers represent different positions) when there are a large number of joints, this application embeds the RFID chip in the pre-buried or edge position of the conveyor belt joint, and writes the joint position and numbering information. The equipment should also install a detection device at the corresponding position to allow the conveyor belt transmission process to continuously detect the chip information to assist in confirmation. Therefore, after detecting the joint and extracting the joint image, it is immediately detected whether there is an RFID tag in the joint ROI area where the joint is located. When an RFID tag is detected in the joint ROI area, the joint information is further determined, and the joint information can be used to locate defects.

[0067] S3, in response to identifying a conveyor belt defect, outputting a defect label of the conveyor belt defect, matching the position and label of the defect area where the conveyor belt defect is located with the joint area, and outputting the defect detection result of the skeleton joint according to the position matching result and the label matching result.

[0068] Conveyor belt defect detection is carried out simultaneously with RFID tag detection and joint detection. After the defect is identified, the defect label, that is, the defect type, is determined, and the position is matched according to the defect ROI area. The purpose of matching the inclusion relationship between the defect ROI area and the joint ROI area is to determine whether the defect is located on the joint or outside the joint. For some special defect types, when they appear in non-joint areas, they may not affect the operation of the system. When a defect appears at the joint position, a timely warning response is required to prevent the joint from breaking or being damaged.

[0069] In summary, this paper uses X-ray image recognition and intelligent detection to identify joints and defect locations, analyze various defects inside and on the surface of the belt, and improve the recognition accuracy of joint target detection. Image recognition integrates RFID information, adaptively numbers joints, and identifies and distinguishes various defects, namely joint defects and non-joint defects, to help improve the accuracy of joint detection reports and provide comprehensive and detailed conveyor belt health monitoring results data to on-site managers in real time.

[0070] In some embodiments, the detection of joints and identification of defects of conveyor belts can be implemented through the following steps:

[0071] A. Send the collected X-ray image to the joint detection model for conveyor belt identification and joint identification, and output the confidence information of the conveyor belt type and joint;

[0072] Figure 3 This is an algorithm flow chart for using the joint detection model to identify joint targets. For the identification of joints, in order to improve the accuracy of joint target detection, this algorithm combines AI target detection and traditional CV algorithm to design the joint detection model. AI target detection uses the standard yolo v5 or yolo v8 model. The addition of the CV confirmation algorithm can greatly reduce the misjudgment of AI recognition. When training AI, it only needs to tend to improve the recall rate. AI target detection will output the confidence information of the conveyor belt type and the joint. When the confidence meets the set threshold, the traditional CV algorithm will be used for confirmation, otherwise the no joint information will be directly output. When the traditional CV algorithm is confirmed, it will perform a separate analysis based on the conveyor belt type, and output the joint information after confirming it as a joint, otherwise it will also output no joint information.

[0073] For the design of the joint detection model, this application uses the yolov5 detection model design, see Figure 4 The network model also includes the first backbone network, the first neck structure and the first head structure. The X-ray image input is 640*640 in size, and the feature map is extracted through the first backbone. The convolution step size of the convolution Conv module is 2, and downsampling is performed to extract features. The spatial pyramid pooling SPPF module performs upsampling and fuses multi-scale features. The feature map output by the first backbone is sent to the first neck structure, and the shallow features and deep semantic features are fused through the feature pyramid FPN structure. The output of the first neck structure is connected to the first head structure, and the recognition result is output through convolution.

[0074] CBL is a convolution module, which consists of a convolution layer Conv, a normalization layer Conv and an activation function Leaky reru. C3 consists of three CBL modules, N Bottleneck modules and a connection layer Concat. SPPF is spatial pyramid pooling, which includes two CBLs, Conact, and three pooling layers MaxPool. The three MaxPools perform serial calculations. The first MaxPool represents a larger scale. The second MaxPool further performs pooling on the basis of the first MaxPool, so the resulting scale will be further reduced. The same is true for the third one. Spatial pyramid pooling can convert feature maps with non-fixed scales into a unified scale. Its main function in yolov5 is to fuse multi-scale features and concat the feature representations of the same feature map at different scales together.

[0075] In order to improve the recall rate of the previously mentioned AI model as much as possible, that is, to reduce the probability of missing joint target detection, the following adjustments were made to the loss function.

[0076] (1) Only the joint target is detected, which belongs to single target detection;

[0077] (2) Increase the target weight;

[0078] Yolov5 default loss weight is as follows:

[0079] L′=0.7*lob j +0.05*lbo x +0.3*lcl s

[0080] The loss function L constructed after the modification of this application is expressed as follows:

[0081] L=1.0*lob j +0.05*lbo x +0.0*lcl s

[0082] Among them, lob j is the target confidence loss; lbo x is the prediction box loss; lcl s is the category judgment loss.

[0083] In the above formula, lob j Weight increase, lbo x The weight remains unchanged, lcl s Only the joint target is detected, so it is cancelled. The modified L training strategy is simulated and tested using 2000 images. Figure 5The following is an example of the detection result. The left side is the original image of the wire rope joint, and the right side is a randomly cropped spliced ​​image. Check the recall rate when the confidence threshold is 0.8. The results are shown in Table 1.

[0084] Table 1 Joint simulation test results

[0085] The recall rate of the test at L′ The recall rate of the test at time L Original image test 100％ 100％ Random cropping and splicing test 97.2％ 98.7％

[0086] Combination Figure 5 As can be seen from Table 1, the recall rates of the two models in the original image test are both 100%. After randomly cropping and splicing the images, the corresponding recall rates are higher.

[0087] B. Confirm the confidence information of the joint through the CV algorithm and output the joint information;

[0088] The CV algorithm here is divided into two aspects. One is for aramid tape, and the other is for the recognition of wire rope tape. The process of recognition, confirmation and output of joint information can be summarized as follows:

[0089] 1) When the output detection result is an aramid conveyor belt, the conveyor belt image is binarized and eroded to extract foreground and background elements;

[0090] 2) Performing connectivity analysis and connected domain filtering on the binary image to obtain a filtered morphological image;

[0091] In the above, the erosion operation is mainly to reduce the foreground pixels in the image. You can set a structural element on the image and slide it to traverse the entire image. Only when the structural element is completely covered by the foreground pixels, the central pixel is retained as a foreground pixel, otherwise it will become the background. The result is that the edge pixels are "eaten", thereby reducing the foreground object. For the image after the erosion operation, you can continue to use the Canny algorithm for edge extraction.

[0092] Connectivity analysis and connected domain filtering are mainly used to filter out "false edges" in the joint borders. Specifically, connected components can be set. The calculation of connected components will assign a unique label to each connected domain. Each pixel point of the image contour will get a label of which connected domain it belongs to. In this way, the length of each connected domain can be obtained. Then, according to the set connected domain scale value, caliper filtering is performed to obtain a clear morphological image.

[0093] 3) When the morphological image matches the joint shape of the aramid conveyor belt joint, the corresponding joint information and joint image are output;

[0094] Figure 6The upper figure is a schematic diagram of image processing for confirming the joint of aramid tape. The left side is the original extracted binary joint image, and the right side is the contour image after morphological processing. Subsequently, only contour comparison is needed to determine the joint information and output the joint image.

[0095] 4) When the output detection result is a steel wire conveyor belt, the conveyor belt image is binarized to identify the steel wire rope features therein;

[0096] 5) Divide the image into small area graphics, calculate the number of wire ropes in different areas and determine the joint position and non-joint position according to the pixel density;

[0097] 6) When the number of steel wire ropes at the joint and non-joint meets the proportional relationship, the corresponding joint information and joint image are output.

[0098] Wire rope joint identification is mainly determined by the ratio of the number of wire ropes at the joint and the number of wire ropes at non-joints. The number of wire ropes at the joint can be counted using a binary image. The wire ropes at the distance of this application should satisfy a quantity relationship of approximately 1.5 times based on the joint manufacturing process. When the relationship is satisfied, the joint information and joint image can be determined.

[0099] The above examples are edge contour recognition and pixel density recognition. The above solutions can also be used to detect other types of conveyor belts.

[0100] C. Send the collected X-ray images to the defect detection model and perform defect detection based on the confirmed conveyor belt type.

[0101] Regardless of whether the joint is confirmed or not, the X-ray image will be sent to the defect detection model for defect detection. The defects in this application include glue marks, scratches, rivets, tears, foreign matter and wear, etc. Compared with the target recognition of conveyor belt joints, the difficulty of defect recognition is greatly increased, and the standard AI model cannot meet the application requirements well. The difficulty in identifying defects in the skeleton belt core of conveyor belt joints is the recognition of small targets, such as defects caused by rivets embedded in the belt joints of aramid conveyor belts; broken wires and loose wires at the joints of wire rope conveyor belts, etc. The essential reason for the difficulty in identifying small targets is that the feature map (feature layer) of commonly used models covers fewer feature points of small targets, resulting in less knowledge that the model can learn. Starting from the original intention of identifying small target problems and improving model accuracy, this application is based on the following design points based on the yolo v5 model:

[0102] (1) The AI ​​model input resolution is increased to 1280*1280 (compared to 640*640), improving the resolution of small objects;

[0103] (2) Add shallow feature map (160*160) to extract features, so that small target prediction has more detailed features;

[0104] (3) Modify the C3 structure to strengthen the context information of small target prediction, that is, make full use of the relationship between small targets and background to improve the prediction accuracy of small targets;

[0105] (4) Add a decoupling head.

[0106] Figure 7 This is a schematic diagram of the network structure of the defect detection model designed based on yolo v5. The X-ray image is input into the defect detection model for detection. The defect detection model includes the second backbone network, the second neck structure, and the decoupling head. The second neck contains four-way feature upper fusion and corresponding feature lower fusion branches. The decoupling head is located in the four shallow feature extraction units, and the four shallow feature extraction units are respectively connected to the output of the four-way feature lower fusion branch.

[0107] The X-ray image input is 1280*1280 in size, and the feature map is extracted through the second backbone. At least one dilated convolution is added to the first C3 layer structure in the second backbone, that is, Figure 7 C3_new in , which uses more hole convolutions to strengthen the contextual connection of small objects to extract small object defect features in X-ray images.

[0108] The introduction of the decoupling head can decompose the target detection task into two subtasks: classification and regression. The classification task is used to predict the category of the target, while the regression task is used to predict the location of the target. This design approach enables the decoupling head to process the target objects in the image more accurately by decomposing the detection task into more specific subtasks, thereby improving the accuracy of detection. The design of the decoupling head reduces the computational burden because it allows the model to focus more on processing specific tasks instead of processing all information at once, thereby improving the efficiency of the model. By decoupling the detection head, the model exhibits better generalization ability when processing similar tasks and can better adapt to different data sets and scenarios. The four shallow feature extraction units can process feature map outputs of up to 160*160, which has greater advantages for small target defect training and detection.

[0109] Figure 8 This is a schematic diagram of defect recognition based on piercing rivets and fracture defects of aramid tape. The left side is the recognition result of embedded rivets in aramid tape, and the right side is the defect result of tear detection. The above designed model was used to simulate 2000 images, and the test accuracy reached 99.6%. Compared with the yolo v5 model with an input of 640*640, the test accuracy was only 96.8%, and the accuracy loss was mainly in the detection of small targets.

[0110] In some embodiments, because the conveyor belt is long and contains many joints, and the joints are usually industrially made, it is difficult to distinguish the subtle appearance, which is not conducive to number identification and positioning, and there may be missed detection. Considering the high reliability of RFID, a double verification mechanism is designed to use database matching and linked lists to perform numbering and check whether there is any missed report.

[0111] Fig. 9 This is the algorithm flow chart for joint numbering through RFID tags. The overall strategy is to assign numbers to each identified image according to the RFID chip at the initial stage of each cycle detection cycle, and establish a linked list based on the joint image and RFID information. The linked list data is continuously improved with the detection, and the joint image of the joint node is also constantly updated. The currently detected joint information is matched with the historical joint information (RFID tag matching or image matching). If the match is successful, the historically compiled joint number is output and the historical joint node information is updated. If the match is not successful, it is considered to be a new joint node that is recognized for the first time. The node is inserted into the linked list. The new node stores the joint image, RFID tag (empty if not recognized) and new number and other information. The specific steps are shown as follows:

[0112] A. In response to not detecting an RFID tag in the joint area, or not matching the same RFID tag information, feature matching the joint image with the historical joint images of each joint node in the linked list;

[0113] B. in response to matching a corresponding historical joint image, replacing the historical joint image of the corresponding joint node with the joint image;

[0114] C. In response to not matching a corresponding historical joint image, insert a new joint node into the linked list based on the detected RFID tag and / or joint image.

[0115] The situation of inserting new nodes here is mainly for those situations that were missed in the early stage, especially when the joint was not detected by X-ray images but the RFID chip was detected. At this time, the inserted joint node does not contain images, but only contains RFID information. The joint image can be determined and added after matching the historical joint images in the database during the later recognition. This behavior of inserting a linked list joint node is a missed detection situation.

[0116] As mentioned above, there are many defects in joints and non-joints, and only defects in specific scenarios need to trigger early warnings or manual intervention. For example, wear in non-joint areas is the result of long-term use, and it does not need to be replaced if it has not reached a certain degree of wear. However, in the case of wear or puncture of rivets at the joints, timely early warning and processing are required. Therefore, this application also provides a processing mechanism to distinguish between defects at joints and defects at non-joints.

[0117] Fig.10 This is a flowchart of the algorithm for numbering the detected joint images in combination with a linked list. The number matching process can be summarized as follows:

[0118] A. In response to the defect area being located inside the joint area, a joint defect label library of the joint area is obtained according to the conveyor belt type, and matched with the defect label of the conveyor belt defect;

[0119] B. In response to matching the same defect label, output the defect location and defect information at the joint node;

[0120] C. In response to the defect area being outside the joint area, a defect label library of the non-joint area is obtained according to the conveyor belt type, and matched with the defect label of the conveyor belt defect;

[0121] D. In response to matching the same defect label, the defect location and defect information at the non-joint are output.

[0122] This algorithm process lacks the defect type and location information output by the aforementioned defect detection model, and then combines it with the joint location information output by the joint detection model to distinguish whether the defect is a joint defect or a non-joint defect. Finally, it outputs the detection results, and warns and generates defect detection reports according to needs.

[0123] In summary, compared with the detection algorithms in related technologies, this application has the following advantages:

[0124] 1. At present, the recognition of conveyor belt joints mainly relies on the glue marks on the belt surface, or is realized based on the recognition of RFID chips embedded in the belt. Compared with the existing technology, this solution can improve the recognition accuracy of joint target detection;

[0125] 2. Based on the combination of AI target model and traditional CV analysis mechanism and the integration of RFID information, the identity of the conveyor belt joint can be more accurately identified. At the same time, the accuracy of identifying small target defects in the joint can be improved by optimizing the AI ​​model design;

[0126] 3. It can provide defect location data for the monitoring system to help improve the accuracy of joint inspection reports;

[0127] Fig.11 The structural block diagram of the conveyor belt skeleton joint defect detection device provided in an embodiment of the present application is shown, and the device comprises:

[0128] The first detection module 1110 is used to obtain an X-ray image of the conveyor belt in real time through an X-ray scanning device, and detect and identify conveyor belt skeleton joints and conveyor belt defects in the X-ray image;

[0129] The second detection module 1120 is used to extract a joint image from the X-ray image in response to identifying a skeleton joint, and detect an RFID tag in a joint area where the skeleton joint is located; the RFID tag has skeleton joint information recorded therein;

[0130] The matching output module 1130 is used to output the defect label of the conveyor belt defect in response to identifying the conveyor belt defect, match the defect area where the conveyor belt defect is located with the joint area in terms of position and label, and output the defect detection result of the skeleton joint according to the position matching result and the label matching result.

[0131] The conveyor belt skeleton joint defect detection device provided in the embodiment of the present application can be applied to the conveyor belt skeleton joint defect detection method provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0132] It should be noted that the conveyor belt skeleton joint defect detection device provided in the embodiment of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the conveyor belt skeleton joint defect detection device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the conveyor belt skeleton joint defect detection method provided in the above method embodiment and the implementation method of the conveyor belt skeleton joint defect detection device provided in this embodiment belong to the same concept. The specific implementation process of the conveyor belt skeleton joint defect detection device provided in this embodiment is detailed in the above method embodiment, which will not be repeated here.

[0133] Fig.12The block diagram of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory may be connected by a bus or in other ways. Among them, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0134] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0135] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0136] In some embodiments, the computer device may further optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit, a display screen and a keyboard.

[0137] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0138] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen is a touch screen, the display screen also has the ability to collect touch signals on the surface or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in other embodiments, the display screen can be at least two, respectively arranged on different surfaces of the computer device or in a folding design; in other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0139] The power supply is used to power various components in the computer device. The power supply can be AC, DC, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0140] Those skilled in the art will appreciate that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0141] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0142] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A conveyor belt skeleton joint defect detection method, characterized in that: The method comprises: Acquire an X-ray image of the conveyor belt in real time through an X-ray scanning device, and detect and identify conveyor belt skeleton joints and conveyor belt defects in the X-ray image; In response to identifying a skeleton joint, extracting a joint image from the X-ray image, and detecting an RFID tag in a joint area where the skeleton joint is located; the RFID tag records skeleton joint information; In response to identifying a conveyor belt defect, a defect label of the conveyor belt defect is output, the defect area where the conveyor belt defect is located is positionally and label matched with the joint area, and a defect detection result of the skeleton joint is output based on the position matching result and the label matching result.

2. The method according to claim 1, characterized in that The detecting and identifying conveyor belt skeleton joints and conveyor belt defects in the X-ray image includes: Sending the collected X-ray image to the joint detection model to perform conveyor belt identification and joint identification, and outputting the confidence information of the conveyor belt type and the joint; Confirm the confidence information of the joint through the CV algorithm and output the joint information; The collected X-ray image is sent to a defect detection model, and defect detection is performed according to the confirmed conveyor belt type; the defects include at least one of rivets, tears, foreign matter and wear.

3. The method according to claim 2, characterized in that The joint detection model is a Yolo detection model, including a first backbone network, a first neck structure and a first head structure; The X-ray image input is 640*640 in size, and the feature map is extracted through the first backbone, wherein the convolution step size of the convolution Conv module is 2, downsampling is performed to extract features, and the spatial pyramid pooling SPPF module is used for upsampling to fuse multi-scale features; the feature map output by the first backbone is sent to the first neck structure, and shallow features and deep semantic features are fused through the feature pyramid FPN structure; the output of the first neck structure is connected to the first head structure, and the recognition result is output through convolution; The loss function L of the joint detection model is expressed as follows: L=1.0*lob j +0.05*lbo x +0.0*lcl s Among them, lob j is the target confidence loss; lbo x is the prediction box loss; lcl s is the category judgment loss.

4. The method according to claim 2, characterized in that: Conveyor belt types are divided into aramid conveyor belts and steel wire conveyor belts; The method of confirming the confidence information of the joint by the CV algorithm and outputting the joint information includes: When the output detection result is an aramid conveyor belt, the conveyor belt image is binarized and eroded to extract foreground and background elements; Performing connectivity analysis and connected domain filtering on the binary image to obtain a filtered morphological image; When the morphological image matches the joint shape of the aramid conveyor belt joint, outputting corresponding joint information and a joint image; When the output detection result is a steel conveyor belt, the conveyor belt image is binarized to identify the steel wire rope features; The image is divided into small area graphics, and the number of wire ropes in different areas is calculated according to the pixel density and the joint position and non-joint position are determined; When the number of steel wire ropes at the joint and non-joint meets the proportional relationship, the corresponding joint information and joint image are output.

5. The method according to claim 1, characterized in that The RFID tag for detecting the joint area where the skeleton joint is located includes: In response to detecting an RFID tag in the joint area, matching the RFID tag with a historical RFID tag in a linked list; the linked list stores RFID tag information of all identified joint nodes and associated historical joint images; In response to matching the same RFID tag information, the joint image is updated to the joint node of the link list.

6. The method according to claim 5, characterized in that In response to not detecting an RFID tag in the joint area, or not matching the same RFID tag information, feature matching the joint image with historical joint images of each joint node in the linked list; In response to matching a corresponding historical joint image, replacing the historical joint image of the corresponding joint node with the joint image; In response to not matching a corresponding historical joint image, a new joint node is inserted into the linked list based on the detected RFID tag and / or the joint image.

7. The method according to claim 6, characterized in that The method of outputting the defect detection result of the skeleton joint according to the position matching result and the label matching result includes: In response to the defect area being located inside the joint area, acquiring a joint defect label library of the joint area according to the conveyor belt type, and matching the library with the defect label of the conveyor belt defect; In response to matching the same defect label, the defect position and defect information at the joint node are output; In response to the defect area being located outside the joint area, acquiring a defect label library of a non-joint area according to the type of the conveyor belt, and matching the defect label with the defect of the conveyor belt; In response to matching the same defect label, the defect position and defect information at the non-joint are output.

8. The method according to claim 3, characterized in that The X-ray image is input to the defect detection model for detection, and the defect detection model includes a second backbone network, a second neck structure, and a decoupling head; the second neck includes four-way feature upper fusion and corresponding feature lower fusion branches, and the decoupling head is located at and includes four shallow feature extraction units, and the four shallow feature extraction units are respectively connected to the outputs of the four-way feature lower fusion branches; The X-ray image input is 1280*1280 in size, and the feature map is extracted through the second backbone. At least one hole convolution is added to the first C3 layer structure in the second backbone to extract the small target defect features in the image.

9. A conveyor belt skeleton joint defect detection device, characterized in that: The device comprises: A first detection module is used to obtain an X-ray image of the conveyor belt in real time through an X-ray scanning device, and detect and identify conveyor belt skeleton joints and conveyor belt defects in the X-ray image; A second detection module is used to extract a joint image from the X-ray image in response to identifying a skeleton joint, and detect an RFID tag in a joint area where the skeleton joint is located; the RFID tag has skeleton joint information recorded therein; A matching output module is used to output a defect label of the conveyor belt defect in response to identifying a conveyor belt defect, match the position and label of the defect area where the conveyor belt defect is located with the joint area, and output the defect detection result of the skeleton joint according to the position matching result and the label matching result.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the conveyor belt skeleton joint defect detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the conveyor belt skeleton joint defect detection method as described in any one of claims 1 to 8.

Images