Defect detection method and apparatus for a conveyor belt carcass splice, device and storage medium

Abstract

A defect detection method and apparatus for a conveyor belt carcass splice, a device and a storage medium, relating to the field of image detection. The detection method comprises: acquiring an X-ray image of a conveyor belt, and identifying a conveyor belt carcass splice and a conveyor belt defect; upon identification of the carcass splice, extracting a splice image, and detecting an RFID tag of a splice region where the splice is located; upon identification of the conveyor belt defect, outputting a defect tag of the conveyor belt defect, and performing location matching and tag matching on a defect region where the conveyor belt defect is located and the splice region; and on the basis of the location matching result and the tag matching result, outputting a defect detection result of the carcass splice. Splice and defect locations are intelligently determined and identified by means of the X-ray image, such that various inner and surface defect problems of a belt body can be analyzed, thus improving the accuracy of splice object detection. Adaptive splice numbering is achieved via integration of RFID information, and splice defects and non-splice defects are distinguished, thus helping to improve the accuracy of splice test reports, and providing conveyor belt monitoring data to on-site management personnel in real time.

Description

Conveyor belt skeleton joint defect detection method, device, equipment and storage medium TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of image detection, in particular to a conveyor belt skeleton joint defect detection method, device, equipment and storage medium. BACKGROUND

[0002] As a key equipment for material transportation, the conveyor belt often causes belt breakage at the joint due to long-term high-load operation of the conveyor belt. Therefore, conveyor belt joint recognition and joint core defect detection are of great significance to ensure the production safety of bulk material transportation and improve production efficiency. TECHNICAL PROBLEM

[0003] In related technologies, the recognition of the conveyor belt joint is mainly realized by the adhesive mark on the surface of the belt or by the recognition of the RFID chip embedded in the belt. Many on-site conveyor belts do not have pre-embedded or conditional embedded RFID chips, which makes it difficult to effectively identify the position of the conveyor belt joint. On the other hand, the working conditions of on-site bulk material transportation are harsh, and the belt is often attached with materials or dust, which makes it difficult for ordinary CMOS cameras to identify the adhesive mark on the surface of the belt, especially the small targets such as rivets and steel needles inserted into the joint of the belt. Traditional image recognition cannot detect the defects on the surface and inside of the joint, so it is impossible to monitor the defects in real time. TECHNICAL SOLUTION

[0004] Embodiments of the present application provide a conveyor belt skeleton joint defect detection method, device, equipment and storage medium, which solves the problem of inaccurate recognition of the conveyor belt skeleton joint and the defects on the surface and inside of the joint.

[0005] In one aspect, the present application provides a conveyor belt skeleton joint defect detection method, which comprises:

[0006] real-time acquisition of X-ray images of the conveyor belt by an X-ray scanning device, detection and identification of the conveyor belt skeleton joint and conveyor belt defects in the X-ray images;

[0007] In response to the identification of the skeleton joint, extracting a joint image from the X-ray image and detecting an RFID tag in the joint area where the skeleton joint is located; the RFID tag records the information of the skeleton joint;

[0008] In response to the identification of the conveyor belt defect, outputting a defect label of the conveyor belt defect, matching the defect area where the conveyor belt defect is located with the joint area in terms of position and label, and outputting a defect detection result of the skeleton joint according to the position matching result and the label matching result.

[0009] Specifically, the detection and identification of the conveyor belt skeleton joint and the conveyor belt defects in the X-ray images comprise:

[0010] The collected X-ray image is sent to a joint detection model for conveyor belt recognition and joint recognition, and the confidence information of the conveyor belt type and the joint is output;

[0011] The confidence information of the joint is confirmed by a CV algorithm, and the joint information is output;

[0012] The collected X-ray image is sent to a defect detection model, and defect detection is performed according to the confirmed conveyor belt type; the defect includes at least one of a rivet, a tear, a foreign object and wear.

[0013] Specifically, the joint detection model is a yolo detection model, including a first backbone network, a first neck structure and a first head structure;

[0014] The X-ray image input is 640*640 size, and the feature map is extracted by the first backbone, wherein the convolution Conv module has a convolution step of 2 for feature extraction by downsampling, and the spatial pyramid pooling SPPF module is used for upsampling and fusion of multi-scale features; The feature map output by the first backbone is sent to the first neck structure, and the shallow feature and deep semantic feature are fused through the feature pyramid FPN structure; The first neck structure output is connected to the first head structure, and the recognition result is output through convolution;

[0015] The loss function L of the joint detection model is represented as follows: L=1.0*lob j +0.05*lbo x +0.0*lcl s

[0016] Wherein, lob j is the target confidence loss; lbo x is the prediction box loss; lcl s is the class judgment loss.

[0017] Specifically, the conveyor belt type is divided into aramid conveyor belt and steel wire conveyor belt;

[0018] The confidence information of the joint is confirmed by the CV algorithm, and the joint information is output, including:

[0019] When the output detection result is an aramid conveyor belt, the conveyor belt image is subjected to binaryzation processing and erosion operation to extract foreground and background elements;

[0020] The binaryzation image is subjected to connectivity analysis and connected domain filtering to obtain a filtered morphological image;

[0021] When the morphological image matches the joint shape of the aramid conveyor belt joint, output corresponding joint information and joint image;

[0022] When the output detection result is a steel wire conveyor belt, perform binaryzation processing on the conveyor belt image to identify the steel wire rope features therein;

[0023] Divide the image into small area patterns, and calculate the number of steel wire ropes in different areas and determine the joint position and non-joint position according to the pixel density;

[0024] When the number of steel wire ropes at the joint and non-joint positions satisfies the proportional relationship, output corresponding joint information and joint image.

[0025] Specifically, the method comprises:

[0026] In response to detecting an RFID tag in the joint area, match the RFID tag with historical RFID tags in a linked list; the linked list stores RFID tag information and associated historical joint images of all identified joint nodes;

[0027] In response to matching the same RFID tag information, update the joint image to a joint node in the linked list.

[0028] Specifically, in response to not detecting an RFID tag in the joint area or not matching the same RFID tag information, perform feature matching on the joint image and historical joint images of each joint node in the linked list;

[0029] In response to matching the corresponding historical joint image, replace the historical joint image of the corresponding joint node with the joint image;

[0030] In response to not matching the corresponding historical joint image, insert a new joint node in the linked list based on the detected RFID tag and / or joint image.

[0031] Specifically, the method comprises:

[0032] In response to the defect area being located inside the joint area, obtain a joint defect tag library of the joint area according to the conveyor belt type, and match the defect tag of the conveyor belt defect;

[0033] In response to matching the same defect tag, output the defect position and defect information at the joint node;

[0034] In response to the defect region being located outside the joint region, a defect label library of a non-joint region is obtained according to a conveyor belt type, and matched with a defect label of the conveyor belt defect;

[0035] In response to matching the same defect label, a defect position and defect information at the non-joint are output.

[0036] Specifically, the X-ray image is input to a defect detection model for detection, the defect detection model comprising a second backbone network, a second neck structure, and a decoupling head; the second neck comprises four feature upper fusion branches and corresponding feature lower fusion branches, and the decoupling head comprises four shallow layer feature extraction units connected to outputs of the four feature lower fusion branches;

[0037] The input X-ray image has a size of 1280*1280, and a feature map is extracted by the second backbone; at least one hole convolution is added in a first C3 layer structure in the second backbone to extract small target defect features in the image.

[0038] In another aspect, the present application provides a conveyor belt skeleton joint defect detection device, the device comprising:

[0039] A first detection module is configured to acquire an X-ray image of a conveyor belt in real time by an X-ray scanning device, and detect and identify a conveyor belt skeleton joint and a conveyor belt defect in the X-ray image;

[0040] A second detection module is configured to extract a joint image from the X-ray image in response to identifying the skeleton joint, and detect an RFID tag in a joint region where the skeleton joint is located; the RFID tag records skeleton joint information;

[0041] A matching output module is configured to output a defect label of the conveyor belt defect in response to identifying the conveyor belt defect, match a defect region where the conveyor belt defect is located with the joint region in terms of position and label, and output a defect detection result of the skeleton joint according to a position matching result and a label matching result.

[0042] In yet another aspect, the present application provides a computer device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the conveyor belt skeleton joint defect detection method of the above-mentioned aspects.

[0043] In yet another aspect, the present application provides a computer readable storage medium, wherein the readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by a processor to implement the conveyor belt skeleton joint defect detection method in the above aspect. Advantages

[0044] The technical scheme provided by the embodiments of the present application has at least the following advantages: joint and defect position can be intelligently detected and identified through X-ray image recognition, various defect problems inside and on the belt body can be analyzed, and the identification accuracy of joint target detection is improved. The image recognition is combined with RFID information, the joints are adaptively numbered, and various defects are identified and distinguished in position, i.e., joint defects and non-joint defects, which helps to improve the accuracy of joint detection report and provides comprehensive and detailed conveyor belt health monitoring result data for on-site management personnel in real time. BRIEF DESCRIPTION OF DRAWINGS

[0045] FIG. 1 is a flowchart of the conveyor belt skeleton joint defect detection method provided by the embodiments of the present application;

[0046] FIG. 2 shows the presentation effect of aramid belt and steel wire rope joint in an X-ray image;

[0047] FIG. 3 is an algorithm flowchart for identifying joint targets using a joint detection model;

[0048] FIG. 4 is a neural network structure diagram of a joint detection model designed using a yolo v5 detection model;

[0049] FIG. 5 is an example diagram of detection results of joint detection model training;

[0050] FIG. 6 is an image processing schematic diagram for confirming aramid belt joint and steel wire rope joint using a CV algorithm;

[0051] FIG. 7 is an algorithm flowchart for numbering joint images detected in combination with a linked list;

[0052] FIG. 8 is a schematic diagram of defect identification of aramid belt puncture rivet and fracture defects;

[0053] FIG. 9 is an algorithm flowchart for joint numbering by RFID tag;

[0054] FIG. 10 is an algorithm flowchart for numbering joint images detected in combination with a linked list;

[0055] FIG. 11 is a structural block diagram of the conveyor belt skeleton joint defect detection device provided by the embodiments of the present application;

[0056] FIG. 12 shows a structural block diagram of a computer device provided by an example embodiment of the present application. Embodiments of the present application

[0057] For the purpose of making the present application, technical solutions and advantages more clear, the following will be further described in detail with the help of the accompanying drawings.

[0058] The "multiple" mentioned in the present application refers to two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents that the front and rear associated objects are in an "or" relationship.

[0059] FIG. 1 is a flowchart of a conveyor belt skeleton joint defect detection method provided by an embodiment of the present application, which includes the following steps:

[0060] S1, acquiring an X-ray image of the conveyor belt in real time through an X-ray scanning device, detecting and identifying the conveyor belt skeleton joint and the conveyor belt defect in the X-ray image;

[0061] The present application is specifically used for detecting defects of the conveyor belt skeleton joint. Because the joint is more prone to wear and tear and breakage than the non-joint, the present application focuses on monitoring the belt body defect problems at the joint. The traditional CMOS camera or color line scanning camera can only detect obvious large target defects such as skin peeling, adhesion of foreign matter, etc., and cannot detect tearing and small target foreign matter, especially internal tearing, breakage and rivet puncture defects, which are extremely easy to cause belt body breakage. The present application discards the use of traditional cameras and selects an X-ray scanning device to acquire an X-ray image of the conveyor belt in real time. Subsequently, the skeleton joint and the defect are detected and identified on the basis of the X-ray image. The X-ray image will not be affected by the covering such as surface glue marks and dust, and the recognition result and accuracy will not be affected. In particular, the steel wire structure inside the steel cord conveyor belt can be clearly presented in the X-ray image, and other belt bodies such as aramid belts will also present different image quality due to thickness and material, and the punctured rivet can also be identified.

[0062] In one possible embodiment, different detection models can be designed and trained respectively for detection and identification, that is, the X-ray image is input into the joint detection model to identify the joint (image), and the X-ray image is input into the defect detection model to identify the conveyor belt defect. It is particularly pointed out that the defect detection model should have joint defect identification and non-joint defect identification.

[0063] S2, in response to identifying the skeleton joint, extracting a joint image from the X-ray image, and detecting an RFID tag in a joint area where the skeleton joint is located;

[0064] Because of the penetrating effect of X-ray image on the conveyor belt, it can adapt to various types of conveyor belt detection. This application takes typical aramid belt and steel wire rope belt as an example for illustration. Figure 2 shows the effect of aramid belt and steel wire rope belt joint in X-ray image. The joint of the upper aramid belt has obvious boundary profile in the image, and the joint of the lower steel wire rope belt has obvious steel wire rope intersection state. Theoretically, the number of steel wire ropes at the intersection is twice the number at the non-joint position. As long as the corresponding joint detection model is designed and trained well, the corresponding conveyor belt skeleton joint can be identified.

[0065] Because the skeleton joint is identified by image recognition, and in the case of a large number of joints, the joint number (different numbers represent different positions) may be misidentified. Therefore, this application embeds an RFID chip in the edge position of the conveyor belt joint or the pre-embedded position, and writes the joint position and number information. The device should also install a detection device at the corresponding position to continuously detect the chip information to assist in confirmation. Therefore, after detecting the joint and extracting the joint image, the next step is to detect whether there is an RFID tag in the joint ROI area where the joint is located. When an RFID tag is detected in the joint ROI area, the joint information is further determined, and the joint information can be used for defect positioning.

[0066] S3, in response to identifying the conveyor belt defect, outputting the defect label of the conveyor belt defect, matching the defect area where the conveyor belt defect is located with the joint area in position and label, and outputting the defect detection result of the skeleton joint according to the position matching result and the label matching result.

[0067] The conveyor belt defect detection is synchronized with the RFID tag detection and the joint detection. After identifying the defect, the defect label is determined, i.e. the defect type, and the position matching is performed according to the defect ROI area. The inclusion relationship between the defect ROI area and the joint ROI area is matched, which aims to determine whether the defect is located on the joint or outside the joint. For some special defects that appear in the non-joint area, they may not affect the system operation. However, when the defect appears in the joint position, timely warning and response are needed to prevent the joint from breaking or being damaged.

[0068] In summary, this paper uses X-ray image recognition to intelligently detect and identify the joint and defect position, can analyze various defect problems inside and on the belt body, and improve the recognition accuracy of joint target detection. Image recognition is combined with RFID information, the joint is numbered adaptively, and various defects are recognized and distinguished in position, i.e. joint defect and non-joint defect, which helps to improve the accuracy of joint detection report and provides comprehensive and detailed conveyor belt health monitoring result data for on-site management personnel in real time.

[0069] In some embodiments, for joint detection and defect identification of the conveying belt, the following steps can be specifically implemented:

[0070] A. The collected X-ray image is input into the joint detection model for conveying belt identification and joint identification, and the confidence information of the conveying belt type and the joint is output.

[0071] Figure 3 is a flowchart of an algorithm for identifying joint targets using a joint detection model. To improve the accuracy of joint target detection, the algorithm combines AI target detection and traditional CV algorithms to design the joint detection model. AI target detection uses a standard yolov5 or yolov8 model, and adding a CV confirmation algorithm can significantly reduce the misjudgment of AI recognition. During AI training, only the recall rate needs to be improved. AI target detection outputs the confidence information of the conveying belt type and the joint. When the confidence meets the set threshold, the traditional CV algorithm is used for confirmation, otherwise no joint information is directly output. When the traditional CV algorithm is used for confirmation, the conveying belt type is analyzed separately, and the joint information is output after the joint is confirmed, otherwise no joint information is output.

[0072] For the design of the joint detection model, the present application uses a yolov5 detection model, as shown in Figure 4. The network model includes a first backbone network, a first neck structure, and a first head structure. The input X-ray image is 640*640 in size. The first backbone extracts feature maps, and the convolution Conv module in the convolution has a convolution step of 2 for feature extraction by downsampling. The spatial pyramid pooling SPPF module performs upsampling and fuses multi-scale features. The feature map output by the first backbone is input into the first neck structure, which fuses shallow features and deep semantic features through the feature pyramid FPN structure. The output of the first neck structure is connected to the first head structure, which outputs the recognition result through convolution.

[0073] CBL is a convolution module, which consists of a convolution layer Conv, a normalization layer Conv and an activation function Leaky relu. C3 consists of three CBL modules, N bottleneck modules and a connection layer Concat. SPPF is a spatial pyramid pooling, which includes two CBLs, a Concat, and three pooling layers MaxPool. The three MaxPools are calculated in series, the first MaxPool represents a larger scale, the second MaxPool further pools on the basis of the first MaxPool, so the resulting scale will be further reduced, and the third one is the same. The spatial pyramid pooling can convert the feature map with variable scale to a unified scale. The main role in yolov5 is to fuse multi-scale features, and the features of different scales in the same feature map are concatenated together.

[0074] For the AI model mentioned earlier, the recall rate is as high as possible, that is, the joint target detection loss probability is reduced, and the following adjustments are made on the loss function.

[0075] (1) Only detect joint targets, which belongs to single target detection;

[0076] (2) Increase the target weight;

[0077] The default loss weight of yolov5 is as follows: L' = 0.7 * lob j + 0.05 * lbo x + 0.3 * lcl s

[0078] The loss function L constructed after the modification of the present application is represented as follows: L = 1.0 * lob j + 0.05 * lbo x + 0.0 * lcl s

[0079] Wherein, lob j is the target confidence loss; lbo x is the prediction box loss; lcl s is the class judgment loss.

[0080] In the above formula, the weight of lob j is increased, the weight of lbo x is unchanged, and lcl s is cancelled because only joint single target is detected. The L training strategy after modification is simulated and tested by using 2000 images, and Fig. 5 is an example of detection result, in which the left side is the original image of the steel wire joint, and the right side is a random cropped and spliced image. The recall rate when the confidence threshold is 0.8 is checked, and the results are shown in Table 1.

[0081] Table 1 Joint simulation test results

[0082] As can be seen from FIG. 5 and Table 1, the recall of the two models is 100% in the original test, and the recall is higher after the random clipping and splicing of the image.

[0083] B, the confidence information of the joint is confirmed by the CV algorithm, and the joint information is output;

[0084] The CV algorithm here is divided into two aspects, one is for aramid belt, and the other is for steel wire rope belt identification, and the process of identifying and outputting joint information can be summarized as follows:

[0085] 1) When the output detection result is an aramid conveyor belt, the conveyor belt image is binarized and eroded to extract foreground and background elements;

[0086] 2) The binarized image is analyzed for connectivity and connected domain filtering to obtain a filtered morphological image;

[0087] In the above, the erosion operation mainly wants to reduce the foreground pixels in the image, which can set a structure element on the image and slide it to traverse the entire image. Only when the structure element is completely covered by the foreground pixels, the center pixel is retained as a foreground pixel, otherwise it will become a background. The result is that the edge pixels are "eaten" and the foreground object is reduced. For the image after completing the erosion operation, the Canny algorithm can be used to continue to extract the edge.

[0088] Connectivity analysis and connected domain filtering are mainly used to filter the "false edges" in the joint frame. The connected component can be set. The calculation of the connected component will assign a unique label to each connected domain, and each pixel point of the image contour will get a label of which connected domain it belongs to. In this way, the length of each connected domain can be obtained, and then the connected domain size value is set to filter the gauge, and a clear morphological image is obtained.

[0089] 3) When the morphological image matches the joint shape of the aramid conveyor belt joint, output the corresponding joint information and joint image;

[0090] The upper image in FIG. 6 is an image processing schematic diagram for confirming the aramid belt joint. The left side is the original extracted binarized joint image, and the right image is the contour image after morphological processing. Subsequently, only contour comparison is needed to determine the joint information and output the joint image.

[0091] 4) When the output detection result is a steel wire conveyor belt, the conveyor belt image is binarized to identify the steel wire rope features therein;

[0092] 5) The image is segmented into small area patterns, and the number of steel wires in different areas is calculated according to the pixel density to determine the joint position and the non-joint position;

[0093] 6) When the number of steel wires at the joint and the non-joint position meets the proportional relationship, the corresponding joint information and joint image are output.

[0094] The steel wire joint recognition is mainly determined by the proportion of the number of steel wires at the joint and the non-joint position. The number of steel wires at the joint and the non-joint position can be counted using the binary image. According to the joint manufacturing process, the distance of the steel wire should meet the proportional relationship of about 1.5 times. When the relationship is met, the joint information and joint image can be determined.

[0095] The above examples are based on edge contour recognition and pixel density recognition. Other types of conveyor belts can also be detected using the above scheme.

[0096] C, the collected X-ray image is sent to the defect detection model, and the defect detection is performed according to the confirmed conveyor belt type.

[0097] Whether the joint is confirmed or not, the X-ray image will be sent to the defect detection model for defect detection. The defects in the present application include glue marks, scratches, rivets, tears, foreign matter and wear, etc. Compared with the conveyor belt joint target recognition, the difficulty of defect recognition is greatly improved, and the standard AI model cannot meet the application requirements well. The difficulty of the skeleton core defect recognition of the conveyor belt joint is small target recognition, such as the rivet embedded in the joint of the aramid conveyor belt causing defects; the steel wire breakage and loosening at the joint of the steel wire conveyor belt. The essential reason for small target recognition difficulty is that the feature points of the small target covered by the commonly used model feature map are less, resulting in less knowledge learned by the model. Based on the yolo v5 model design points, the present application is designed from the beginning of small target recognition and improving model accuracy as follows:

[0098] (1) The AI model input resolution is improved to 1280*1280 (relative to 640*640), which improves the small target resolution;

[0099] (2) Increase the shallow layer feature map (160*160) to extract features, so that small target prediction has more detailed features;

[0100] (3) Modify the C3 structure to strengthen the context information of small target prediction, that is, fully utilize the relationship between small targets and background to improve the prediction accuracy of small targets;

[0101] (4) Increase the decoupling head.

[0102] FIG. 7 is a schematic diagram of the network structure of a defect detection model designed based on yolo v5. The X-ray image is input into the defect detection model for detection. The defect detection model includes a second backbone network, a second neck structure, and a decoupling head. The second neck includes four feature fusion branches and corresponding feature fusion branches. The decoupling head is located in four shallow layer feature extraction units, and each shallow layer feature extraction unit is connected to the output of the four feature fusion branches.

[0103] The input X-ray image is 1280*1280 in size. The feature map is extracted by the second backbone. At least one hole convolution is added in the first C3 layer structure in the second backbone, that is, C3_new in FIG. 7. More hole convolutions can be used to strengthen the context connection of small targets to extract small target defect features in the X-ray image.

[0104] The introduction of the decoupling head can divide the target detection task into two sub-tasks: classification and regression. The classification task is used to predict the category of the target, and the regression task is used to predict the position of the target. This design method divides the detection task into more specific sub-tasks, so that the decoupling head can more accurately process the target objects in the image, thereby improving the accuracy of detection. The design of the decoupling head reduces the computational burden, because it allows the model to focus more on processing specific tasks, rather than processing all information at once, thereby improving the efficiency of the model. By decoupling the detection head, the model shows better generalization ability when processing similar tasks, and can better adapt to different data sets and scenarios. The four shallow layer feature extraction units can process feature map outputs of up to 160*160, which has greater advantages for small target defect training and detection.

[0105] FIG. 8 is a schematic diagram of defect identification with aramid belt puncture rivet and fracture defects. The left side is the identification result of the aramid belt embedded rivet, and the right side is the defect result of the tear. Using the above designed model, 2000 images are simulated and tested, and the test accuracy reaches 99.6%. Compared with the yolo v5 model with an input of 640*640, the test accuracy is only 96.8%, and the accuracy loss is mainly in the detection of small targets.

[0106] In some embodiments, because the belt body of the conveyor belt is long and contains many joints, the joints are usually made industrially and are difficult to distinguish in appearance, which is not conducive to numbering identification and positioning, and may also exist in the case of missed detection. Considering the high reliability of RFID, a double verification mechanism is designed to use database matching and linked lists to number and check whether there is a missed report situation.

[0107] Figure 9 is a flow chart of an algorithm for joint numbering by RFID tags, the overall strategy is to detect at each cycle, at the beginning of the stage for each identified image according to the RFID chip allocation number, according to the joint image and RFID information to establish a linked list, and the linked list data is constantly improved with detection, and the joint image of the joint node is also constantly updated. The current detected joint information is matched with the historical joint information (RFID tag matching or image matching), and if the matching is successful, the historical joint number is output, and the historical joint node information is updated, if the matching is not successful, it is considered to be a new joint node identified for the first time, the node is inserted into the linked list, and the new node stores the image, RFID tag (not identified is empty) and new number of the joint. The specific steps are as follows:

[0108] A, in response to not detecting an RFID tag in the joint area, or not matching the same RFID tag information, feature matching the joint image with the historical joint image of each joint node in the linked list;

[0109] B, in response to matching the corresponding historical joint image, replacing the joint image with the historical joint image of the corresponding joint node;

[0110] C, in response to not matching the corresponding historical joint image, inserting a new joint node in the linked list based on the detected RFID tag and / or joint image.

[0111] The situation of inserting a new node here is mainly for those missed detection situations, especially the situation of not detecting the joint using X-ray image, but detecting the RFID chip, at this time, the inserted joint node does not contain the image, but only contains the RFID information, and the joint image can be determined and added from the historical joint image matching of the database at a later identification. The behavior of inserting the joint node in the linked list is represented as a missed detection situation.

[0112] As mentioned earlier, there are many defects in joints and non-joints, only defects in specific scenarios need to trigger early warning or manual intervention, such as non-joint area wear and tear, which is the result of long-term use, and does not need to be replaced until a certain degree of wear and tear is reached, while wear and tear or piercing of rivets at the joint needs to be timely warned and processed, so the application also provides a processing mechanism for distinguishing joint defects and non-joint defects.

[0113] Figure 10 is a flow chart of an algorithm for joint image numbering in combination with linked list detection, the numbering matching process can be summarized as follows:

[0114] A, in response to the defect area being located inside the joint area, obtaining the joint defect label library of the joint area according to the type of the conveying belt, and matching with the defect label of the defect of the conveying belt;

[0115] B, in response to matching to the same defect label, output the defect position and defect information at the joint node;

[0116] C, in response to the defect area being located outside the joint area, acquiring a defect label library of the non-joint area according to the conveyor belt type, and matching with the defect label of the conveyor belt defect;

[0117] D, in response to matching to the same defect label, output the defect position and defect information at the non-joint.

[0118] This algorithm flow lacks the defect type and position information output by the foregoing defect detection model, and combines the joint position information output by the joint detection model to distinguish whether the defect belongs to a joint defect or a non-joint defect, finally outputs the detection result, and generates a defect detection report according to the requirement.

[0119] In summary, compared with the detection algorithm in the related art, the present application has the following advantages:

[0120] 1. At present, the recognition of the conveyor belt joint mainly relies on the bonding mark on the surface of the belt, or is realized according to the recognition of the RFID chip embedded in the belt. Compared with the prior art, the present application can improve the recognition accuracy of joint target detection;

[0121] 2. Based on the combination of the AI target model and the traditional CV analysis mechanism and the fusion of RFID information, the present application can realize more accurate recognition of the identity of the conveyor belt joint, improve the recognition accuracy of the joint small target defect through optimization of the AI model design, and improve the accuracy of the joint detection report;

[0122] 3. The present application can provide defect position data for the monitoring system to assist in improving the accuracy of the joint detection report;

[0123] FIG. 11 shows a structural block diagram of a conveyor belt skeleton joint defect detection device provided by an embodiment of the present application, which comprises:

[0124] A first detection module 1110 is configured to acquire an X-ray image of a conveyor belt in real time through an X-ray scanning device, detect and identify a conveyor belt skeleton joint and a conveyor belt defect in the X-ray image;

[0125] A second detection module 1120 is configured to extract a joint image from the X-ray image in response to identifying the skeleton joint, and detect an RFID tag of a joint area where the skeleton joint is located; the RFID tag records skeleton joint information;

[0126] A matching output module 1130 is configured to output a defect label of the conveyor belt defect in response to identifying the conveyor belt defect, match a defect area where the conveyor belt defect is located with the joint area in terms of position and label, and output a defect detection result of the skeleton joint according to the position matching result and the label matching result.

[0127] The conveyor belt skeleton joint defect detection device provided in the embodiments of the present application can be applied to the conveyor belt skeleton joint defect detection method provided in the above embodiments, and the related details are referred to the above method embodiments, which have similar implementation principles and technical effects, and will not be described here.

[0128] It should be noted that the conveyor belt skeleton joint defect detection device provided in the embodiments of the present application is only exemplified by the above division of each functional module / functional unit, and in actual application, the above functions can be completed by different functional modules / functional units according to needs, that is, the internal structure of the conveyor belt skeleton joint defect detection device is divided into different functional modules / functional units to complete all or part of the above described functions. In addition, the implementation manner of the conveyor belt skeleton joint defect detection method provided in the above method embodiments and the implementation manner of the conveyor belt skeleton joint defect detection device provided in the present embodiment belong to the same concept, and the specific implementation process of the conveyor belt skeleton joint defect detection device provided in the present embodiment is described in the above method embodiments, which will not be described here.

[0129] FIG. 12 shows a structural block diagram of a computer device provided in an example embodiment of the present application. The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer device can include, but is not limited to, a processor and a memory. The processor and the memory can be connected through a bus or other means. The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, graphics processing units (GPUs), embedded neural network processing units (NPUs) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components, and the like chips, or combinations of the above chips.

[0130] The processor can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor can also include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0131] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, i.e. implements the methods in the above method embodiments. The memory can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created by the processor, and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] In some embodiments, the computer device can also optionally include a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit, a display screen, a keyboard.

[0133] The peripheral interface can be used to connect at least one I / O (Input / Output) related peripheral device to the processor and the memory. In some embodiments, the processor, the memory and the peripheral interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory and the peripheral interface can be implemented on a separate chip or circuit board, to which the embodiments are not limited.

[0134] The display screen is used to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen is a touch display screen, the display screen also has the ability to collect touch signals on or above the surface of the display screen. The touch signals can be input as control signals to the processor for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in some other embodiments, the display screen can be at least two, arranged on different surfaces of the computer device or in a folding design; in some other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can also be arranged in an irregular shape other than a rectangle, i.e. a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0135] The power supply is used to supply power to various components in the computer device. The power supply can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0136] Those skilled in the art can understand that the structure shown in the embodiments does not constitute a limitation on the computer device, and the computer device can include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0137] The embodiments of the present application further disclose a computer readable storage medium. Specifically, the computer readable storage medium is used to store a computer program, and the computer program is executed by a processor to implement the method in the above method embodiments. Those skilled in the art can understand that all or part of the processes in the above method embodiments of the present application can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above method embodiments. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0138] The specific embodiments are only an explanation of the present application, and are not a limitation of the present application. Those skilled in the art can make modifications to the embodiments according to the needs after reading the present specification, and the modifications do not contribute to the creativity, but are protected by the patent law as long as the modifications are within the scope of the claims of the present application.

Claims

1. A method of detecting a joint defect of a conveyor belt carcass, characterized in that, The method comprises: Real-time acquisition of an X-ray image of the conveying belt by an X-ray scanning device, detection and identification of the conveying belt skeleton joint and the conveying belt defect in the X-ray image; In response to identification of the skeleton joint, extraction of a joint image from the X-ray image, and detection of an RFID tag in a joint area where the skeleton joint is located; the skeleton joint information is recorded in the RFID tag; In response to identification of the conveying belt defect, output of a defect label of the conveying belt defect, position and label matching of a defect area where the conveying belt defect is located and the joint area, and output of a defect detection result of the skeleton joint according to the position matching result and the label matching result.

2. The method of claim 1, wherein, The detection and identification of the conveying belt skeleton joint and the conveying belt defect in the X-ray image comprise: feeding the collected X-ray image into a joint detection model for conveying belt identification and joint identification, and outputting the confidence information of the conveying belt type and the joint; confirming the confidence information of the joint by a CV algorithm, and outputting the joint information; feeding the collected X-ray image into a defect detection model, and performing defect detection according to the confirmed conveying belt type; the defect comprises at least one of a rivet, a tear, a foreign matter and wear.

3. The method of claim 2, wherein, The joint detection model is a yolo detection model, comprising a first backbone network, a first neck structure and a first head structure; The input X-ray image has a size of 640*640, and the feature map is extracted by the first backbone; the convolution step of the convolution module Conv in the first backbone is 2, the feature is extracted by down-sampling, the spatial pyramid pooling SPPF module is used for up-sampling, and the multi-scale features are fused; the feature map output by the first backbone is fed into the first neck structure, the shallow features and the deep semantic features are fused by the feature pyramid FPN structure, the first neck structure output is connected with the first head structure, and the recognition result is output by convolution; The loss function L of the joint detection model is represented as follows: L = 1.0 * lbo j + 0.05 * lbo x + 0.0 * lcl s wherein, lob j is the target confidence loss; lbo x is the bounding box loss; lcl s is the class judgment loss.

4. The method of claim 2, wherein, The conveying belt type is divided into aramid fiber conveying belt and steel wire conveying belt; The confirmation of the confidence information of the joint by the CV algorithm and the output of the joint information comprise: when the output detection result is the aramid fiber conveying belt, the conveying belt image is subjected to binaryzation processing and erosion operation to extract foreground and background elements; the binaryzation image is subjected to connectivity analysis and connected domain filtering to obtain a filtered morphological image; when the morphological image matches the joint shape of the aramid fiber conveying belt joint, the corresponding joint information and joint image are output; when the output detection result is the steel wire conveying belt, the conveying belt image is subjected to binaryzation processing to identify the steel wire rope features therein; the image is segmented into small area graphics, the number of steel wire ropes in different areas is calculated according to the pixel density, and the joint position and the non-joint position are determined; when the number of steel wire ropes at the joint and the non-joint position satisfies the proportional relationship, the corresponding joint information and joint image are output.

5. The method of claim 1, wherein, The detection of the RFID tag in the joint area where the skeleton joint is located comprises: in response to detecting the RFID tag at the joint region, matching the RFID tag with historical RFID tags in a chain table; the chain table storing RFID tag information and associated historical joint images of all identified joint nodes; in response to matching the same RFID tag information, updating the joint image to a joint node in the chain table.

6. The method of claim 5, wherein, in response to not detecting the RFID tag at the joint region or not matching the same RFID tag information, performing feature matching of the joint image with historical joint images of each joint node in the chain table; in response to matching the corresponding historical joint image, replacing the historical joint image of the corresponding joint node with the joint image; in response to not matching the corresponding historical joint image, inserting a new joint node in the chain table based on the detected RFID tag and / or joint image.

7. The method of claim 6, wherein, The defect detection result of the skeleton joint is output according to the position matching result and the tag matching result, including: in response to the defect region being located inside the joint region, obtaining a joint defect tag library of the joint region according to the type of the conveying belt, and matching the defect tag of the conveying belt defect; in response to matching the same defect tag, outputting the defect position and defect information at the joint node; in response to the defect region being located outside the joint region, obtaining a defect tag library of a non-joint region according to the type of the conveying belt, and matching the defect tag of the conveying belt defect; in response to matching the same defect tag, outputting the defect position and defect information at the non-joint.

8. The method of claim 3, wherein, The X-ray image is input to a defect detection model for detection, and the defect detection model includes a second backbone network, a second neck structure, and a decoupling head; the second neck includes four feature upper fusion branches and corresponding feature lower fusion branches, and the decoupling head is located in four shallow layer feature extraction units connected to the outputs of the four feature lower fusion branches. The input X-ray image is 1280*1280 in size, and the feature map is extracted by the second backbone. At least one hole convolution is added in the first C3 layer structure in the second backbone to extract small target defect features in the image.

9. A conveyor belt carcass joint defect detection apparatus characterized by, The device includes: a first detection module for acquiring an X-ray image of a conveying belt in real time through an X-ray scanning device, detecting and identifying a skeleton joint of the conveying belt and a conveying belt defect in the X-ray image; a second detection module for extracting a joint image from the X-ray image and detecting an RFID tag of a joint region where the skeleton joint is located in response to identifying the skeleton joint; the RFID tag records the skeleton joint information; a matching output module for outputting a defect tag of the conveying belt defect in response to identifying the conveying belt defect, performing position and tag matching of a defect region where the conveying belt defect is located with the joint region, and outputting a defect detection result of the skeleton joint according to a position matching result and a tag matching result.

10. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to realize the conveyor belt skeleton joint defect detection method as claimed in any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to realize the conveyor belt skeleton joint defect detection method as claimed in any one of claims 1 to 8.

Images