Crack detection method, electronic equipment and computer readable storage medium

Abstract

The invention provides a crack detection method, electronic equipment and a computer readable storage medium, and the method comprises the steps: carrying out the crack recognition of image data of a target conveyor belt, so as to obtain an initial recognition result; performing region segmentation on the image data of the target conveyor belt to identify a belt region and a non-belt region so as to obtain a segmentation result; and determining a target identification result of the target conveyor belt based on the initial identification result and the segmentation result.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a crack detection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Conveyor belts are used to transport items, but it is inevitable that there may be some friction during the conveying process, which may cause damage such as cracks on the conveyor belts. In order to improve the safety of conveyor belts, crack detection of conveyor belts is required. However, the current detection method has a low accuracy rate. Summary of the invention

[0003] The purpose of the present application is to provide a crack detection method, an electronic device and a computer-readable storage medium, which can improve the accuracy of crack detection.

[0004] In a first aspect, the present invention provides a crack detection method, comprising: performing crack identification on image data of a target conveyor belt to obtain an initial identification result; performing region segmentation on the image data of the target conveyor belt to identify a belt area and a non-belt area to obtain a segmentation result; based on the initial identification result and the segmentation result, determining a target identification result of the target conveyor belt, wherein if the crack of the initial identification result is located in the belt area, then determining the target identification result as the presence of a crack in the target conveyor belt.

[0005] In the above embodiment, the crack detection result and the identification result of the belt area and the non-belt area can be combined to determine the final crack detection result based on the two types of results, which can avoid identifying the texture of the non-belt area as cracks in the conveyor belt, and can improve the accuracy of crack identification.

[0006] In an optional embodiment, the initial recognition result includes the relative position of the crack in the image; determining the recognition result of the target conveyor belt based on the recognition result and the segmentation result includes: determining whether the relative position is in the belt area based on the relative position in the recognition result and the belt area in the segmentation result; if the relative position is in the belt area, determining that the target recognition result is that there is a crack in the target conveyor belt.

[0007] In an optional embodiment, the image data includes an image data stream of the target conveyor belt in a moving state; and the method further includes: determining a current position of the crack based on the image data stream.

[0008] In an optional embodiment, determining the current position of the crack based on the image data stream includes: determining the moving pixel distance of the crack based on the image data stream; determining the current position of the crack based on the actual width of the target conveyor belt, the pixel width of the target conveyor belt and the moving pixel distance.

[0009] In the above implementation, cracks can also be located, which can help relevant technicians find cracks more quickly.

[0010] In an optional embodiment, the method further includes: if the target recognition result of the target conveyor belt is that there are cracks, outputting an alarm message.

[0011] In the above implementation, an alarm message can also be output based on the identification result, so as to promptly prompt relevant technical personnel and take relevant protective measures in time to improve the operational safety of the conveyor belt.

[0012] In an optional embodiment, the performing crack recognition on the image data of the target conveyor belt to obtain an initial recognition result includes: using a target detection network model to perform crack recognition on the image data of the target conveyor belt to obtain an initial recognition result.

[0013] In an optional embodiment, a training data set used to train the target detection network model is constructed in the following manner: for each initial crack image in the initial crack image set, the initial crack image is transformed to obtain multiple belt crack transformation images corresponding to the initial crack image to obtain a training data set.

[0014] In the above implementation, more images can be obtained as training data sets by performing transformation processing based on a small number of initial crack images.

[0015] In an optional embodiment, the transforming process is performed on the initial crack image to obtain a plurality of belt crack transformation images corresponding to the initial crack image, comprising: transforming the initial crack image to obtain a belt crack transformation image; segmenting the belt crack transformation image to obtain a belt area division result, wherein the belt area division result includes a belt area and a non-belt area; based on the band area division result, identifying whether the crack in the belt crack transformation image is located in a belt area; if the crack in the belt crack transformation image is located in a belt area, then classifying the belt crack transformation image into a training data set.

[0016] In the above embodiment, the band area and the non-band area can be first identified by segmenting the initial crack image, so that cracks can be found in the band area of ​​the image in the training data obtained by the transformation. The training data set obtained in this way can be more accurate and can also reduce the difficulty of collecting training data.

[0017] In an optional embodiment, the transforming process of the initial crack image to obtain a belt crack transformation image includes: deforming the initial crack image to obtain a deformed image; and fusing the deformed image with a conveyor belt image to obtain a belt crack transformation image.

[0018] In an optional embodiment, the method further includes: if the crack in the belt crack transformation image is located in a non-belt area, re-merging the deformed image with the conveyor belt image to obtain a secondary belt crack transformation image; segmenting the secondary belt crack transformation image to obtain a belt area division result, the belt area division result includes a belt area and a non-belt area; based on the belt area division result, identifying whether the crack in the secondary belt crack transformation image is located in a belt area; if the crack in the secondary belt crack transformation image is located in a belt area, classifying the secondary belt crack transformation image into a training data set.

[0019] In the above implementation, when the transformed image does not meet the condition that the crack is in the band area, it can be directly reintegrated. While protecting the obtained crack image as much as possible, there is no need to re-perform the calculation required for the deformation processing, which can improve the efficiency of obtaining training data and ensure the validity of the training data.

[0020] In a second aspect, the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method described in any one of the aforementioned implementations.

[0021] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are executed.

[0022] In a fourth aspect, the present invention provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method described in any one of the aforementioned embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of the operating environment of the crack detection method provided in the embodiment of the present application;

[0025] Figure 2 A block diagram of an electronic device provided in an embodiment of the present application;

[0026] Figure 3 A flow chart of a crack detection method provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of an application scenario of the crack detection method provided in an embodiment of the present application;

[0028] Figure 5 A partial flow chart of the crack detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] Conveyor belts are often used in the industrial field, and there may be some damage to the conveyor belts during use. In order to improve the safety of conveyor belts, it is necessary to detect the condition of the conveyor belts in use. At present, conveyor belt crack detection can be mainly divided into two types: non-destructive evaluation methods and digital image processing methods. Non-destructive evaluation methods mainly include ultrasonic conveyor belt crack detection, laser scanning conveyor belt crack detection, etc.; digital image processing methods mainly include abnormal detection conveyor belt crack recognition, conveyor belt crack detection based on machine learning, conveyor belt crack detection based on deep learning, etc.

[0032] For the non-destructive evaluation of conveyor belt crack detection methods: the ultrasonic conveyor belt crack detection method can penetrate thicker materials with a large penetration depth on the one hand, and can detect tiny cracks and defects on the other hand, providing high-precision detection results. However, it requires the use of a coupling agent between the probe and the conveyor belt surface, and the operation process is relatively cumbersome. In addition, the conveyor belt surface to be detected needs to be relatively flat, and the surface roughness will affect the detection effect. Therefore, it is often difficult to meet the requirements for conveyor belt surface crack detection with some more complex working conditions; the laser scanning conveyor belt crack detection method can provide very high resolution and high accuracy in detecting tiny cracks. It does not require contact with the conveyor belt and is suitable for high temperature, dangerous or inaccessible environments. However, external factors such as the smoothness of the conveyor belt surface, light and temperature may affect the accuracy and stability of the laser scanning, and the detection results are easily affected by the environment.

[0033] For the conveyor belt crack recognition method based on digital image processing method: Although the anomaly detection conveyor belt crack recognition method has the advantages of high degree of automation and sensitivity, it has the following two important defects: on the one hand, it depends on data quality: the detection effect depends on the quality and diversity of training data, and insufficient or poor data quality will affect the detection accuracy; on the other hand, the false alarm rate is high: a high false alarm rate may occur under complex backgrounds, affecting the detection effect. The conveyor belt crack detection method based on machine learning can adapt to various complex environments and backgrounds through learning and training to improve the detection accuracy, but it has the following shortcomings: on the one hand, the model complexity is high, the training process is time-consuming, and high-performance computing equipment is required; on the other hand, a large number of positive samples are required: when there are insufficient positive samples or the data distribution changes greatly, the generalization ability of the model may be limited, and it is difficult to accurately detect cracks for conveyor belts with complex working conditions. The current conveyor belt crack detection method based on deep learning has the advantages of low solution cost and simple model training, but it has the following disadvantages: on the one hand, it relies on high-quality real crack images: deep learning training requires a clear conveyor belt crack training set, but the working conditions in industrial sites are complex and changeable, and it is often difficult to collect a large number of clear pictures, and the number of cracks in industrial conveyor belts is very small, making it difficult to collect a large number of them; on the other hand, it relies on a large amount of multi-scene data: deep learning training requires a large amount of multi-scene data to ensure the generalization of the model, but conveyor belt cracks have problems such as small number, difficult collection and limited sample scenes.

[0034] The above-mentioned various conveyor belt crack detection methods also have the following common shortcomings: On the one hand, it is difficult to cope with complex working conditions: the existing crack detection methods have poor processing capabilities for complex working conditions such as the background and interference of the conveyor belt, which leads to the fact that the detection effects of these methods in complex environments are often difficult to meet the requirements. On the other hand, the generalization is poor: in the existing conveyor belt crack detection and identification, since each factory will purchase conveyor belts of different models and brands, non-destructive detection methods and traditional image recognition methods have limited adaptability and limited generalization of different working conditions. Another aspect is the inability to track the crack position: the current conveyor belt crack detection public patent focuses on crack detection and ignores crack position tracking, resulting in the inability to quickly and accurately locate the conveyor belt crack in the industrial site.

[0035] In view of the above-mentioned deficiencies, the embodiments of the present application provide a crack detection method, an electronic device and a computer-readable storage medium, which can achieve more accurate positioning without requiring high requirements for the collection of training data.

[0036] To facilitate understanding of this embodiment, the operating environment of a crack detection method disclosed in the embodiment of the present application is first introduced in detail.

[0037] like Figure 1 , which is a schematic diagram of the operating environment of the crack detection method provided in the embodiment of the present application. The operating environment may include an electronic device 110, which can process the collected image of the conveyor belt to identify whether there is a crack in the conveyor belt.

[0038] In this embodiment, the electronic device 110 may be a network server, a database server, etc., or may be a personal computer (PC), a tablet computer, a smart phone, a personal digital assistant (PDA), etc.

[0039] Exemplarily, the crack detection method can be used in a factory with a conveyor belt, and one or more acquisition devices 120 can be set in the operating environment, and the acquisition device 120 can collect pictures or videos of the conveyor belt. The acquisition device 120 can transmit the collected pictures or videos to the electronic device 110 for the electronic device 110 to identify and process them.

[0040] In an optional scenario, a plurality of collection devices 120 may be provided in the operating environment, and each collection device 120 may be installed at a plurality of positions within the operating range of the conveyor belt, so as to realize collection of the conveyor belt at a plurality of positions.

[0041] In some examples, at least two collection devices 120 may be provided, one of which is provided at a corner of the conveyor belt, and the other is provided at a third of the conveyor belt. Of course, more or fewer collection devices 120 may be provided based on actual needs. For example, if the conveyor belt is relatively long, more collection devices 120 may be provided; for another example, if the conveyor belt is relatively short, fewer collection devices 120 may be provided.

[0042] In some optional implementations, the electronic device 110 and the acquisition device 120 may also be two-in-one devices, that is, the electronic device 110 is integrated with an acquisition function for acquiring image data of the conveyor belt. On this basis, no additional acquisition device may be provided in the operating environment.

[0043] like Figure 2 , which is a block diagram of an electronic device. The electronic device 110 may include a memory 111 and a processor 113. A person skilled in the art may understand that Figure 2 The structure shown is only for illustration and does not limit the structure of the electronic device 110. For example, the electronic device 110 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown.

[0044] The memory 111 and the processor 113 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The processor 113 is used to execute the executable module stored in the memory.

[0045] The memory 111 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 111 is used to store programs, and the processor 113 executes the program after receiving the execution instruction. The method executed by the electronic device 110 defined by the process disclosed in any embodiment of the present application can be applied to the processor 113, or implemented by the processor 113.

[0046] The processor 113 may be an integrated circuit chip with signal processing capability. The processor 113 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), a dedicated integrated circuit.

[0047] (Application Specific Integrated Circuit, referred to as ASIC), Field Programmable Gate Array (Field Programmable Gate Array, referred to as FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0048] The electronic device 110 in this embodiment can be used to execute each step in each method provided in the embodiments of the present application. The implementation process of the crack detection method is described in detail below through several embodiments.

[0049] See also Figure 3 , is a flow chart of a crack detection method provided in an embodiment of the present application. The crack detection method provided in an embodiment of the present application can be applied to an electronic device, and the steps in the crack detection method are performed by the electronic device. Figure 3 The specific process shown is described in detail.

[0050] Step 210, performing crack recognition on the image data of the target conveyor belt to obtain an initial recognition result.

[0051] Optionally, the identification of cracks can be achieved by a pre-trained machine learning model, which can classify the image data to classify whether there are cracks.

[0052] Alternatively, image recognition may be used. Exemplarily, the image data may be a depth image, and the presence of cracks may be determined by comparing pixels of the depth image. For example, adjacent pixels may be compared to screen out pixels with pixel mutations, which may indicate that cracks may exist there.

[0053] Step 220 , performing region segmentation on the image data of the target conveyor belt to identify the belt area and the non-belt area to obtain a segmentation result.

[0054] Optionally, edge recognition may be performed to determine the edge of the target conveyor belt in the image data. Then, the belt area and the non-belt area in the image data may be identified based on the determined edge. The belt area represents the area where the target conveyor belt is located in the image data, and the non-belt area represents the area in the image data that does not belong to the target conveyor belt.

[0055] Step 230 , determining a target recognition result of the target conveyor belt based on the initial recognition result and the segmentation result.

[0056] Among them, if the crack of the initial recognition result is located in the belt area, then the target recognition result is determined to be that there is a crack in the target conveyor belt.

[0057] In one scenario, if the presence of a crack is detected in step 210 and the location of the crack is detected in the belt area in step 220, it can be indicated that a crack exists on the target conveyor belt.

[0058] In one scenario, if the presence of a crack is detected in step 210 and the location of the crack is detected in step 220 to be located in the belt area and the non-belt area, it can be indicated that a crack exists on the target conveyor belt.

[0059] In one scenario, if the presence of a crack is detected in step 210 and the location of the crack is detected in a non-belt area in step 220, it can be indicated that there is no crack on the target conveyor belt.

[0060] In one scenario, if no cracks are detected in step 210 , it can be concluded that no cracks exist on the target conveyor belt without knowing the result of step 220 .

[0061] In the above implementation, the detection result of the conveyor belt not only depends on the detection of cracks, but also depends on obtaining the position of the cracks in the image, thereby reducing the situation where some marks outside the conveyor belt area are identified as cracks, and improving the accuracy of conveyor belt crack identification.

[0062] In this embodiment, the initial recognition result includes the existence of a crack and the relative position of the crack in the image. The image of the relative position of the crack in the image may be image data of the target conveyor belt based on the detection to obtain the initial recognition result.

[0063] Exemplarily, the relative position may be represented by a pixel position, for example, the pixel coordinates of the crack in the image data are used as the above relative position.

[0064] The above-mentioned step 230 may include: determining whether the relative position is within the band based on the relative position in the recognition result and the band in the segmentation result.

[0065] If the relative position is in the belt area, the target recognition result is determined to be that the target conveyor belt has cracks. If the relative position is in the non-belt area, the target recognition result is determined to be that the target conveyor belt does not have cracks.

[0066] Optionally, the relative position of the crack in the image can be represented by pixel coordinates in the image data. The pixel coordinates of the crack can be compared with the pixel coordinates of the belt area to determine whether the pixel coordinates of the crack are within the pixel coordinate range of the belt area. If the pixel coordinates of the crack fall within the pixel coordinate range of the belt area, it can be determined that the target recognition result is that there is a crack in the target conveyor belt.

[0067] Optionally, the relative position of the crack in the image can be represented by a two-dimensional coordinate in a two-dimensional coordinate system constructed based on the target conveyor belt. The two-dimensional coordinate of the crack can be compared with the coordinate of the belt area to determine whether the two-dimensional coordinate of the crack is within the coordinate range of the belt area.

[0068] Optionally, the relative position of the crack in the image can be represented by a three-dimensional coordinate in a three-dimensional coordinate system constructed based on the target conveyor belt. The three-dimensional coordinates of the crack can be compared with the coordinates of the belt area to determine whether the three-dimensional coordinates of the crack are within the coordinate range of the belt area.

[0069] Through the above recognition logic, crack recognition results only in the belt area can be screened out, which can make the recognition results more reliable.

[0070] In a usage scenario, the target conveyor belt may be several kilometers or even longer. If the existence of a crack is confirmed, it is relatively difficult to find the crack within a few kilometers. Therefore, in order to facilitate users to locate the crack more quickly, the crack can be located and tracked.

[0071] Based on this, the image data used in the embodiment of the present application may include an image data stream when the target conveyor belt is in a moving state. For example, the image data may be used to detect and track cracks when the target conveyor belt is in use.

[0072] The crack detection method of this embodiment further includes: step 240, determining the current position of the crack according to the image data stream.

[0073] Optionally, the time difference can be determined based on the time when the crack is detected and the current time. The moving speed of the target conveyor belt can be determined based on the image data stream to determine the moving distance of the crack.

[0074] For example, the actual width of the target conveyor belt can be known in advance, and the number of pixels in the width direction of the target conveyor belt can be determined based on the image data of the target conveyor belt. The pixel movement distance per unit time can be determined based on the image data stream, and finally the moving speed of the target conveyor belt can be determined based on the actual width of the target conveyor belt and the pixel movement distance per unit time.

[0075] The actual width of the target conveyor belt may be given in centimeters.

[0076] Optionally, the moving speed of the conveyor belt may be known in advance, and the crack moving distance may be determined based on the moving speed of the target conveyor belt and the time difference determined based on the time when the crack is detected and the current time. Exemplarily, the crack moving distance may be obtained by multiplying the time difference with the moving speed of the target conveyor belt.

[0077] Based on the crack travel distance and the detected crack position, the current position of the crack is determined.

[0078] In one implementation, the above step 240 may include: step 241 and step 242 .

[0079] Step 241, determining the moving pixel distance of the crack according to the image data stream.

[0080] Exemplarily, tracking may be performed based on the image data stream to determine the moving pixel distance from the time when the crack is detected to the current time.

[0081] Step 242, determining the current position of the crack according to the actual width of the target conveyor belt, the pixel width of the target conveyor belt, and the moving pixel distance.

[0082] In this embodiment, the actual width of the target conveyor belt and the number of pixels of the target conveyor belt in the width direction can be stored in advance. Based on the actual width of the target conveyor belt and the number of pixels of the target conveyor belt in the width direction, the relationship between the number of pixels and the actual length can be determined, and then based on the moving pixel distance and the relationship between the number of pixels and the actual length, the distance moved by the crack can be determined. Finally, based on the detected position of the crack and the distance moved by the crack, the current position of the crack can be determined.

[0083] The following combines some practical algorithms to implement the determination of the moving pixel distance in the above step 241.

[0084] Optionally, a corner point detection method may be used to extract feature points in the target conveyor belt, and the corner point detection method may be a Shi-Tomasi corner point detection method.

[0085] First, the Shi-Tomasi corner point detection method is used to extract the feature points in the conveyor belt. The autocorrelation matrix of the conveyor belt image at the coordinates can be expressed as:

[0086]

[0087] in represents the gradient of the image in the x direction, It represents the gradient of the image in the y direction. When the minimum eigenvalue of the autocorrelation matrix M is less than the set threshold, the point (x, y) is considered to be a feature point.

[0088] Then, an optical flow method may be used to track the motion of the feature point. For example, the optical flow method may be a Lucas-Kanade optical flow method.

[0089] The optical flow equation of the target conveyor belt at the feature point (x, y) can be expressed as:

[0090] I x u+I y v+I t =0;

[0091] Among them, u and v represent the components of the optical flow vector in the x and y directions respectively. Represents the gradient of the image data of the target conveyor belt in the direction of time t. The above optical flow equation can be used to solve the optical flow vectors u and v. The new position (x′, y′) of the feature point of the target conveyor belt can be calculated by the following formula:

[0092]

[0093] Optionally, in order to filter out erroneous matching points, a random sampling consensus algorithm (RANSAC for short) may be used to filter out abnormal points, thereby improving the stability and accuracy of tracking.

[0094] Repeated execution of the optical flow method can obtain the trajectory of the characteristic points of the target conveyor belt and its linear displacement. Combined with the image data stream of the target conveyor belt, the time difference before and after can be determined to infer the speed v of the target conveyor belt. belt , by integrating the speed of the target conveyor belt, the conveyor belt moving pixel distance s′(pixel) is calculated. The calculation process can be shown as follows:

[0095] s′=∫v belt dt;

[0096] In this embodiment, the moving distance s of the target conveyor belt can be calculated based on the pixels m (pixel) in the width direction of the target conveyor belt and the actual width n (mm) of the target conveyor belt. The calculation formula can be as follows:

[0097]

[0098] In some practical application scenarios, the conveyor belt may not run in a completely straight line, but may run in a circular shape. Figure 4 As shown, the two ends run in semicircular trajectories and the middle runs in a straight line.

[0099] exist Figure 4 In the example shown, the position of the semicircular trajectory at one end can be set as a reference point O, and the distance from the reference point O can be used to represent the relative reference point position D of the crack.

[0100] The relative reference point position D of the crack can be expressed by the following formula:

[0101]

[0102] Where d represents the distance between the reference point O and the crack monitoring point P, and l represents the total length of the target conveyor belt.

[0103] The crack monitoring point can indicate the location where the crack is detected, or it can be the location of the electronic device or acquisition device.

[0104] By tracking the distance the crack moves, users can find the location of the crack on the conveyor belt more quickly and improve the operating safety of the target conveyor belt.

[0105] In order to make the detection results more quickly known to the user and facilitate the subsequent crack processing, an alarm can also be output based on the detection results. Based on this requirement, the crack detection method of the present application can also include the step of outputting an alarm message if the target recognition result of the target conveyor belt is that there is a crack.

[0106] The alarm message may include a prompt signal, and the prompt signal may include a sound prompt signal, a light prompt signal, etc.

[0107] Optionally, the output alarm message may also include a location of the crack.

[0108] Based on the output of alarm messages, relevant technical personnel can be prompted in time to improve the operating safety of the conveyor belt.

[0109] In this embodiment, the above-mentioned initial recognition result can be obtained by using a network model. Based on this, the above-mentioned step 210 can include: using the target detection network model to perform crack recognition on the image data of the target conveyor belt to obtain the initial recognition result.

[0110] Optionally, the target detection network model may be a model determined by a YOLOv8-ST target detection network.

[0111] Exemplarily, the YOLOv8-ST target detection network may be a detection network obtained by replacing the basic network module of YOLOv8 with an improved Swin-Transformer, which may be referred to as a YOLOv8-ST network.

[0112] Considering that there may be only a few conveyor belts with cracks in reality, the image data of cracks used to train the model will also be relatively small. In order to increase the amount of training data, more training data can be obtained by deforming a small number of crack images. Based on this, the training data set used by the training target detection network model is constructed in the following way: for each initial crack image in the initial crack image set, the initial crack image is transformed to obtain multiple belt crack transformation images corresponding to the initial crack image, so as to obtain a training data set.

[0113] The initial crack image set may be image data obtained by photographing a transmission having cracks.

[0114] like Figure 5 As shown, the above-mentioned transformation processing of the initial crack image to obtain a plurality of belt crack transformation images corresponding to the initial crack image may include steps 310 to 330.

[0115] Step 310, transforming the initial crack image to obtain a belt crack transformation image.

[0116] In this embodiment, the initial crack images in the initial crack image set may be similar conveyor belt crack images collected by crawler technology.

[0117] Exemplarily, the transformation process may include a deformation process and a fusion process.

[0118] The deformation process may include enlarging the cracks therein. The deformation may also include shrinking the cracks therein. The deformation may also include rotating the cracks therein.

[0119] Optionally, the deformation may be one or more of the above-mentioned enlargement processing, reduction processing, and rotation deformation. Exemplarily, one of the processes may be performed to obtain the transformed crack; the superposition effect of the three deformations of enlargement processing, reduction processing, and rotation deformation may be performed to obtain the transformed crack; and two deformations may be selected from the three deformations of enlargement processing, reduction processing, and rotation deformation to obtain the transformed crack.

[0120] Optionally, after deformation, the cracks in the initial crack image can be deformed. The crack image obtained after deformation is then fused with the conveyor belt image to obtain a conveyor belt image with cracks, which is used as the belt crack transformation image.

[0121] Step 320, segmenting the belt crack transformation image to obtain a belt area division result, where the belt area division result includes a belt area and a non-belt area.

[0122] The segmentation processing logic used in step 320 may be the same as the segmentation processing logic used in the aforementioned step 220 .

[0123] Step 330: Based on the belt area division result, identify whether the crack in the belt crack transformation image is located in the belt area.

[0124] If the crack in the belt crack transformation image is located in the belt area, the belt crack transformation image is included in the training data set.

[0125] Optionally, if the crack in the belt crack transformation image is located in the non-belt area and the belt area does not contain the crack, the belt crack transformation image is deleted, and the logic of the above steps 310 to 330 is used again to generate training data.

[0126] Optionally, if the crack in the belt crack transformation image is located in the non-belt area, and the belt area does not contain cracks, step 310 can be used to perform a re-fusion process to obtain a conveyor belt image with cracks, that is, a secondary belt crack transformation image. Then, the method of step 320 is used to verify it to determine whether there is a crack in the belt area. Repeat the above operation until a belt crack transformation image with cracks in the belt area is obtained.

[0127] The transformation process implemented in the above step 310 is introduced below with reference to formulas.

[0128] The initial crack image can be scaled and linearly interpolated. For example, the pixel coordinates of the crack image (x, y) are the pixel coordinates of the scaled crack image (x′, y′). In this example, the scaling factors in the x-axis and y-axis directions are f and f respectively. x and f y , then the pixel value I′(x′, y′) on the target image after scaling transformation can be calculated by the following formula:

[0129] I′(x′,y′)=(1-a)(1-b)I(x1,y1)+a(1-b)I(x2,y1)+(1-a)bI(x1y2)+abI(x2,y2)

[0130] in,

[0131]

[0132] After the initial crack image is scaled, it is then rotated. The transformation process can be achieved by the following formula:

[0133]

[0134] Where x″ and y″ are the coordinates after rotation transformation, c x and c y is the coordinate of the rotation center point.

[0135] After completing the scaling and rotation transformation of the network conveyor belt crack image, we will perform Alpha fusion on it and the conveyor belt image. The fusion process is as follows:

[0136] I′(x, y)=α(x, y)I 1 (x, y) + (1 - α (x, y)) I 2 (x, y);

[0137] Among them, I′(x, y) represents the pixel value of the fused image at the coordinate (x, y), I 1 (x, y) represents the pixel value of the crack image at the coordinate (x, y) after the transformation. 2 (x, y) represents the pixel value of the conveyor belt image at the coordinate (x, y), and α(x, y) is the mask value of Alpha fusion at (x, y).

[0138] In this example, the crack image obtained by the transformation process is fused with the conveyor belt image to obtain an image that can be determined as a belt crack transformation image.

[0139] In order to make the obtained belt crack transformation image accurately represent the conveyor belt with cracks, the segmentation process can be combined to determine whether the crack in the belt crack transformation image is in the belt area. If the position of the crack exceeds the conveyor belt area, it can be re-fused to obtain a secondary belt crack transformation image, and then identify whether it is in the belt area until the crack is fused to the conveyor belt area, and the image of the crack fused to the conveyor belt area is used as training data.

[0140] In this embodiment, the loss function used by the trained target detection network model can be as follows:

[0141]

[0142] Among them, L cls , L box and L obj represents the separation loss, bounding box loss and objectness loss. cls represents the classification loss coefficient; λ box represents the bounding box loss coefficient; λ obj represents the target loss coefficient; y i represents the category label; IOU represents the intersection-over-union ratio; b represents the center point of the predicted rectangular box; bg t represents the center point of the target rectangular box; ρ represents the Euclidean distance between the predicted rectangular box and the true rectangular box; c represents the diagonal distance between the closed area of ​​the predicted rectangular box and the true rectangular box; v represents the consistency of the relative proportions of the predicted rectangular box and the true rectangular box; α represents the consistency weight coefficient; y obj represents the target label; p obj Probability of target existence.

[0143] The following is an introduction to the implementation of segmentation processing using some formulas.

[0144] Optionally, a semantic segmentation model may be used to segment the image to identify band areas and non-band areas in the image.

[0145] Exemplarily, the semantic segmentation model may be a UNet semantic segmentation model.

[0146] The UNet semantic segmentation model can be obtained through supervised training. For example, clear conveyor belt images can be collected at industrial sites and segmented and annotated to obtain a conveyor belt segmentation dataset. The UNet segmentation model is trained based on the dataset, and the Dice loss is used as the loss function during the training process. The specific definition is as follows:

[0147]

[0148] Where N is the total number of pixels, y i represents the true label of the i-th element, p i Represents the predicted probability of the i-th element.

[0149] In the embodiment of the present application, similar conveyor belt crack images can be collected first, and the crack images can be transformed using graphic transformation processing technology to ensure the diversity of crack image samples. The crack images can also be fused with normal conveyor belts using image fusion to obtain a large number of simulated conveyor belt crack data sets, which can be used to train a crack detection model. This can alleviate the problem of difficulty in obtaining real conveyor belt crack images, because crack samples can be made diverse through crack image transformation processing, which can also solve the problem of poor crack detection effect in new conveyor belt scenarios.

[0150] Furthermore, in the embodiment of the present application, normal conveyor belt images collected at industrial sites are used to train a conveyor belt segmentation model to ensure high accuracy in the crack fusion and detection process. The UNet segmentation model segments the fused conveyor belt to ensure that the cracks are fused into the conveyor belt area, and the segmentation model segments the real-time running conveyor belt to ensure the reliability of the detection results.

[0151] Furthermore, in the embodiments of the present application, not only the crack detection is realized, but also the crack tracking can be realized based on the real-time collected image data stream of the conveyor belt, so as to realize the real-time feedback of the position of the crack on the conveyor belt, which is convenient for later maintenance and crack location. After determining that there is a crack or determining the moving distance of the crack, an alarm can also be output, so that the user can obtain the detection result more timely.

[0152] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the crack detection method described in the above method embodiment are executed.

[0153] The computer program product of the crack detection method provided in the embodiment of the present application includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the crack detection method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0154] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0155] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0156] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variation thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the presence of other identical elements in the process, method, article or apparatus that includes the elements.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0158] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A crack detection method, characterized in that: include: Perform crack recognition on the image data of the target conveyor belt to obtain an initial recognition result; Performing regional segmentation on the image data of the target conveyor belt to identify the belt area and the non-belt area to obtain a segmentation result; Based on the initial recognition result and the segmentation result, a target recognition result of the target conveyor belt is determined, wherein if the crack of the initial recognition result is located in the belt area, then the target recognition result is determined to be that the target conveyor belt has a crack.

2. The method according to claim 1, characterized in that The initial recognition result includes the relative position of the crack in the image; The step of determining the recognition result of the target conveyor belt based on the recognition result and the segmentation result includes: Based on the relative position in the recognition result and the band in the segmentation result, determining whether the relative position is in the band; If the relative position is located in the belt area, it is determined that the target recognition result is that cracks exist in the target conveyor belt.

3. The method according to claim 1, characterized in that The image data includes an image data stream of the target conveyor belt in a moving state; The method further includes: determining a moving distance of the crack according to the image data stream.

4. The method according to claim 3, characterized in that Determining the current position of the crack according to the image data stream includes: Determine the moving pixel distance of the crack according to the image data stream; The current position of the crack is determined according to the actual width of the target conveyor belt, the pixel width of the target conveyor belt and the moving pixel distance.

5. The method according to claim 1, characterized in that: The method further comprises: If the target recognition result of the target conveyor belt is that there are cracks, an alarm message is output.

6. The method according to any one of claims 1 to 5, characterized in that: The performing crack recognition on the image data of the target conveyor belt to obtain an initial recognition result includes: The target detection network model is used to perform crack recognition on the image data of the target conveyor belt to obtain the initial recognition result.

7. The method according to claim 6, characterized in that The training dataset used to train the target detection network model is constructed in the following way: For each initial crack image in the initial crack image set, the initial crack image is transformed to obtain a plurality of belt crack transformation images corresponding to the initial crack image, so as to obtain a training data set.

8. The method according to claim 7, characterized in that The transforming process is performed on the initial crack image to obtain a plurality of belt crack transformation images corresponding to the initial crack image, including: Performing transformation processing on the initial crack image to obtain a belt crack transformation image; Segmenting the belt crack transformation image to obtain a belt area division result, wherein the belt area division result includes a belt area and a non-belt area; Based on the belt zone division result, identifying whether the crack in the belt crack transformation image is located in the belt zone; If the crack in the belt crack transformation image is located in the belt area, the belt crack transformation image is included in the training data set.

9. The method according to claim 8, characterized in that The transforming process of the initial crack image to obtain the belt crack transformation image includes: Performing deformation processing on the initial crack image to obtain a deformed image; The deformed image is fused with the conveyor belt image to obtain a belt crack transformation image.

10. The method according to claim 9, characterized in that The method further comprises: If the crack in the belt crack transformation image is located in the non-belt area, the deformed image is re-fused with the conveyor belt image to obtain a secondary belt crack transformation image; Segmenting the secondary belt crack transformation image to obtain a belt area division result, wherein the belt area division result includes a belt area and a non-belt area; Based on the belt zone division result, identifying whether the crack in the secondary belt crack transformation image is located in the belt zone; If the crack in the secondary belt crack transformation image is located in the belt area, the secondary belt crack transformation image is included in the training data set.

11. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are executed.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images