Conveyor belt damage detection method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a conveyor belt damage detection method, an electronic device and a storage medium, and relate to the field of belt detection. The method acquires a continuous image data stream output by a line array camera, extracts images in a sliding window manner, inputs a current frame image into a target detection model, judges whether a first damage detection box touches a preset boundary condition when the first damage detection box is detected from the current frame image, directly outputs the first damage detection box if it does not touch the preset boundary condition, and acquires a second damage detection box that meets a preset spatial proximity condition with the first damage detection box in a next frame image if it touches the preset boundary condition. Feature similarity matching is performed on the first damage detection box and the second damage detection box, the first damage detection box and the second damage detection box are merged by set union if the matching is successful, and a complete detection box after cross-frame splicing is output. Thus, the visual continuity of cross-frame targets is ensured, and the missed detection rate and false positive rate of long strip-shaped damage are effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of belt inspection, and more specifically, to a method for detecting damage to conveyor belts, an electronic device, and a storage medium. Background Technology

[0002] In industrial vision inspection systems that use continuous acquisition by online array cameras, existing conventional solutions typically employ a technical architecture of "hard line frequency triggered slicing + 2D image target detection + encoder mileage mapping".

[0003] However, since there is no overlapping area between adjacent image frames and the cutting boundary depends entirely on the mechanical line frequency (i.e., a fixed physical time interval), when the length of a long strip-shaped damage (such as a longitudinal tear) extending along the belt running direction is greater than the field of view height of a single frame image, or happens to be distributed at the cutting seam between two adjacent frames, the continuity of the target in physical space will be forcibly broken by the hard cutting command.

[0004] This segmentation behavior will directly cause the segmented independent sub-targets to lose their original elongated semantic structure in a single frame image, making them very easy for target detection models (such as deep learning neural networks) to judge as multiple independent short anomalies or background noise, severely reducing the model's detection rate for elongated targets. Summary of the Invention

[0005] In view of this, the object of the present invention is to provide a conveyor belt damage detection method, electronic device and storage medium to at least partially improve the above-mentioned problems.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide a method for detecting damage to a conveyor belt, comprising: The system acquires a continuous image data stream output from a line scan camera and extracts images using a sliding window method; where adjacent images have overlapping regions. The current frame image is input into the target detection model. When a first damage detection box is detected from the current frame image, it is determined whether the first damage detection box touches a preset boundary condition. If not touched, the first damage detection box is output directly; If the first damage detection box is touched, a second damage detection box that satisfies the preset spatial proximity condition with the first damage detection box is obtained in the next frame image, and feature similarity matching is performed between the first damage detection box and the second damage detection box. If a match is successful, the first damage detection box and the second damage detection box are merged into a union to output a complete detection box after cross-frame stitching.

[0007] Optionally, the preset boundary condition is that the lower edge of the first damage detection box satisfies:

[0008] in, The vertical pixel coordinates of the center point of the first damage detection box are: The height of the first damage detection frame. The height of the current frame image. For scale-adaptive lower boundary approximation threshold:

[0009] in, The defect scale perception coefficient ensures that the larger the target, the higher the trigger threshold. This refers to the system's static safety tolerance. The height of the overlapping region. Limit the maximum utilization of overlapping areas to prevent threshold overflow.

[0010] Optionally, the step of obtaining a second damage detection box in the next frame of the image that satisfies a preset spatial proximity condition with the first damage detection box, and performing feature similarity matching between the first damage detection box and the second damage detection box, includes: The next frame image is input into the target detection model. When at least one damage detection box is detected from the next frame image, the spatial distance between the first damage detection box and each of the damage detection boxes is calculated respectively, and the damage detection box with the smallest spatial distance is taken as the candidate detection box. Determine whether the spatial distance between the candidate detection box and the first damage detection box is less than or equal to the height of the overlapping region; If not, the first damage detection box is output directly; If so, the candidate detection box is determined to be the second damage detection box, and the first damage detection box and the second damage detection box are respectively input into the feature extraction network to obtain the first image features and the second image features; Similarity matching is performed between the first image features and the second image features.

[0011] Optionally, the similarity matching between the first image features and the second image features includes: Calculate the squared Euclidean distance between the first image feature and the second image feature; If the squared Euclidean distance is less than a preset distance threshold, then the first damage detection box and the second damage detection box are determined to be the same physical damage.

[0012] Optionally, the method further includes: Obtain the absolute physical mileage corresponding to each of the first damage detection boxes that have been output; Based on the absolute physical mileage, the distance between each of the first damage detection frames is calculated in the global physical space of the conveyor belt; If the distance between any two first damage detection boxes is less than a preset suppression threshold, then the two first damage detection boxes are deduplicated.

[0013] Optionally, the step of obtaining the absolute physical mileage includes: Obtain the cumulative number of encoder pulses and pulse equivalent, and calculate the theoretical physical mileage corresponding to the current scan line; The vertical pixel center point of the first damage detection box is obtained, and the physical offset of the first damage detection box relative to the current scan line is calculated by combining the spatial resolution of the line scan camera in the direction of the conveyor belt running. The absolute physical mileage corresponding to the first damage detection frame is obtained by adding the theoretical physical mileage and the physical offset, and then taking the modulus of the total physical circumference of the conveyor belt.

[0014] Optionally, the method further includes: During the first loop of the conveyor belt or within a preset time period, at least one visual anchor point on the surface of the conveyor belt is identified, and the reference image features of each visual anchor point and its corresponding reference physical absolute mileage in the conveyor belt are obtained respectively. During the operation of the conveyor belt, the current theoretical mileage is calculated in real time, and the target visual anchor point that will soon arrive at the current field of view of the line scan camera is predicted based on the positive distance between the current theoretical mileage and the reference physical absolute mileage corresponding to each visual anchor point. When the shortest circular distance between the target visual anchor point and the current field of view is less than a preset wake-up threshold, the normalized features of the target visual anchor point in the current field of view are obtained. The normalized features are matched with the features of each of the reference images; If the match is successful, calculate the deviation between the baseline physical absolute mileage corresponding to the target visual anchor point and the current theoretical mileage; The encoder's cumulative mileage is corrected based on the deviation.

[0015] Optionally, calculating the deviation between the baseline physical absolute mileage corresponding to the target visual anchor point and the current theoretical mileage includes: Calculate the difference between the theoretical mileage calculated by the current encoder and the reference physical absolute mileage corresponding to the visual anchor point, and use it as the initial deviation value; If the initial deviation value is greater than half of the total physical circumference of the conveyor belt, then the initial deviation value is subtracted from the total physical circumference of the conveyor belt to obtain the final deviation. If the initial deviation value is less than half of the negative physical total circumference of the conveyor belt, then the initial deviation value is added to the physical total circumference of the conveyor belt to obtain the final deviation.

[0016] In a second aspect, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method described in any of the above-mentioned embodiments.

[0017] Thirdly, embodiments of the present invention provide a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any of the above-mentioned embodiments.

[0018] This invention provides a conveyor belt damage detection method, electronic device, and storage medium. By extracting images using a sliding window approach and preserving overlapping areas between adjacent image frames, it ensures that any damage target extending along the conveyor belt's running direction appears completely in at least one frame (if the target size exceeds the field of view of a single frame, its parts fall into the overlapping areas of adjacent frames). This fundamentally avoids the feature truncation problem caused by the traditional hard slicing method where the cutting boundary is located exactly in the middle of the damage target. When the detection box touches a preset boundary condition, a matching target is searched in the neighboring region of the next frame image. After a successful match, the two frame detection boxes are merged, allowing the same damage fragments that were originally placed in adjacent frames due to slicing to be spatially reconnected, forming a complete cross-frame damage bounding box. This ensures the visual continuity and geometric integrity of cross-frame damage, effectively reducing the false alarm and missed detection rates of long strip damage.

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic structural block diagram of an electronic device provided in an embodiment of the present invention; Figure 2 A schematic flowchart of a conveyor belt damage detection method provided in an embodiment of the present invention; Figure 3 A flowchart illustrating step S24 provided in an embodiment of the present invention; Figure 4 This is another schematic flowchart of a conveyor belt damage detection method provided in an embodiment of the present invention; Figure 5 This is another schematic diagram of a conveyor belt damage detection method provided in an embodiment of the present invention.

[0022] Icons: 100 - Electronic device; 101 - Memory; 102 - Communication interface; 103 - Processor; 104 - Communication bus. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] In industrial vision inspection systems that rely on continuous acquisition by linear scan cameras, existing conventional solutions typically employ a technical architecture of "hard line frequency triggered slicing + 2D image target detection + encoder odometer mapping". Specifically, the system uses a fixed line frequency (i.e., a fixed physical time interval) to non-overlap, equidistantly slice the continuous line array data stream output by the linear scan camera at a preset fixed window height H, forming several independent 2D image frames. Each frame is then fed into a target detection model for damage identification. Finally, based on the cumulative encoder pulse count, the pixel coordinates of the identified target in the image are converted into physical spatial odometer distance.

[0028] However, when the aforementioned conventional architecture is applied to the detection of long, strip-shaped targets such as longitudinal tears in conveyor belts, the lack of overlap between adjacent image frames and the fact that the cutting boundaries depend entirely on the mechanical travel frequency mean that when the length of the long strip-shaped damage (such as a longitudinal tear) extending along the belt's running direction exceeds the field of view height of a single frame, the damage will inevitably be divided into multiple independent segments by several cutting boundaries. Even if the damage length is less than the field of view height of a single frame, if it happens to be located at the cutting seam between two adjacent frames, it will still be divided in two by the cutting boundaries. In these cases, the spatial continuity of the same physical damage is forcibly severed by the rigid cutting command, causing its image features to lose their integrity within a single frame.

[0029] Based on the above, embodiments of the present invention provide a conveyor belt damage detection method, electronic device, and storage medium, which can ensure the visual continuity and geometric integrity of cross-frame damage in conveyor belt damage detection, and effectively reduce the false alarm rate and false alarm rate of long strip damage.

[0030] To implement the process steps and functions of this invention, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic structural block diagram of an electronic device 100 provided in an embodiment of the present invention. The electronic device 100 includes a memory 101 and a processor 103, which are electrically connected directly or indirectly to each other to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses 104 or signal lines. The memory 101 can be used to store software programs and modules, and the processor 103 executes the software programs and modules stored in the memory 101, thereby performing various functional applications and data processing.

[0031] Electronic device 100 can be, but is not limited to, a personal computer (PC), a server, a distributed computer, etc. It is understood that electronic device 100 is not limited to a physical server, but can also be a virtual machine on a physical server, a virtual machine built on a cloud platform, or any other computer that can provide the same functionality as the server or virtual machine. The operating system of electronic device 100 can be, but is not limited to, Windows, Linux, etc.

[0032] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0033] The communication connection between the electronic device 100 and external devices is achieved through at least one communication interface 102 (which can be wired or wireless).

[0034] Processor 103 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of this embodiment can be completed by integrated logic circuits in the hardware of processor 103 or by instructions in software form. Processor 103 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0035] Understandable. Figure 1 The structure shown is for illustrative purposes only; the electronic device 100 may also include components that are more advanced than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0036] The following is an exemplary description of the conveyor belt damage detection method provided in an embodiment of the present invention. See also... Figure 2 The subject executing this method can be one of the above. Figure 1 The electronic device 100 shown, the method includes as follows Figure 2 The following steps are described: S21: Acquire the continuous image data stream output by the line scan camera and extract the images using a sliding window method; wherein there is an overlapping area between adjacent images.

[0037] The line scan camera continuously outputs a continuous linear data stream during the conveyor belt operation. This continuous image data stream is acquired, and image frames are extracted using a sliding window method. An overlapping region is maintained between adjacent image frames, and the height of this overlapping region is less than the height of the sliding window.

[0038] In this way, any long strip of damage extending along the direction of the conveyor belt, even if its length exceeds the field of view height of a single frame image, will fall into the overlapping area of ​​adjacent frames, thus avoiding the damage being completely cut off at the cutting boundary due to hard slicing, and providing a data basis for subsequent cross-frame stitching.

[0039] For example, the height of the base sliding window is set to... Pixel, the The starting row coordinates of the frame image window are To prevent long-distance damage from being truncated, a mandatory height must be maintained between adjacent windows. The overlapping redundant region. Then the first The starting coordinates of a frame image are defined as follows:

[0040] It should be noted that the height of the overlapping area between adjacent image frames can be flexibly set according to the actual detection accuracy requirements and system computing resources. The larger the height of the overlapping area, the higher the success rate of completely stitching the cross-frame target, but the data redundancy also increases accordingly. Those skilled in the art can make reasonable configurations based on specific factors such as the conveyor belt running speed and the size of the damage features, and the embodiments of the present invention do not impose specific limitations on this.

[0041] S22: Input the current frame image into the target detection model. When the first damage detection box is detected in the current frame image, determine whether the first damage detection box touches the preset boundary conditions.

[0042] The current frame image is input into a pre-trained object detection model (such as the YOLO series models). The model outputs predicted bounding boxes for all possible damage components in the current frame image. Each predicted bounding box contains the target's category information, confidence score, and location information (such as center point coordinates, width, and height). When at least one damage detection box is detected in the current frame image, for any first damage detection box, it is further determined whether the damage detection box touches a preset boundary condition.

[0043] The preset boundary conditions are used to determine whether the current damage detection box is at risk of being truncated across frames. When the damage detection box is close to or located at the bottom edge of the current frame image, it indicates that the damage may extend beyond the current frame image, i.e., there is a risk of being truncated across frames; conversely, when the detection box is far from the bottom edge of the image, it indicates that the damage is fully imaged within the current frame, and there is no risk of being truncated across frames.

[0044] S23: If not touched, output the first damage detection box directly.

[0045] If the first damage detection bounding box does not touch the preset boundary conditions, it means that the damage is completely imaged within the current frame, and its overall shape and semantic structure have not been destroyed by the cutting boundary. In this case, the position information and category information corresponding to the detection bounding box are directly output as the detection result of the damage. Subsequently, its absolute position in the global physical coordinate system of the conveyor belt can be further calculated based on the position information of the detection bounding box.

[0046] S24: If touched, a second damage detection box that satisfies the preset spatial proximity condition with the first damage detection box is obtained in the next frame image, and feature similarity matching is performed between the first damage detection box and the second damage detection box.

[0047] If the first damage detection bounding box touches a preset boundary condition, it indicates that the damage may extend beyond the current frame image, posing a risk of cross-frame truncation. In this case, the detection bounding box is not output directly, but the system waits for the detection results of the next frame image.

[0048] After acquiring the next frame image and completing target detection, candidate detection boxes that meet the preset spatial proximity condition with the first damage detection box are selected in the next frame image and used as the second damage detection box.

[0049] The spatial proximity condition is used to narrow the matching search range. Since there is only a sliding step displacement between adjacent frames, the positional offset of the same physical damage in adjacent frames is limited. Therefore, the search is only performed in a specific spatial range of the next frame image (e.g., the area near the bottom of the detection box in the previous frame), which can quickly locate the possible corresponding target and avoid the waste of computing power caused by global search.

[0050] After acquiring the second damage detection box, the image features corresponding to the first and second damage detection boxes are extracted, and their similarity is calculated. The purpose of similarity matching is to confirm whether the two detection boxes correspond to the same physical damage. Since the acquisition time interval between adjacent frames is extremely short, and environmental factors such as lighting and dust remain almost unchanged, the image features of the same physical damage in adjacent frames are highly similar. Therefore, feature similarity matching can accurately determine whether two boxes belong to the same target. If the feature similarity is higher than a preset threshold, they are determined to be visual truncations of the same physical damage in adjacent slices; if it is lower than the preset threshold, they are determined to be different damage targets. In this case, the first and second damage detection boxes should be output independently.

[0051] S25: If the match is successful, the first damage detection box and the second damage detection box are merged into a union and the complete detection box after cross-frame stitching is output.

[0052] If the feature similarity of the first and second damage detection boxes is successful, it indicates that they correspond to the same physical damage. Since the first damage detection box is located in the bottom region of the previous frame and the second damage detection box is located in the top region of the next frame, they are physically adjacent and partially overlap or contiguous. Projecting both boxes onto the same global pixel coordinate system and performing union fusion, the bounding rectangle of both boxes is taken as the complete detection box output after cross-frame stitching.

[0053] In scenarios where matching fails, if the feature similarity between the first and second damage detection boxes fails to match, it is determined that they are not the same physical damage. If the second damage detection box is detected in the next frame, the first and second damage detection boxes are output independently, each serving as an independent damage alarm.

[0054] It should be noted that for the complete detection box output after successful matching of adjacent frames, if the bottom of the complete detection box touches the preset boundary condition again during subsequent continuous detection, a new round of cross-frame matching and stitching will be triggered. That is to say, for ultra-long damage (such as long strip-shaped vertical tears spanning three or more frames), the stitching process of S22 to S25 above will be executed sequentially between adjacent frames. After stitching the i-th frame with the i+1-th frame, the stitching result will be stitched with the i+2-th frame again, and so on, until all fragments of the ultra-long damage are stitched together to form the final complete damage detection box output. Through this iterative stitching mechanism, even if the damage length far exceeds the field of view height of a single frame, it can still be completely restored through continuous cross-frame stitching.

[0055] For the first and second damage detection boxes that are output independently after a matching failure, physical coordinate mapping can be performed on each box based on its location information before output, so as to determine its spatial relationship with other detection results later. The mapping method is the same as described above, that is, converting the encoder's cumulative pulse count and camera resolution into absolute physical mileage.

[0056] Furthermore, it should be noted that steps S21 to S25 above are exemplary descriptions of a single first damage detection box in the current frame image. In practical applications, multiple damage detection boxes may exist simultaneously in the current frame image (e.g., multiple damages at different locations and of different categories). The judgment and processing flow of S22 to S25 above is executed for each damage detection box detected in the current frame image to ensure that each damage in the current frame is correctly processed. For detection boxes in the current frame that do not touch the boundary conditions, they are directly output; for detection boxes that touch the boundary conditions, their features are temporarily stored and await matching and stitching in the next frame.

[0057] In one alternative implementation, when the target detection model is at the... Output the predicted bounding box of a damaged target within the frame window. When (representing the x-coordinate, y-coordinate, width, and height of the center point respectively), in order to take into account the truncation sensitivity at the boundary for defects of different scales (such as fine cracks and long tears), a scale-adaptive lower boundary approximation threshold is dynamically calculated for the target. The adaptive threshold is defined as being related to the current defect prediction height. and the height of the overlapping area Strongly correlated piecewise constraint functions:

[0058] in, The defect scale perception coefficient ensures that the larger the target, the higher the trigger threshold. This refers to the system's static safety tolerance. The height of the overlapping region. Limit the maximum utilization of overlapping areas to prevent threshold overflow.

[0059] The preset boundary conditions can be satisfied by the lower edge of the first damage detection box:

[0060] in, The vertical pixel coordinates of the center point of the first damage detection bounding box. The height of the first damage detection frame. The height of the current frame image. The lower boundary approximation threshold is scale-adaptive.

[0061] Since the displacement between adjacent frames is determined by the step distance of the sliding window (i.e., the difference between the window height and the overlap area height), the spatial offset of the same physical damage in adjacent frames is finite, and the upper limit of this offset is the height of the overlap area. Therefore, candidate targets can be quickly located by spatial distance filtering, avoiding feature matching of all detection boxes in the next frame image, thus significantly reducing the matching computation. For this purpose, in one optional implementation, see [link to implementation details]. Figure 3 Step S24 may include the following sub-steps: S241: Input the next frame image into the target detection model. When at least one damage detection box is detected in the next frame image, calculate the spatial distance between the first damage detection box and each damage detection box respectively, and take the damage detection box with the smallest spatial distance as the candidate detection box.

[0062] Spatial distance can refer to the pixel distance between two detection boxes along the conveyor belt's running direction (i.e., the vertical axis of the image). Since the displacement between adjacent frames only exists in the vertical direction, the offset in the horizontal direction is negligible. Therefore, calculating only the distance in the vertical direction is sufficient to meet the filtering requirements. This spatial distance can be calculated by using the absolute value of the difference between the bottom vertical coordinate of the first damage detection box and the top vertical coordinate of each detection box in the next frame as the distance metric.

[0063] S242: Determine whether the spatial distance between the candidate detection box and the first damage detection box is less than or equal to the height of the overlapping area.

[0064] Because the height of the overlapping area between adjacent frames is The positional offset of the same physical damage in adjacent frames cannot exceed the height of the overlapping region. If the distance between the candidate detection box and the first damage detection box is greater than the height of the overlapping region, it means that the two are neither overlapping nor adjacent in spatial location, and cannot correspond to the same physical damage. Therefore, using the height of the overlapping region as the threshold for determining spatial proximity has clear physical meaning and engineering rationality.

[0065] S243: If not, output the first damage detection box directly.

[0066] If the spatial distance between the candidate detection box and the first damage detection box is greater than the height of the overlapping area, it is determined that there is no physical damage corresponding to the first damage detection box in the next frame. In this case, the first damage detection box has no matching target in the next frame, its cross-frame stitching fails, and the first damage detection box is directly output as an independent damage alarm.

[0067] S244: If so, the candidate detection box is determined to be the second damage detection box. The first damage detection box and the second damage detection box are respectively input into the feature extraction network to obtain the first image features and the second image features.

[0068] If the spatial distance between the candidate detection box and the first damage detection box is less than or equal to the height of the overlapping area, the candidate detection box is determined to meet the spatial proximity condition and is used as the second damage detection box.

[0069] Subsequently, the first and second damage detection boxes are input into the feature extraction network to obtain the corresponding first and second image features.

[0070] Feature extraction networks can employ pre-trained deep convolutional neural networks (such as lightweight networks like ResNet-18 and MobileNet) to map input image regions into fixed-dimensional feature vectors. These feature vectors can characterize the visual semantic information of damaged regions, including texture, edges, and shape, providing a data foundation for subsequent similarity matching.

[0071] S245: Perform similarity matching between the first image features and the second image features.

[0072] Similarity matching is performed on the first image features and the second image features to confirm whether they correspond to the same physical damage. Specific similarity matching methods may include, but are not limited to, cosine similarity, Euclidean distance, etc. Those skilled in the art can choose an appropriate similarity measurement method according to actual needs; this embodiment of the invention does not specifically limit this method.

[0073] In one optional implementation, since the acquisition time interval between two adjacent image frames is extremely short, the image environment (such as lighting, dust, etc.) hardly changes, and the visual appearance of the same physical damage remains consistent in adjacent frames, the measurement method with the lowest computational cost among various feature similarity measurement methods can meet the requirement of accurate matching. Therefore, step S245 may include: S2451: Calculate the squared Euclidean distance between the first image feature and the second image feature.

[0074] The calculation formula is as follows:

[0075] in, Indicates the first The first frame feature vector dimensional components, Indicates the first The first frame feature vector Dimensional components.

[0076] S2452: If the squared Euclidean distance is less than the preset distance threshold, then the first damage detection box and the second damage detection box are determined to be the same physical damage.

[0077] If the squared Euclidean distance is less than the preset distance threshold If the first damage detection box and the second damage detection box correspond to the same physical damage, then the subsequent union fusion and cross-frame splicing output will be performed.

[0078] In the cross-frame stitching process, the second damage detection box is first translated by coordinates, and a coordinate translation transformation matrix is ​​set. ,matrix The translation parameters are determined by the base window height. With overlap area height The rigidity of the difference ensures absolute physical alignment of the coordinate systems of the preceding and following frames. Due to the use of a sliding window mechanism with a fixed overlap rate, the first... Frame relative to the first The actual physical distance a frame travels in the running direction is a known constant, namely the base window height. With overlap area height The difference. Therefore, the translation transformation matrix. Without relying on complex visual feature alignment, a rigid scalar offset of the ordinate based solely on the difference is sufficient to achieve the desired result, much like stitching together images. Figure 1 In this way, the target that breaks across frames is perfectly restored in the same global coordinate system, ensuring the absolute accuracy of the actual physical size.

[0079] Projecting both frames onto the same coordinate system and performing a union fusion, the complete global boundary box of the damage spanning both frames is calculated. :

[0080] However, overlapping sampling mechanisms may cause the same physical damage to be captured repeatedly in the overlapping area of ​​adjacent frames, resulting in a large number of coordinate ghosting and duplicate alarms. Related technologies rely solely on traditional intra-image non-maximum suppression using two-dimensional image coordinates, or simply set time thresholds for filtering. This cannot effectively distinguish whether consecutive alarms are due to real dense damage or repeated exposure of the same target, easily leading to a large number of redundant false alarms and significantly increasing the cost of manual review.

[0081] Therefore, in one alternative implementation, see [link to implementation details]. Figure 4 The method may also include: S31: Obtain the absolute physical mileage corresponding to each of the output first damage detection boxes.

[0082] The output damage detection boxes can include: the first damage detection boxes that were directly output in step S23 without touching the boundary conditions, and the complete detection boxes that were output after cross-frame stitching in step S25. The absolute physical mileage of each output damage detection box in the global physical coordinate system of the conveyor belt is calculated.

[0083] The steps to obtain absolute physical mileage may include: S311: Obtain the cumulative number of encoder pulses and pulse equivalent, and calculate the theoretical physical mileage corresponding to the current scan line.

[0084] Real-time reading of encoder cumulative pulse count and pulse equivalent (Unit: mm / pulse) Calculate the theoretical physical mileage corresponding to the current scan line (i.e., the latest line of image data currently acquired by the camera). Since the conveyor belt is a ring-shaped structure with a total physical circumference of L, a ring-shaped modulo mapping is performed on the theoretical physical mileage:

[0085] Should This indicates the theoretical absolute position of the current scan line in the global physical circular coordinate system of the conveyor belt.

[0086] S312: Obtain the vertical pixel center point of the first damage detection box, and calculate the physical offset of the first damage detection box relative to the current scan line by combining the spatial resolution of the line scan camera in the direction of the conveyor belt running.

[0087] Obtain the vertical coordinate pixels of the center point of the damage detection box. (That is, the pixel row number of the center point of the detection box along the vertical axis of the image). Since the vertical axis of the image is in the same direction as the conveyor belt, the larger the h is, the farther the pixel is from the current scan line (the latest row), that is, the damage is located in front of the current scan line in the direction of belt movement.

[0088] Combining the spatial resolution of the line scan camera in the conveyor belt running direction (Y-axis) (Unit: mm / pixel), calculate the physical offset of the damage detection box relative to the current scan line:

[0089] This offset represents the physical distance from the current scan line position along the conveyor belt running direction to the center point of the damage detection frame.

[0090] S313: Add the theoretical physical mileage and the physical offset, and then take the modulus of the total physical circumference of the conveyor belt to obtain the absolute physical mileage corresponding to the first damage detection frame.

[0091] The theoretical physical mileage of the current scan line obtained in step S311 Add the physical offset obtained in step S312, and take the modulus of the total physical circumference L of the conveyor belt to obtain the absolute physical mileage corresponding to the damage detection frame. :

[0092] Using the above formula, the two-dimensional pixel coordinates of any damage detection box are rigidly mapped to a unique absolute mileage value in the global physical circular coordinate system of the conveyor belt.

[0093] S32: Calculate the distance between each first damage detection frame in the global physical space of the conveyor belt based on the absolute physical mileage.

[0094] For any two output damage detection boxes, let their corresponding absolute physical mileages be respectively... and The physical circumference of the conveyor belt is L. Since the conveyor belt has a ring-shaped structure, the shortest actual spatial distance between two absolute physical mileages needs to be calculated along the ring topology:

[0095] The physical meaning of the above formula is as follows: two points can be connected by two paths in a circular coordinate system: direct distance (along one direction) and detour distance (along the other direction). The shorter of these two paths is taken as the actual spatial distance between the two points on the conveyor belt. By calculating this shortest circular distance, a unique spatial distance metric can be obtained between any two damage detection boxes, independent of the conveyor belt speed and the detection time sequence.

[0096] S33: If the distance between any two first damage detection boxes is less than the preset suppression threshold, then the two first damage detection boxes are deduplicated.

[0097] Set the preset suppression threshold. (For example, 0.2 meters; the specific value can be flexibly set according to factors such as the conveyor belt speed and detection accuracy requirements. This embodiment of the invention does not impose a specific limitation on this.) For any two damage detection frames, if their shortest distance in the global physical space of the conveyor belt is... Less than the preset suppression threshold If the two detection boxes correspond to the same physical damage, then duplicate detection is performed on both.

[0098] Optionally, the criteria for deduplication can be further limited to: the spatial distance between any two damage detection boxes. Furthermore, both damage categories must be identical. Only when both of these conditions are met is it determined that the two detection boxes correspond to the same physical damage and deduplication is performed.

[0099] The deduplication process can be done by comparing the confidence scores of two damage detection boxes, keeping only the detection box with the highest confidence score, and discarding the other.

[0100] However, obtaining the absolute physical mileage depends on the encoder's accumulated pulse count and pulse equivalent. During conveyor belt operation, slippage between the drive pulley and the belt, as well as encoder pulse loss, inevitably occur, leading to a cumulative error between the calculated theoretical mileage and the actual physical mileage. The longer the operating time, the greater the error accumulation, eventually causing a significant deviation between the system's output damage location and the actual location, making it impossible for maintenance personnel to locate the damage point based on the alarm coordinates.

[0101] In related technologies, positioning and calibration are typically achieved by embedding RFID chips, magnetic media, or attaching manual markers to the conveyor belt surface. However, these methods require embedding additional physical hardware into the conveyor belt itself, which not only damages the belt's tensile structure and increases the safety risk of belt breakage, but also makes physical modification difficult, costly, and time-consuming for existing conveyor belts already in operation, thus limiting the large-scale application of the solution.

[0102] Therefore, in one alternative implementation, see [link to implementation details]. Figure 5 The method may also include: S41: During the first loop of the conveyor belt or within a preset time period, identify at least one visual anchor point on the surface of the conveyor belt, and obtain the reference image features of each visual anchor point and its corresponding reference physical absolute mileage in the conveyor belt.

[0103] During the first loop of the conveyor belt (or within a preset time period as needed, such as the recalibration phase after equipment maintenance), visual anchor points on the surface of the conveyor belt are identified sequentially along the direction of belt operation.

[0104] Among them, visual anchor points are physical marks inherent to the surface of the conveyor belt that have unique visual recognizability.

[0105] In one alternative implementation, the visual anchor point can be a vulcanized joint of the conveyor belt: during installation, the conveyor belt is typically connected at both ends using a vulcanization process, forming a joint area with a specific texture, pattern, or shape. This area has visually distinctive features that differentiate it from other surfaces of the conveyor belt, and it appears periodically along the length of the conveyor belt. Its physical location is fixed and unique in the global physical coordinate system of the conveyor belt. In another alternative implementation, the visual anchor point can also be a natural or artificial mark (e.g., QR code, color block, reflective sticker, etc.) pre-sprayed or pasted onto the surface of the conveyor belt. Those skilled in the art can flexibly choose the specific form of the visual anchor point according to the actual conveyor belt type and site conditions; this embodiment of the invention does not impose specific limitations on this.

[0106] Image sequences of regions of interest for each visual anchor point are extracted using an object detection network. The anchor point images are input into a pre-trained feature extraction network to extract the baseline image features for each anchor point. These baseline image features are then L2-norm normalized to ensure that the magnitude of each baseline image feature vector is 1. (Baseline image features) Represented as:

[0107] in, This is a feature extraction network.

[0108] Synchronous latch underlying encoder accumulated pulses and pulse equivalent Calculate the basic physical absolute mileage at this time. :

[0109] Build in memory by An absolute physical reference dictionary composed of anchor points:

[0110] S42: During the operation of the conveyor belt, the current theoretical mileage is calculated in real time, and the target visual anchor point that will soon arrive at the current field of view of the line scan camera is predicted based on the positive distance between the current theoretical mileage and the reference physical absolute mileage corresponding to each visual anchor point.

[0111] After the anchor point benchmark library is established, the system enters the routine monitoring and operation phase. During this phase, the energy-intensive visual anchor point matching process is dormant, with only the low-computing-power background one-dimensional spatial prediction module running continuously.

[0112] The prediction module reads the encoder's cumulative pulse count and pulse equivalent in real time to calculate the current theoretical mileage. Then, the anchor point benchmark library is traversed, and for any visual anchor point k, the current theoretical mileage is calculated. The reference physical absolute mileage corresponding to this anchor point The positive distance between them along the direction of conveyor belt operation:

[0113] The physical meaning of the above formula is: the physical distance required to walk from the current position to the anchor point along the normal operating direction (positive direction) of the conveyor belt. (Take all...) The anchor point with the smallest value is taken as the target visual anchor point, which is the next visual anchor point to be reached along the running direction.

[0114] Through the aforementioned prediction mechanism, it is possible to predict at any time when the next visual anchor is expected to arrive in the current field of vision, without having to run a high-energy-consuming visual matching process.

[0115] S43: When the shortest circular distance between the target visual anchor point and the current field of view is less than the preset wake-up threshold, obtain the normalized features of the target visual anchor point in the current field of view.

[0116] Calculate the theoretical mileage estimated by the current encoder. Physical absolute mileage of the target anchor point The shortest distance in the ring between :

[0117] Set preset wake-up threshold (For example, 2.0 meters. This value can be flexibly set according to factors such as conveyor belt speed and response delay; this embodiment of the invention does not impose a specific limitation on it.) Comparison and :

[0118] when At that time, the feature matching process is activated to obtain the normalized features of the target visual anchor point in the current field of view.

[0119] S44: Match the normalized features with the features of each reference image.

[0120] The normalized features obtained in step S43 are matched and verified with the features of each reference image in the anchor point reference library established in step S41 to confirm which anchor point in the reference library corresponds to the visual anchor point in the current field of view.

[0121] In one alternative implementation, the matching verification is achieved by calculating the cosine similarity between the normalized features and the features of each reference image, as shown in the following formula:

[0122] Matching rules such as: finding the most similar match. And set strict admission thresholds. :

[0123] S45: If the match is successful, calculate the deviation between the baseline physical absolute mileage corresponding to the target visual anchor point and the current theoretical mileage.

[0124] When the matching verification in step S44 is successful, it has been confirmed that the visual anchor point in the current field of view is the k-th anchor point in the anchor point reference library, and its corresponding reference physical absolute mileage is... At this point, the theoretical mileage estimated by the current encoder is calculated. Compared with the physical absolute mileage of the anchor point The deviation between them.

[0125] Optionally, S45 may include: S451: Calculate the difference between the theoretical mileage calculated by the current encoder and the reference physical absolute mileage corresponding to the visual anchor point, and use it as the initial deviation value.

[0126] Let the theoretical mileage calculated by the current encoder be... The reference physical absolute mileage corresponding to the visual anchor point is The initial deviation value is:

[0127] The initial deviation value may be positive (the encoder's estimated mileage is greater than the actual anchor point location, i.e., the encoder over-calculated), negative (the encoder's estimated mileage is less than the actual anchor point location, i.e., the encoder under-calculated), or zero.

[0128] S452: If the initial deviation value is greater than half of the physical circumference of the conveyor belt, then subtract the physical circumference of the conveyor belt from the initial deviation value to obtain the final deviation.

[0129] S453: If the initial deviation value is less than half of the negative physical total circumference of the conveyor belt, then add the initial deviation value to the physical total circumference of the conveyor belt to obtain the final deviation.

[0130] S452 and S453 constrain the initial deviation value to... Within the interval:

[0131] S46: Correct the encoder's cumulative mileage based on the deviation.

[0132] The final deviation value calculated in step S45 The feedback is sent to the encoder's cumulative pulse counter, where the following correction is performed:

[0133] in, To correct the theoretical mileage calculated by the previous encoder, This is the theoretical mileage calculated by the corrected encoder. The essence of this correction is to reset the accumulated errors caused by encoder slippage or pulse loss to zero, thus realigning the encoder's theoretical mileage with the actual physical mileage of the visual anchor point. After this correction, the absolute physical mileage calculations for all subsequent detection frames are based on the corrected encoder values, restoring positioning accuracy.

[0134] After the correction is completed, the feature matching process immediately releases computing resources and switches back to sleep mode, resuming low-power operation by maintaining only the one-dimensional spatial prediction module, until it is awakened again when approaching the next visual anchor point. Through the above-mentioned sleep, wake-up, correction, and sleep cycle mechanism, the computing power overhead caused by visual anchor point matching is minimized while ensuring long-term positioning accuracy.

[0135] When a match fails (e.g., the normalized features of the visual anchor point cannot be successfully matched with any baseline features due to reasons such as coal slime obstruction or sudden changes in light source), no correction operation is performed. The encoder's current cumulative mileage remains unchanged, and it continues to operate normally, waiting for the next round of wake-up and matching process.

[0136] In summary, the conveyor belt damage detection method, electronic device, and storage medium provided by this invention, through a sliding window sampling method that preserves overlapping areas between adjacent frames, combined with a cross-frame feature matching and union fusion mechanism when the target detection box touches the boundary, effectively avoids the hard cutting of long strip-shaped damages at the slice boundary, ensuring the visual continuity and complete detection rate of cross-frame targets, and fundamentally eliminating blind spots. Furthermore, by rigidly mapping the two-dimensional pixel coordinates of the damage detection box to the absolute physical mileage in the global physical circular coordinate system of the conveyor belt, and calculating the shortest distance between each damage detection box in the circular physical space, combined with damage category consistency... The system determines that duplicate detection boxes with a distance less than a preset suppression threshold are deduplicated, effectively solving the problem of duplicate alarms caused by the overlapping sampling mechanism and achieving deduplication at the physical space level. Furthermore, by using the joints or marks on the conveyor belt surface as visual anchor points, an anchor point reference library is established. Combined with a dynamic wake-up mechanism based on one-dimensional kinematic prediction, visual matching verification is triggered only when the anchor point is expected to reach the field of view. After successful matching, a closed-loop correction is performed on the encoder's cumulative error under a half-circumference constraint. Thus, without relying on additional physical hardware or damaging the belt structure, the long-term cumulative error of the encoder is periodically cleared, ensuring the long-term stability of positioning accuracy.

[0137] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0138] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0139] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0141] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for detecting damage to a conveyor belt, characterized in that, include: The system acquires a continuous image data stream output from a line scan camera and extracts images using a sliding window method; where adjacent images have overlapping regions. The current frame image is input into the target detection model. When a first damage detection box is detected from the current frame image, it is determined whether the first damage detection box touches a preset boundary condition. If not touched, the first damage detection box is output directly; If the first damage detection box is touched, a second damage detection box that satisfies the preset spatial proximity condition with the first damage detection box is obtained in the next frame image, and feature similarity matching is performed between the first damage detection box and the second damage detection box. If a match is successful, the first damage detection box and the second damage detection box are merged into a union to output a complete detection box after cross-frame stitching.

2. The method according to claim 1, characterized in that, The preset boundary condition is that the lower edge of the first damage detection box satisfies: in, The vertical pixel coordinates of the center point of the first damage detection box are: The height of the first damage detection frame. The height of the current frame image. For scale-adaptive lower boundary approximation threshold: in, The defect scale perception coefficient ensures that the larger the target, the higher the trigger threshold. This refers to the system's static safety tolerance. The height of the overlapping region. Limit the maximum utilization of overlapping areas to prevent threshold overflow.

3. The method according to claim 1, characterized in that, The step of obtaining a second damage detection box in the next frame that satisfies a preset spatial proximity condition with the first damage detection box, and performing feature similarity matching between the first damage detection box and the second damage detection box, includes: The next frame image is input into the target detection model. When at least one damage detection box is detected from the next frame image, the spatial distance between the first damage detection box and each of the damage detection boxes is calculated respectively, and the damage detection box with the smallest spatial distance is taken as the candidate detection box. Determine whether the spatial distance between the candidate detection box and the first damage detection box is less than or equal to the height of the overlapping region; If not, the first damage detection box is output directly; If so, the candidate detection box is determined to be the second damage detection box, and the first damage detection box and the second damage detection box are respectively input into the feature extraction network to obtain the first image features and the second image features; Similarity matching is performed between the first image features and the second image features.

4. The method according to claim 3, characterized in that, The similarity matching between the first image features and the second image features includes: Calculate the squared Euclidean distance between the first image feature and the second image feature; If the squared Euclidean distance is less than a preset distance threshold, then the first damage detection box and the second damage detection box are determined to be the same physical damage.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the absolute physical mileage corresponding to each of the first damage detection boxes that have been output; Based on the absolute physical mileage, the distance between each of the first damage detection frames is calculated in the global physical space of the conveyor belt; If the distance between any two first damage detection boxes is less than a preset suppression threshold, then the two first damage detection boxes are deduplicated.

6. The method according to claim 5, characterized in that, The steps for obtaining the absolute physical mileage include: Obtain the cumulative number of encoder pulses and pulse equivalent, and calculate the theoretical physical mileage corresponding to the current scan line; The vertical pixel center point of the first damage detection box is obtained, and the physical offset of the first damage detection box relative to the current scan line is calculated by combining the spatial resolution of the line scan camera in the direction of the conveyor belt running. The absolute physical mileage corresponding to the first damage detection frame is obtained by adding the theoretical physical mileage and the physical offset, and then taking the modulus of the total physical circumference of the conveyor belt.

7. The method according to claim 6, characterized in that, The method further includes: During the first loop of the conveyor belt or within a preset time period, at least one visual anchor point on the surface of the conveyor belt is identified, and the reference image features of each visual anchor point and its corresponding reference physical absolute mileage in the conveyor belt are obtained respectively. During the operation of the conveyor belt, the current theoretical mileage is calculated in real time, and the target visual anchor point that will soon arrive at the current field of view of the line scan camera is predicted based on the positive distance between the current theoretical mileage and the reference physical absolute mileage corresponding to each visual anchor point. When the shortest circular distance between the target visual anchor point and the current field of view is less than a preset wake-up threshold, the normalized features of the target visual anchor point in the current field of view are obtained. The normalized features are matched with the features of each of the reference images; If the match is successful, calculate the deviation between the baseline physical absolute mileage corresponding to the target visual anchor point and the current theoretical mileage; The encoder's cumulative mileage is corrected based on the deviation.

8. The method according to claim 7, characterized in that, The calculation of the deviation between the baseline physical absolute mileage corresponding to the target visual anchor point and the current theoretical mileage includes: Calculate the difference between the theoretical mileage calculated by the current encoder and the reference physical absolute mileage corresponding to the visual anchor point, and use it as the initial deviation value; If the initial deviation value is greater than half of the total physical circumference of the conveyor belt, then the initial deviation value is subtracted from the total physical circumference of the conveyor belt to obtain the final deviation. If the initial deviation value is less than half of the negative physical total circumference of the conveyor belt, then the initial deviation value is added to the physical total circumference of the conveyor belt to obtain the final deviation.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

Images