Conveyor belt edge defect detection method, device, computer equipment and storage medium

Abstract

The embodiment of the present invention discloses a conveyor belt edge defect detection method, device, computer equipment and storage medium. The method includes: obtaining a picture of the conveyor belt to obtain a picture to be detected; performing binary semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing the conveyor belt area and background; performing edge straight line detection on the semantic segmentation picture to obtain the parametric equation of the straight line to determine the conveyor belt edge reference plane; performing roller detection on the picture to be detected to obtain the upper edge roller and the lower edge roller; determining the roller reference plane; determining the degree of conveyor belt edge wear based on the roller reference plane, the conveyor belt edge reference plane and the conveyor belt area; and performing alarm processing based on the degree of conveyor belt edge wear. By implementing the method of the embodiment of the present invention, the problem of inaccurate parametric equation of the edge straight line caused by conveyor belt edge wear can be avoided, edge wear detection can be identified, and the degree of wear can be given, which greatly saves costs and simplifies deployment steps.

Description

Technical Field

[0001] The present invention relates to a conveyor belt, and more particularly to a conveyor belt edge defect detection method, device, computer equipment and storage medium. Background Art

[0002] Cargo conveyor belt edge wear detection is a very important part of port cargo transportation inspection. However, most current inspections rely too much on hardware devices such as sensors, which are often too expensive and relatively complicated to deploy.

[0003] Existing detection methods use photoelectric sensor modules, proprietary sensor modules, RFID modules, and a host computer. Photoelectric sensors detect damage and then transmit the information to the host computer. The advantages of this method are high accuracy and timeliness, but the addition of hardware greatly increases costs, and once a problem occurs in a piece of hardware, the entire system will have problems. Another detection method is to use a method for detecting conveyor belt wear based on a detection system. The detection system includes some conveyor belt structural hardware installation, a signal collector, and an alarm system, and uses voltage to analyze the roller speed value. This method has high requirements for the roller, and if a problem occurs with the roller, the system may generate a false alarm. The existing conveyor belt edge baseline uses Hough transform for straight line detection, but this method often causes unevenness due to wear on the conveyor belt edge, resulting in multiple edges being detected or some edges not being detected during conveyor belt edge detection.

[0004] Therefore, it is necessary to design a new method to avoid the problem of inaccurate parametric equations of edge lines caused by edge wear of the conveyor belt. It can identify edge wear detection and also give the degree of wear, which greatly saves costs and simplifies deployment steps. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a conveyor belt edge defect detection method, device, computer equipment and storage medium.

[0006] To achieve the above-mentioned purpose, the present invention adopts the following technical solution: a conveyor belt edge defect detection method, comprising:

[0007] Get a picture of the conveyor belt to get the picture to be detected;

[0008] Performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing the conveyor belt area and the background;

[0009] Performing edge line detection on the semantic segmentation image to obtain a parametric equation of the line to determine the edge reference plane of the conveyor belt;

[0010] Performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller;

[0011] Determine the roller reference surface according to the upper edge roller and the lower edge roller;

[0012] determining the degree of wear on the conveyor belt edge based on the roller reference surface, the conveyor belt edge reference surface, and the conveyor belt area;

[0013] An alarm is issued according to the degree of wear on the edge of the conveyor belt.

[0014] A further technical solution is: performing semantic segmentation on the image to be detected to obtain an intermediate semantic segmentation image containing the conveyor belt area and the background, including:

[0015] Performing semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image;

[0016] The segmented image is subjected to black-and-white binary mapping to obtain a semantic segmentation image containing the conveyor belt area and the background.

[0017] Its further technical solution is: the semantic segmentation network adopts the resnet50 model as the skeleton network, and the resnet50 model is formed by combining the context path and spatial path mechanism through the feature fusion module to perform feature fusion, and the loss function of the semantic segmentation network is changed to the Dice loss function, and the output of the attention refinement module before the feature fusion module is selected to add two features as the auxiliary loss function, and the dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network.

[0018] A further technical solution is: performing edge line detection on the semantic segmentation image to obtain a parametric equation of the line to determine the edge reference plane of the conveyor belt, including:

[0019] Performing feature extraction on the semantic segmentation image to obtain a feature map;

[0020] Performing a Hough transform on the feature map to obtain a changed feature map;

[0021] The depth information of each layer in the changed feature map is converted from the spatial domain to the parameter domain to obtain the parametric equation of the line;

[0022] The conveyor belt edge reference plane is determined based on the parametric equation of the straight line.

[0023] A further technical solution is: performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller, including:

[0024] The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained;

[0025] Calculate the center point coordinates of each roller according to the image coordinate position of each roller;

[0026] The distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

[0027] A further technical solution is as follows: determining the roller reference surface according to the upper edge roller and the lower edge roller includes:

[0028] Calculate the endpoint coordinates of the upper edge roller and the lower edge roller;

[0029] Determine an upper reference line and a lower reference line according to the endpoint coordinates;

[0030] The roller reference surface is determined based on the upper reference line and the lower reference line.

[0031] The present invention also provides a conveyor belt edge defect detection device, comprising:

[0032] An image acquisition unit, used to acquire an image of the conveyor belt to obtain an image to be detected;

[0033] a segmentation unit, configured to perform binary semantic segmentation on the image to be detected to obtain a semantically segmented image containing a conveyor belt area and a background;

[0034] a line detection unit, configured to perform edge line detection on the semantic segmentation image to obtain a parametric equation of the line, so as to determine a conveyor belt edge reference plane;

[0035] A roller detection unit, configured to perform roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller;

[0036] A roller reference plane determining unit, configured to determine a roller reference plane based on an upper edge roller and a lower edge roller;

[0037] a degree determination unit, configured to determine a degree of wear on the edge of the conveyor belt based on the roller reference surface, the conveyor belt edge reference surface, and the conveyor belt area;

[0038] An alarm processing unit is used to perform alarm processing according to the degree of wear on the edge of the conveyor belt.

[0039] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0040] The present invention also provides a storage medium storing a computer program, which can implement the above method when executed by a processor.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention greatly saves costs and simplifies deployment steps by detecting edge defects in the form of pictures; performs semantic segmentation on pictures, edge straight line detection, roller detection, determination of roller and conveyor edge reference planes, and determination of the degree of wear on the conveyor edge; identifies edge wear based on deep learning, can identify edge wear detection, and can also give the degree of wear; performs semantic segmentation of conveyor belt parts based on the semantic segmentation network, and then performs Hough transform and transfomer feature extraction network fusion on the basis of the semantic segmentation results to perform straight line detection to obtain an accurate conveyor belt edge, thereby avoiding the problem of inaccurate parameter equation of the edge straight line caused by conveyor belt edge wear.

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic diagram of an application scenario of the conveyor belt edge defect detection method provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a process for detecting conveyor belt edge defects according to an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a sub-process of a conveyor belt edge defect detection method provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a sub-process of a conveyor belt edge defect detection method provided by an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of a sub-process of a conveyor belt edge defect detection method provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of a sub-process of a conveyor belt edge defect detection method provided by an embodiment of the present invention;

[0050] Figure 7A schematic block diagram of a conveyor belt edge defect detection device provided by an embodiment of the present invention;

[0051] Figure 8 A schematic block diagram of a segmentation unit of a conveyor belt edge defect detection device provided by an embodiment of the present invention;

[0052] Figure 9 A schematic block diagram of a linear detection unit of a conveyor belt edge defect detection device provided by an embodiment of the present invention;

[0053] Figure 10 A schematic block diagram of a roller detection unit of a conveyor belt edge defect detection device provided by an embodiment of the present invention;

[0054] Figure 11 A schematic block diagram of a roller reference plane determination unit of a conveyor belt edge defect detection device provided by an embodiment of the present invention;

[0055] Figure 12 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0058] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0059] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0060] See also Figure 1 and Figure 2 , Figure 1Schematic diagram of an application scenario of the conveyor belt edge defect detection method provided by an embodiment of the present invention. Figure 2 This is a schematic flow chart of a conveyor belt edge defect detection method provided by an embodiment of the present invention. This conveyor belt edge defect detection method is applied to a server. The server exchanges data with a terminal and a camera. The camera is used to capture images of the conveyor belt. The server then performs semantic segmentation, edge line detection, roller detection, roller reference surface detection, and edge wear detection to determine the degree of edge wear and initiate alarm processing.

[0061] Figure 2 FIG. 1 is a flow chart of a conveyor belt edge defect detection method according to an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S170.

[0062] S110: Acquire a picture of the conveyor belt to obtain a picture to be detected.

[0063] In this embodiment, the image to be detected refers to an image formed by collecting image data at a fixed position set on the cargo transport conveyor belt. During the collection process, the focal length and height of the pan-tilt camera are given fixed preset values.

[0064] Eliminating the reliance on hardware such as sensors, the wear and tear on the edge of cargo conveyor belts can be detected based solely on images, greatly saving costs and simplifying deployment steps.

[0065] S120 , performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background.

[0066] In this embodiment, the semantic segmentation image refers to an image including the conveyor belt area and the background.

[0067] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S122.

[0068] S121 . Perform semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image.

[0069] In this embodiment, the segmented images refer to two types of semantic segmentation images: conveyor belt mask and background mask.

[0070] Specifically, the semantic segmentation network adopts the resnet50 model as the skeleton network. The resnet50 model is formed by combining the context path and spatial path mechanisms through the feature fusion module to perform feature fusion, and the loss function of the semantic segmentation network is changed to the Dice loss function. The output of the attention refinement module before the feature fusion module is selected to add two features as the auxiliary loss function, and the dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network.

[0071] Specifically, the semantic segmentation of cargo conveyor belts uses the ResNet50 backbone network. This model combines the context path and spatial path mechanisms, and fuses features through the Feature Fusion module to produce a semantic segmentation network. The loss function used here is the Dice loss function l1. Two auxiliary loss functions, l2 and l3, are added to the output of the ARM (Attention Refinement Module) before Feature Fusion. These are then summed in a 1:1:1 ratio to form the final loss function: Loss = l1 + l2 + l3.

[0072] In order to ensure the accuracy of the conveyor belt mask prediction, the image to be detected is first used as the input of the network with its original width of 1920 and height of 1080 pixels. The output categories of the semantic segmentation network are two types of semantic segmentation maps: conveyor belt mask and background mask.

[0073] S122 , performing black-white binary mapping on the segmented image to obtain a semantic segmented image containing the conveyor belt area and the background.

[0074] When performing black and white binary mapping on the segmented image, the three channels of the pixels in the conveyor belt area are set to 255 pixels, and the three channels of the background area are set to 0 pixels, thereby forming a semantic segmentation image containing the conveyor belt area and the background. At the same time, the conveyor belt area is defined as s 传送带 .

[0075] S130 , performing edge line detection on the semantic segmentation image to obtain a parametric equation of the line to determine a conveyor belt edge reference plane.

[0076] In this embodiment, the parametric equation of the straight line refers to the parametric equation of the conveyor belt edge straight line. The conveyor belt edge reference plane refers to the area formed by the upper and lower edge straight lines of the conveyor belt.

[0077] Edge line detection is semantic segmentation line detection based on the Hough transform. Improvements have been made to the feature extraction process, using a transformer-based feature extraction method and the Feature Pyramid Network (FPN). The output feature map is then subjected to a Hough transform, simultaneously converting the spatial domain into a parametric domain. The final output is the parametric equation of the line, which is converted to the spatial domain.

[0078] In one embodiment, see Figure 4 , the above-mentioned step S130 may include steps S131 to S134.

[0079] S131. Perform feature extraction on the semantic segmentation image to obtain a feature map.

[0080] In this embodiment, the feature map refers to a picture composed of features of the upper and lower layers of the conveyor belt.

[0081] S132. Perform Hough transform on the feature map to obtain a changed feature map.

[0082] Line detection uses a combination of deep learning algorithms and traditional image algorithms. It extracts features based on transformers and FPN, converts the depth information of each layer from the spatial domain to the parameter domain, and performs Hough transform on it. The Hough transform output is then interpolated to adapt to the resolution of different features, and finally the features after interpolation are obtained. The loss function used here is Here g is the binary annotation map of the parameter domain, g i，j =1 refers to the straight line in the parameter domain i, j, k is a 5*5 Gaussian kernel, This is a convolution operation. When testing, the parameters in the parameter domain are directly converted to parameters in the spatial domain to calculate the linear coordinates.

[0083] S133. Convert the depth information of each layer in the changed feature map from the spatial domain to the parameter domain to obtain the parametric equation of the straight line.

[0084] S134. Determine the conveyor belt edge reference plane based on the parametric equation of the straight line.

[0085] Based on the obtained parametric equations of the upper and lower conveyor belt edge lines and the image size, the coordinates of the intersection points of the parametric equations of the two lines with the original image boundary are obtained. The conveyor belt edge reference surface S is calculated through the closed box surrounded by these intersection coordinates. 边缘 .

[0086] The main idea is to divide the polygon into multiple triangles, then add up the areas of all triangles, and calculate the area of ​​the triangle using the cross product of the vectors. in, is a vector, which can be obtained from the coordinates of points A and B; |AB| is The modulus of ∠BAC; sin(BAC) is the sin value of ∠BAC

[0087] The semantic segmentation of the conveyor belt parts is performed based on the semantic segmentation network. Then, based on the semantic segmentation results, the Hough transform and the transfomer feature extraction network are fused to perform straight line detection to obtain the accurate conveyor belt edge, thereby avoiding the problem of inaccurate parameter equation of the edge line caused by conveyor belt edge wear.

[0088] S140: Perform roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller.

[0089] In this embodiment, the rollers of the conveyor belt include an upper edge roller and a lower edge roller.

[0090] In one embodiment, see Figure 5 , the above-mentioned step S140 may include steps S141 to S143.

[0091] S141. Use the Yolov5 target detection model to locate the rollers in the image to be detected, and obtain the image coordinate position of each roller.

[0092] In this embodiment, the image coordinate position of each roller refers to the coordinates of the upper edge roller and the lower edge roller in the image.

[0093] Specifically, the rollers are positioned using the yolov5 target detection model to obtain the image coordinate position (x min ,y min , x max ,y max The YoloV5 object detection model was selected with a detection confidence of 0.6. To ensure accurate detection of the roller's tip coordinates, the YoloV5x architecture was used for detection. CIOU loss was used as the corresponding loss function for the regression box coordinates, combined with foreground-background cross entropy loss and classification cross entropy loss to form the overall loss function.

[0094] When detecting the roller, it is important to note that if a roller is located at the edge of the image, this may easily cause the roller to be incomplete, resulting in inaccurate endpoint values. Based on this situation, the position of the roller is judged separately. If any of the following conditions is met, the endpoint coordinates of the roller are not considered.

[0095] (1)x min <α;(2)width-x max <α;(3)y min <α;(4)height-y max <α; here width is the width of the image, height is the height of the image, and α is set to 10.

[0096] S142. Calculate the center point coordinates of each roller according to the image coordinate position of each roller.

[0097] In this embodiment, the center point coordinates of each roller frame are calculated separately.

[0098] S143, calculating the distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt to determine the upper edge roller and the lower edge roller.

[0099] In this embodiment, by calculating the center point coordinates (c x , c y ) and to the upper edge l 传送带 The distance d 上边缘 and to the lower edge l 传送带 The distance d 下边缘 If d 下边缘 <d 上边缘 , then the roller is determined to be the lower edge roller, otherwise it is called the upper edge roller.

[0100] S150: Determine a roller reference plane according to the upper edge roller and the lower edge roller.

[0101] In this embodiment, the roller reference surface refers to the area between two reference lines defined by the upper edge roller and the lower edge roller.

[0102] In one embodiment, see Figure 6 , the above-mentioned step S150 may include steps S151 to S153.

[0103] S151, calculating the endpoint coordinates of the upper edge roller and the lower edge roller;

[0104] S152, determining an upper reference line and a lower reference line according to the endpoint coordinates;

[0105] S153, determining the roller reference surface according to the upper reference line and the lower reference line.

[0106] If the slope of the conveyor belt is k 传送带 > 0, for the upper edge roller, select the upper right corner coordinate of each upper edge roller as its endpoint coordinate pi(k＞0，top) (x max ,y min ), the lower edge roller selects the coordinates of the upper left corner of each lower edge roller as its endpoint coordinates p i(k＞0，bottom) (x min ,y min ). If the slope k of the conveyor belt 传送带 <0, for the upper edge roller, the coordinates of the upper left corner of the roller are selected as its endpoint coordinates p i(k＜0，top )(x min ,y min ), the lower edge roller selects the coordinates of the upper right corner of the roller as its endpoint coordinates p i(k＜0，top) (x max ,y min ), and finally two baselines l are fitted through these endpoints. 基准 The roller reference surface s can be obtained by the upper and lower reference lines. 滚轴 .

[0107] Specifically, the polygon is divided into multiple triangles, and then the areas of all triangles are added together. The area of ​​the triangle is calculated using the cross product of the vectors. is a vector, which can be obtained from the coordinates of points A and B; |AB| is The modulus of ; sin(BAC) is the sin value of ∠BAC.

[0108] S160: Determine the degree of wear on the conveyor belt edge according to the roller reference surface, the conveyor belt edge reference surface, and the conveyor belt area.

[0109] In this embodiment, based on the roller reference surface s obtained above, 滚轴 , through the formula here It represents the exclusive OR operation. To evaluate the degree of wear on the edge of the conveyor belt, the larger the μ, the more serious the wear on the conveyor belt.

[0110] Edge wear can be identified based on deep learning. Edge wear detection can not only identify the wear, but also give the degree of wear, which can provide early warning of conveyor belt wear.

[0111] S170: Perform alarm processing according to the degree of wear on the edge of the conveyor belt.

[0112] When μ>λ, a wear alarm will be issued. Here, λ is set to 0.03. Of course, it can be set according to actual conditions.

[0113] The above-mentioned conveyor belt edge defect detection method greatly saves costs and simplifies deployment steps by detecting edge defects in the form of pictures; performs semantic segmentation on the pictures, edge straight line detection, roller detection, determination of the roller and conveyor belt edge reference plane, and determination of the degree of conveyor belt edge wear, and recognizes edge wear based on deep learning. It can also identify edge wear detection and give the degree of wear. Semantic segmentation of the conveyor belt parts is performed based on the semantic segmentation network, and then the Hough transform and the transfomer feature extraction network are fused on the basis of the semantic segmentation results to perform straight line detection to obtain the accurate conveyor belt edge, thereby avoiding the problem of inaccurate parameter equation of the edge straight line caused by conveyor belt edge wear.

[0114] Figure 7 FIG. 3 is a schematic block diagram of a conveyor belt edge defect detection device 300 provided by an embodiment of the present invention. Figure 7 As shown, corresponding to the above conveyor belt edge defect detection method, the present invention also provides a conveyor belt edge defect detection device 300. The conveyor belt edge defect detection device 300 includes a unit for executing the above conveyor belt edge defect detection method, and the device can be configured in a server. Specifically, please refer to Figure 7 The conveyor belt edge defect detection device 300 includes an image acquisition unit 301, a segmentation unit 302, a straight line detection unit 303, a roller detection unit 304, a roller reference plane determination unit 305, a degree determination unit 306 and an alarm processing unit 307.

[0115] The image acquisition unit 301 is used to acquire an image of the conveyor belt to obtain an image to be detected; the segmentation unit 302 is used to perform binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing the conveyor belt area and the background; the straight line detection unit 303 is used to perform edge straight line detection on the semantic segmentation image to obtain a parametric equation of the straight line to determine the conveyor belt edge reference plane; the roller detection unit 304 is used to perform roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller; the roller reference plane determination unit 305 is used to determine a reference plane based on the upper edge roller and the lower edge roller; the degree determination unit 306 is used to determine the degree of conveyor belt edge wear based on the roller reference plane, the conveyor belt edge reference plane and the conveyor belt area; the alarm processing unit 307 is used to perform alarm processing based on the degree of conveyor belt edge wear.

[0116] In one embodiment, if Figure 8 As shown, the segmentation unit 302 includes a semantic segmentation subunit 3021 and a mapping subunit 3022.

[0117] The semantic segmentation subunit 3021 is used to perform semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image; the mapping subunit 3022 is used to perform black and white binary mapping on the segmented image to obtain a semantic segmentation image containing the conveyor belt area and the background.

[0118] In one embodiment, if Figure 9 As shown, the straight line detection unit 303 includes a feature extraction subunit 3031, a Hough transformation subunit 3032, a conversion subunit 3033 and a conveyor belt edge reference plane determination unit 3034.

[0119] The feature extraction subunit 3031 is used to extract features from the semantic segmentation image to obtain a feature map; the Hough transform subunit 3032 is used to perform a Hough transform on the feature map to obtain a changed feature map; the conversion subunit 3033 is used to convert the depth information of each layer in the changed feature map from the spatial domain to the parameter domain to obtain the parametric equation of the straight line; the conveyor belt edge reference plane determination unit 3034 is used to determine the conveyor belt edge reference plane based on the parametric equation of the straight line.

[0120] In one embodiment, if Figure 10 As shown, the roller detection unit 304 includes a target positioning subunit 3041 , a center point coordinate calculation subunit 3042 , and a distance calculation subunit 3043 .

[0121] The target positioning subunit 3041 is used to use the yolov5 target detection model to locate the rollers in the image to be detected and obtain the image coordinate position of each roller; the center point coordinate calculation subunit 3042 is used to calculate the center point coordinates of each roller according to the image coordinate position of each roller; the distance calculation subunit 3043 is used to calculate the distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt to determine the upper edge roller and the lower edge roller.

[0122] In one embodiment, if Figure 11 As shown, the roller reference surface determination unit 305 includes an endpoint coordinate calculation subunit 3051 , a reference line determination subunit 3052 and a surface determination subunit 3053 .

[0123] The endpoint coordinate calculation subunit 3051 is used to calculate the endpoint coordinates of the upper edge roller and the lower edge roller; the baseline determination subunit 3052 is used to determine the upper baseline and the lower baseline based on the endpoint coordinates; the surface determination subunit 3053 is used to determine the roller reference surface based on the upper baseline and the lower baseline.

[0124] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned conveyor belt edge defect detection device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment. For the convenience and brevity of the description, it will not be repeated here.

[0125] The conveyor belt edge defect detection device 300 can be implemented in the form of a computer program. The computer program can be used in Figure 12 Runs on the computer device shown.

[0126] See also Figure 12 , Figure 12 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0127] See Figure 12 The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0128] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can enable the processor 502 to perform a conveyor belt edge defect detection method.

[0129] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0130] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a conveyor belt edge defect detection method.

[0131] The network interface 505 is used to communicate with other devices through the network. Figure 12 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0132] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0133] Acquire a picture of the conveyor belt to obtain a picture to be detected; perform binary semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing the conveyor belt area and the background; perform edge straight line detection on the semantic segmentation picture to obtain a parametric equation of the straight line to determine the conveyor belt edge reference plane; perform roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; determine the roller reference plane based on the upper edge roller and the lower edge roller; determine the degree of conveyor belt edge wear based on the roller reference plane, the conveyor belt edge reference plane and the conveyor belt area; and perform alarm processing based on the degree of conveyor belt edge wear.

[0134] In one embodiment, when the processor 502 implements the step of performing binary semantic segmentation on the image to be detected to obtain a semantically segmented image containing the conveyor belt area and the background, the processor 502 specifically implements the following steps:

[0135] A semantic segmentation network is used to perform semantic segmentation on the image to be detected to obtain a segmented image; and black and white binary mapping is performed on the segmented image to obtain a semantic segmented image containing the conveyor belt area and the background.

[0136] Among them, the semantic segmentation network adopts the resnet50 model as the skeleton network. The resnet50 model is formed by combining the context path and spatial path mechanism through the feature fusion module to perform feature fusion, and the loss function of the semantic segmentation network is changed to the Dice loss function. The output of the attention refinement module before the feature fusion module is selected to add two features as the auxiliary loss function, and the dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network.

[0137] In one embodiment, when the processor 502 performs the step of detecting edge lines on the semantic segmentation image to obtain a parametric equation of the line, the processor 502 specifically implements the following steps:

[0138] Feature extraction is performed on the semantic segmentation image to obtain a feature map; Hough transform is performed on the feature map to obtain a changed feature map; the depth information of each layer in the changed feature map is converted from a spatial domain to a parameter domain to obtain a parametric equation of a straight line; and a conveyor belt edge reference plane is determined based on the parametric equation of the straight line.

[0139] The depth information of each layer includes the coordinates of the intersection of the parametric equations of the two conveyor belt lines and the slopes of the two conveyor belt lines.

[0140] In one embodiment, when the processor 502 performs the step of performing roller detection on the image to be detected to obtain the upper edge roller and the lower edge roller, the processor 502 specifically implements the following steps:

[0141] The yolov5 target detection model is used to locate the rollers in the middle image to obtain the image coordinate position of each roller; the center point coordinates of each roller are calculated based on the image coordinate position of each roller; the distance from the center point coordinates to the upper edge conveyor belt and the distance from the center point coordinates to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

[0142] In one embodiment, when the processor 502 implements the step of determining the roller reference plane according to the upper edge roller and the lower edge roller, the processor 502 specifically implements the following steps:

[0143] Calculate the endpoint coordinates of the upper edge roller and the lower edge roller; determine the upper reference line and the lower reference line according to the endpoint coordinates; and determine the roller reference plane according to the upper reference line and the lower reference line.

[0144] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0145] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0146] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0147] Acquire a picture of the conveyor belt to obtain a picture to be detected; perform binary semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing the conveyor belt area and background; perform edge straight line detection on the semantic segmentation picture containing the conveyor belt area and background to obtain a parametric equation of the straight line to determine the conveyor belt edge reference plane; perform roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; determine the roller reference plane based on the upper edge roller and the lower edge roller; determine the degree of conveyor belt edge wear based on the roller reference plane, the conveyor belt edge reference plane and the conveyor belt area; and perform alarm processing based on the degree of conveyor belt edge wear.

[0148] In one embodiment, when the processor executes the computer program to implement the step of performing binary semantic segmentation on the image to be detected to obtain a semantically segmented image containing a conveyor belt area and a background, the processor specifically implements the following steps:

[0149] A semantic segmentation network is used to perform semantic segmentation on the image to be detected to obtain a segmented image; and black and white binary mapping is performed on the segmented image to obtain a semantic segmented image containing the conveyor belt area and the background.

[0150] Among them, the semantic segmentation network adopts the resnet50 model as the skeleton network. The resnet50 model is formed by combining the context path and spatial path mechanism through the feature fusion module to perform feature fusion, and the loss function of the semantic segmentation network is changed to the Dice loss function. The output of the attention refinement module before the feature fusion module is selected to add two features as the auxiliary loss function, and the dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network.

[0151] In one embodiment, when the processor executes the computer program to implement the step of performing edge line detection on the semantic segmentation image to obtain a parametric equation of a line, the processor specifically implements the following steps:

[0152] Feature extraction is performed on the semantic segmentation image to obtain a feature map; Hough transform is performed on the feature map to obtain a changed feature map; the depth information of each layer in the changed feature map is converted from a spatial domain to a parameter domain to obtain a parametric equation of a straight line; and a conveyor belt edge reference plane is determined based on the parametric equation of the straight line.

[0153] The depth information of each layer includes the coordinates of the intersection of the parametric equations of the two conveyor belt lines and the slopes of the two conveyor belt lines.

[0154] In one embodiment, when the processor executes the computer program to implement the step of performing roller detection on the image to be detected to obtain the upper edge roller and the lower edge roller, the processor specifically implements the following steps:

[0155] The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained; the center point coordinates of each roller are calculated based on the image coordinate position of each roller; the distance from the center point coordinates to the upper edge conveyor belt and the distance from the center point coordinates to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

[0156] In one embodiment, when the processor executes the computer program to implement the step of determining the roller reference plane according to the upper edge roller and the lower edge roller, the processor specifically implements the following steps:

[0157] Calculate the endpoint coordinates of the upper edge roller and the lower edge roller; determine the upper reference line and the lower reference line according to the endpoint coordinates; and determine the roller reference plane according to the upper reference line and the lower reference line.

[0158] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0159] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0160] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0161] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0162] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0163] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A conveyor belt edge defect detection method, characterized in that: include: Get a picture of the conveyor belt to get the picture to be detected; Performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing the conveyor belt area and the background; Performing edge line detection on the semantic segmentation image to obtain a parametric equation of the line to determine the edge reference plane of the conveyor belt; Performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller; Determine the roller reference surface according to the upper edge roller and the lower edge roller; determining the degree of wear on the conveyor belt edge based on the roller reference surface, the conveyor belt edge reference surface, and the conveyor belt area; Performing alarm processing according to the degree of wear on the edge of the conveyor belt; The semantic segmentation of the image to be detected to obtain an intermediate semantic segmentation image containing the conveyor belt area and the background includes: Performing semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image; Performing black-white binary mapping on the segmented image to obtain a semantic segmentation image containing the conveyor belt area and the background; The semantic segmentation network uses the ResNet50 model as the skeleton network. The ResNet50 model is formed by combining the context path and spatial path mechanisms through a feature fusion module to perform feature fusion. The loss function of the semantic segmentation network is changed to the Dice loss function. The output of the attention refinement module before the feature fusion module is selected to add two features as the auxiliary loss function. The Dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network. The performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller includes: The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained; Calculate the center point coordinates of each roller according to the image coordinate position of each roller; Calculate the distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt to determine the upper edge roller and the lower edge roller; Determining the roller reference plane according to the upper edge roller and the lower edge roller includes: Calculate the endpoint coordinates of the upper edge roller and the lower edge roller; Determine an upper reference line and a lower reference line according to the endpoint coordinates; The roller reference surface is determined based on the upper reference line and the lower reference line.

2. The conveyor belt edge defect detection method according to claim 1, characterized in that: The performing edge line detection on the semantic segmentation image to obtain a parametric equation of the line to determine the conveyor belt edge reference plane includes: Performing feature extraction on the semantic segmentation image to obtain a feature map; Performing a Hough transform on the feature map to obtain a changed feature map; The depth information of each layer in the changed feature map is converted from the spatial domain to the parameter domain to obtain the parametric equation of the line; The conveyor belt edge reference plane is determined based on the parametric equation of the straight line.

3. Conveyor belt edge defect detection device, characterized in that, include: An image acquisition unit, used to acquire an image of the conveyor belt to obtain an image to be detected; a segmentation unit, configured to perform binary semantic segmentation on the image to be detected to obtain a semantically segmented image containing a conveyor belt area and a background; a line detection unit, configured to perform edge line detection on the semantic segmentation image to obtain a parametric equation of the line, so as to determine a conveyor belt edge reference plane; A roller detection unit, configured to perform roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller; A roller reference plane determining unit, configured to determine a roller reference plane based on an upper edge roller and a lower edge roller; a degree determination unit, configured to determine a degree of wear on the edge of the conveyor belt based on the roller reference surface, the conveyor belt edge reference surface, and the conveyor belt area; An alarm processing unit, configured to perform alarm processing according to the degree of wear on the edge of the conveyor belt; The segmentation unit includes a semantic segmentation subunit and a mapping subunit; A semantic segmentation subunit, configured to perform semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image; A mapping subunit is used to perform black-and-white binary mapping on the segmented image to obtain a semantic segmentation image containing the conveyor belt area and background; the semantic segmentation network uses a resnet50 model as a skeleton network, and the resnet50 model is formed by combining context path and spatial path mechanisms through a feature fusion module to perform feature fusion, and the loss function of the semantic segmentation network is changed to a Dice loss function, and two features are added to the output of the attention refinement module before the feature fusion module as an auxiliary loss function, and the Dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network; The roller detection unit includes a target positioning subunit, a center point coordinate calculation subunit and a distance calculation subunit; A target positioning subunit is used to locate the rollers in the image to be detected using a Yolov5 target detection model to obtain the image coordinate position of each roller; a center point coordinate calculation subunit, for calculating the center point coordinates of each roller according to the image coordinate position of each roller; a distance calculation subunit, for calculating the distance from the center point coordinates to the upper edge conveyor belt and the distance from the center point coordinates to the lower edge conveyor belt, so as to determine the upper edge roller and the lower edge roller; The roller reference surface determination unit includes an endpoint coordinate calculation subunit, a reference line determination subunit, and a surface determination subunit; An endpoint coordinate calculation subunit, used to calculate the endpoint coordinates of the upper edge roller and the lower edge roller; A baseline determination subunit, configured to determine an upper baseline and a lower baseline according to the endpoint coordinates; The surface determination subunit is used to determine the roller reference surface according to the upper reference line and the lower reference line.

4. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 2 when executing the computer program.

5. A storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 2 can be implemented.

Images