Conveyor belt deviation degree detection method, device, computer equipment and storage medium

Abstract

The embodiment of the present invention discloses a conveyor belt deviation degree detection method, device, computer equipment and storage medium. The method includes: obtaining a picture of the conveyor belt to obtain a picture to be detected; performing binary semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing the conveyor belt area and the background; performing edge key point detection on the picture to be detected; determining the parametric equation of the straight line; determining the conveyor belt edge angle bisector; performing roller detection on the picture to be detected; determining the roller reference line angle bisector; determining the angle offset angle; determining the global offset area; determining the conveyor belt deviation degree according to the angle offset angle and the global offset area; and performing alarm processing according to the conveyor belt deviation degree. By implementing the method of the embodiment of the present invention, the problem of being unable to perform detection due to the problem of instrument deployment angle can be solved, and the situation where multiple edges are detected or some edges cannot be detected in the conveyor belt edge detection can be avoided.

Description

Technical Field

[0001] The present invention relates to a conveyor belt, and more specifically to a conveyor belt deviation degree detection method, device, computer equipment and storage medium. Background Art

[0002] Cargo transport conveyor belts are widely used in short-distance transportation scenarios such as shipping ports and processing plants. Their advantages such as simple structure, large conveying capacity and wide application scenarios make them applicable to all walks of life. However, during the transportation process, the cargo transport conveyor belts may shift due to installation position deviation, material overweight, offset and other problems. Long-term deviation is more likely to cause the conveyor belt to break, seriously affecting the service life of the conveyor belt. Once a problem occurs, it will also affect the entire production process.

[0003] The existing mainstream methods for detecting deviation of cargo conveyor belts are mostly contact, laser infrared, Hall magnetic induction, etc. These detection methods either limit the displacement of the conveyor belt through the card slot, which will cause friction on the edge of the conveyor belt and cause damage to the conveyor belt. Or they need to be detected by very expensive instruments. Once the sensor has problems, the results of the detection system itself will also be seriously affected.

[0004] Chinese patent CN 201910421387.0 uses video to detect and track multiple rollers on both sides and track the target occlusion time. When more than two fixed-point rollers are blocked at the same time, the target occlusion time is tracked. When the time exceeds a certain threshold, the server extracts the image around the blocked roller position and gives it to the image classification algorithm to identify whether it is a conveyor belt, and then perform alarm processing. However, this method will only detect after the roller is covered, and at this time the conveyor belt may have already experienced a large deviation, and the timeliness of deviation detection needs to be improved. Chinese patent CN201510221251.7 is based on machine vision, and marks the central axis of the conveyor belt on the non-bearing surface of the cargo transport conveyor belt. The angle between the vertical line of the detected central axis of the conveyor belt and the horizontal direction of the image coordinate system is used to describe the deviation and direction of the conveyor belt. The advantage of this method is that it maintains high accuracy without fitting through historical sample data, but it also has many disadvantages, such as only being able to shoot videos with a vertical perspective, and the need to manually mark the central axis on the conveyor belt. Some scenes are not suitable for vertical perspectives. Marking the central axis on the conveyor belt will cause the central axis to be blurred due to dust adsorption, or the conveyor belt will deform after long-term load, causing unstable detection. There are also some existing technologies that use real-time collected images, detect the conveyor belt transmission image frame by frame by using Hough line detection, and use SVM classification algorithm to screen out the edge straight line of the conveyor belt; obtain historical sample data, calculate the threshold interval of the conveyor belt offset; compare the screened edge straight line with the conveyor belt offset threshold space, and calculate the degree of conveyor belt offset, but because SVM has no general solution to nonlinear problems, it is sometimes impossible to find a suitable kernel function. And it is very sensitive to missing data.

[0005] In summary, due to the complex on-site environment of the port, the existing cargo transport conveyor belts will wear out and slip after long-term operation, and eventually deviate. At the same time, due to the limitations of the on-site environment, the detection instrument cannot directly view the conveyor belt for shooting. The existing method only detects the conveyor belt at a straight-on angle. The conveyor belt deviation detection basically uses Hough change for straight line detection, but this type of method often causes the uneven edges of the conveyor belt, resulting in multiple edges being detected or some edges not being detected during the conveyor belt edge detection. The deviation is classified but the degree of deviation cannot be specifically quantified. Some quantification of the degree of deviation can only measure the overall degree of deviation, and the degree of local deviation cannot be measured, resulting in the inability to provide early warning of the conveyor belt deviation. The detection benchmark often needs to be manually predetermined, which makes the on-site deployment extremely complicated. At the same time, it is very difficult to debug when the cargo pipeline is working.

[0006] Therefore, it is necessary to design a new method to solve the problems existing in the prior art. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device, computer equipment and storage medium for detecting the deviation degree of a conveyor belt.

[0008] To achieve the above object, the present invention adopts the following technical solution: A method for detecting the deviation degree of a conveyor belt, comprising:

[0009] Get a picture of the conveyor belt to get the picture to be detected;

[0010] Performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background;

[0011] Performing edge key point detection on the image to be detected to obtain edge key points;

[0012] Determine a parametric equation of a straight line according to the edge key points;

[0013] Determine the conveyor belt edge angle bisector according to the parametric equation of the straight line;

[0014] Performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller;

[0015] Determine the roller reference line angle bisector according to the upper edge roller and the lower edge roller;

[0016] Determine the angle deviation angle according to the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line;

[0017] Determine the global offset area according to the area of ​​a polygonal frame formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected;

[0018] Determining the degree of conveyor belt deviation according to the included angle deviation angle and the global deviation area;

[0019] An alarm is issued according to the degree of deviation of the conveyor belt.

[0020] A further technical solution is: performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background, including:

[0021] Using a semantic segmentation network to perform semantic segmentation on the image to be detected to obtain a segmented image;

[0022] The segmented image is subjected to black-and-white binary mapping to obtain a semantic segmented image containing the conveyor belt area and the background.

[0023] A further technical solution is: performing edge key point detection on the image to be detected to obtain edge key points includes:

[0024] The CPN network is used to identify the coordinates of four key points of the image to be detected to obtain edge key points.

[0025] A further technical solution is: determining the conveyor belt edge angle bisector according to the parametric equation of the straight line comprises:

[0026] Determine the coordinates of the intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt;

[0027] The conveyor belt edge angle bisector is determined according to the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

[0028] A further technical solution is: performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller, including:

[0029] The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained;

[0030] Calculate the coordinates of the center point of each roller according to the image coordinate position of each roller;

[0031] The distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

[0032] A further technical solution is: determining the global offset area according to the area of ​​a polygonal frame formed by the intersection of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected, including:

[0033] Determine a set of intersections between the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line and the edge of the image to be detected to obtain a polygonal frame;

[0034] The area of ​​the polygonal box is calculated to obtain the global offset area.

[0035] A further technical solution is: determining the conveyor belt deviation degree according to the included angle deviation angle and the global deviation area comprises:

[0036] When the global offset area is greater than the offset area threshold and the included angle offset angle is less than the angle offset threshold, determining the degree of conveyor belt offset according to the global offset area;

[0037] When the global offset area is greater than the offset area threshold and the included angle offset angle is greater than the angle offset threshold, determining the degree of conveyor belt offset according to the included angle offset angle;

[0038] When the global offset area is smaller than the offset area threshold and the angle offset angle is larger than the angle offset threshold, the conveyor belt offset degree is determined according to the angle offset angle.

[0039] The present invention also provides a conveyor belt deviation degree detection device, comprising:

[0040] A picture acquisition unit, used to acquire a picture of the conveyor belt to obtain a picture to be detected;

[0041] A segmentation unit, used for performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background;

[0042] A key point detection unit, used to perform edge key point detection on the image to be detected to obtain edge key points;

[0043] An equation determination unit, used for determining a parameter equation of a straight line according to the edge key points;

[0044] A first determining unit, used for determining the conveyor belt edge angle bisector according to the parametric equation of the straight line;

[0045] A roller detection unit, used for performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller;

[0046] A second determining unit, configured to determine a roller reference line angle bisector according to an upper edge roller and a lower edge roller;

[0047] An angle deviation angle determination unit, used for determining the angle deviation angle according to the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line;

[0048] A global offset area determination unit, used to determine the global offset area according to the area of ​​a polygonal frame formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected;

[0049] A degree determination unit, used to determine the degree of conveyor belt deviation according to the included angle deviation angle and the global deviation area;

[0050] An alarm processing unit is used to perform alarm processing according to the degree of deviation of the conveyor belt.

[0051] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0052] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program can implement the above method when executed by a processor.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention uses the form of pictures to detect edge defects, which greatly saves costs and simplifies deployment steps; semantic segmentation, key point detection, determination of straight line parameter equations, determination of conveyor belt edge angle bisectors, roller detection, roller baseline angle bisector detection, angle offset angle determination, global offset area determination and conveyor belt offset detection are performed to determine the degree of conveyor belt offset. Based on semantic segmentation, target detection and straight line detection algorithms, conveyor belt offsets at any angle can be detected to solve the problem that detection cannot be performed due to instrument deployment angle problems. Conveyor belt offset evaluation is performed through local angle offset and global area offset strategies, without the need to manually define a benchmark, and early warning of conveyor belt offset is given, and it can avoid the situation where multiple edges are detected or some edges cannot be detected during conveyor belt edge detection.

[0054] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0056] Figure 1 A schematic diagram of an application scenario of the conveyor belt deviation degree detection method provided by an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of a flow chart of a method for detecting the degree of conveyor belt deviation provided by an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of a sub-process of a method for detecting a conveyor belt deviation degree provided by an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of a sub-process of a method for detecting a conveyor belt deviation degree provided by an embodiment of the present invention;

[0060] Figure 5A schematic diagram of a sub-process of a method for detecting a conveyor belt deviation degree provided by an embodiment of the present invention;

[0061] Figure 6 A schematic diagram of a sub-process of a method for detecting a conveyor belt deviation degree provided by an embodiment of the present invention;

[0062] Figure 7 A schematic block diagram of a conveyor belt deviation degree detection device provided by an embodiment of the present invention;

[0063] Figure 8 A schematic block diagram of a segmentation unit of a conveyor belt deviation degree detection device provided by an embodiment of the present invention;

[0064] Fig. 9 A schematic block diagram of a first determination unit of a conveyor belt deviation degree detection device provided by an embodiment of the present invention;

[0065] Fig.10 A schematic block diagram of a roller detection unit of a conveyor belt deviation degree detection device provided by an embodiment of the present invention;

[0066] Fig.11 A schematic block diagram of a global deviation area determination unit of a conveyor belt deviation degree detection device provided by an embodiment of the present invention;

[0067] Fig.12 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0070] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0071] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0072] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the method for detecting the degree of conveyor belt deviation provided in an embodiment of the present invention. Figure 2 A schematic flow chart of a conveyor belt deviation degree detection method provided in an embodiment of the present invention. The conveyor belt deviation degree detection method is applied in a server. The server exchanges data with a terminal and a camera, wherein the camera is used to take pictures of the conveyor belt, and the server performs semantic segmentation, edge key point detection, determination of parameter equations of edge straight lines, determination of conveyor belt edge angle bisectors, roller detection, roller baseline angle bisector detection, determination of angle deviation angles, determination of global deviation areas, and conveyor belt deviation detection to determine the degree of conveyor belt deviation and perform alarm processing.

[0073] Figure 2 FIG. 1 is a flow chart of a method for detecting the degree of conveyor belt deviation provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S210.

[0074] S110, obtaining a picture of the conveyor belt to obtain a picture to be detected.

[0075] In this embodiment, the image to be detected refers to an image formed by collecting image data at a fixed position set on the cargo transport conveyor belt. During the collection process, fixed preset values ​​are given to the focal length and height of the pan-tilt camera.

[0076] Get rid of the dependence on hardware such as sensors, and detect the edge wear of the cargo conveyor belt based only on images, which greatly saves costs and simplifies the deployment steps. It can detect the deviation of the conveyor belt at any angle, solving the problem of being unable to perform detection due to the angle of instrument deployment.

[0077] S120 , performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background.

[0078] In this embodiment, the semantic segmentation image refers to an image including a conveyor belt area and a mask corresponding to the background.

[0079] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S122.

[0080] S121. Perform semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image.

[0081] In this embodiment, the segmented images refer to two types of semantic segmentation images: conveyor belt mask and background mask.

[0082] Specifically, the semantic segmentation network adopts the resnet50 model as the skeleton network. The resnet50 model is formed by combining the context path and the spatial path mechanism through the feature fusion module, and the loss function of the semantic segmentation network is changed to the Dice loss function. The output of the attention refinement module before the feature fusion module is selected to add two features as the auxiliary loss function, and the dice loss function and the auxiliary loss function are added in proportion to form the final loss function of the semantic segmentation network.

[0083] Specifically, the semantic segmentation of the cargo conveyor belt uses resnet50 as the backbone network. This model combines the context path and spatial path mechanisms to perform feature fusion through the Feature Fusion module, and finally obtains the semantic segmentation network. The loss function used here is changed to the dice loss function l1. Here, the ARM (Attention Refinement Module) output before FeatureFusion is selected to add two features as auxiliary loss functions l2 and l3, and finally they are added in a ratio of 1:1:1 as the final loss function Loss=l1+l2+l3.

[0084] In order to ensure the accuracy of the conveyor belt mask prediction, the image to be detected is first used as the input of the network with an original width of 1920 and a height of 1080 pixels. The output categories of the semantic segmentation network are two types of semantic segmentation maps: conveyor belt mask and background mask.

[0085] S122, performing black-and-white binary mapping on the segmented image to obtain a semantic segmented image containing a conveyor belt area and a background.

[0086] After performing black-and-white binary mapping on the segmented image, the three channels of the pixels in the conveyor belt area are set to 255 pixels, and the three channels of the background area are set to 0 pixels, thus forming a semantic segmentation image containing the conveyor belt area and the background. At the same time, the conveyor belt area is defined as s 传送带 .

[0087] S130: Perform edge key point detection on the image to be detected to obtain edge key points.

[0088] In this embodiment, the edge key points refer to four points: the starting end and the ending end of the upper edge of the conveyor belt, and the starting end and the ending end of the lower edge.

[0089] Specifically, a CPN network is used to identify the coordinates of four key points of the image to be detected to obtain edge key points.

[0090] The confidence level of the key points of the CPN network is 0.5 to 0.7. The coordinates of the four key points are spatially clustered using k-means, and the center point of the cluster is selected as the edge key point.

[0091] When detecting the edge of the conveyor belt, the effect of straight line detection based on Hough change is not good because the edge of the conveyor belt is uneven. Here, detection is performed based on the key points of the conveyor belt, and the starting and ending points of the upper and lower edges of the conveyor belt are identified respectively. The model used here is the CPN (Cascaded Pyramid Network) network to identify four key points. The algorithm based on key point detection detects the vertex coordinates at both ends of the conveyor belt edge, and derives the parametric equation of the edge line based on the coordinates of its two ends.

[0092] Through this neural network, we can get the two key points of the starting point and the end point of the upper edge. The upper edge point is p s,top , p e,top , the lower edge point is p s,bottom , p e,bottom In order to prevent redundant key points from being detected due to edges and key points from being missed, the key point confidence is slightly lowered to 0.6. On the basis of improving the key point recall rate, k-means clustering is performed on the key point coordinates in space, k=4 is set, and the center point of the cluster is selected as its representative point.

[0093] S140: Determine a parametric equation of a straight line according to the edge key points.

[0094] In this embodiment, the parametric equation of the straight line refers to the parametric equation of the edge straight line of the conveyor belt.

[0095] The parametric equations of the straight lines of the upper and lower conveyor belt edges can be determined based on the upper edge points, i.e., two key points on the upper conveyor belt edge, and the lower edge points, i.e., two key points on the lower conveyor belt edge.

[0096] S150. Determine the conveyor belt edge angle bisector according to the parametric equation of the straight line.

[0097] In this embodiment, the conveyor belt edge angle bisector refers to the angle bisector of the upper edge of the conveyor belt and the lower edge of the conveyor belt.

[0098] In one embodiment, see Figure 4, the above-mentioned step S150 may include steps S151 to S152.

[0099] S151, determine the coordinates of the intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

[0100] Specifically, based on the parametric equation of the straight line, the edge point where the corresponding straight line intersects with the image to be detected is obtained, and two edge line segments l of the conveyor belt are drawn in sequence. 输送带top and 输送带down .

[0101] S152, determining the conveyor belt edge angle bisector according to the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

[0102] Specifically, the above l 输送带top and 输送带down The coordinates of the intersection of the two conveyor belt straight line equations (x 交点 ,y 交点 ) and the slopes of the two lines into the half-angle formula The slope k of the parametric equation of the straight line at the upper edge of the conveyor belt is 输送带top The angle A between the straight line and the coordinate system can be obtained. Similarly, the slope k of the parametric equation of the straight line at the lower edge of the conveyor belt is 输送带down Obtain the angle B, substitute it into the right side of the half-angle formula, and obtain the slope k of the angle bisector, that is, Then we can get the slope k of the angle bisector of the two edge lines. 输送带 , and then substitute it into the intersection coordinates (x 交点 ,y 交点 ), we get the equation of the angle bisector f(x) 角平分线 =k*x+b.

[0103] This embodiment optimizes the feature extraction part of key point detection, combines the FPN (Feature Pyramid Networks) network with the Transformer mechanism to extract model features, and combines GlobalNet and RefineNet for feature point recognition. In order to prevent the detection of redundant edge key points and the omission of key points, the key point confidence is reduced, and k-means clustering is performed on the key point coordinates in space. Here, k=4 is set, and the center point of the cluster is selected as its representative point. The OHEM strategy is also used in the training process. Only the top 3 key points with the largest loss are returned, because individual key points will be difficult to identify due to light and other dust. After obtaining the corresponding two conveyor belt edge lines, the conveyor belt edge reference plane is determined.

[0104] S160: Perform roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller.

[0105] In this embodiment, the rollers of the conveyor belt include an upper edge roller and a lower edge roller.

[0106] In one embodiment, see Figure 5 , the above-mentioned step S160 may include steps S161 to S163.

[0107] S161, using the yolov5 target detection model to locate the rollers in the image to be detected, and obtaining the image coordinate position of each roller.

[0108] In this embodiment, the image coordinate position of each roller refers to the coordinates of the upper edge roller and the lower edge roller in the image.

[0109] Specifically, the rollers are located by the yolov5 target detection model, and the image coordinate position (x min ,y min , x max ,y max ). The detection confidence of the yolov5 target detection model is selected as 0.6. In order to detect the accuracy of the coordinates of the top of the roller, the structure selected is the yolo5x structure for detection. Here, the loss function of the regression box coordinates selects CIOU loss as its corresponding loss function, and combines the foreground background cross entropy loss and the classification cross entropy loss to form the overall loss function.

[0110] What needs to be noted in the roller detection here is that if a roller is located at the edge of the image, it will easily cause the roller to be incomplete, resulting in inaccurate endpoint values. Based on this situation, the position of the roller is judged separately. If any of the following conditions is met, the endpoint coordinates of the roller are not considered.

[0111] (1)x min <α; (2) width-x max <α; (3)y min <α; (4) height-y max <α; here width is the width of the image, height is the height of the image, and α is set to 10.

[0112] S162, calculating the coordinates of the center point of each roller according to the image coordinate position of each roller.

[0113] In this embodiment, the center point coordinates of each roller frame are calculated respectively.

[0114] S163, calculating the distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt to determine the upper edge roller and the lower edge roller.

[0115] In this embodiment, by calculating the center point coordinates (c x , c y ) and to the upper edge l 传送带 The distance d 上边缘 and to the lower edge l 传送带 The distance d 下边缘 If d 下边缘 <d 上边缘 , then the roller is determined to be the lower edge roller, otherwise it is called the upper edge roller.

[0116] S170, determining the roller reference line angle bisector according to the upper edge roller and the lower edge roller.

[0117] In this embodiment, the roller reference line angle bisector refers to the angle bisector of two reference lines determined by the upper edge roller and the lower edge roller.

[0118] Specifically, the endpoint coordinates of the upper edge roller and the lower edge roller are calculated; the upper reference line and the lower reference line are determined according to the endpoint coordinates; the roller reference line angle bisector is determined according to the upper reference line, the lower reference line and the intersection of the two reference lines; if the slope k of the conveyor belt 传送带 >0, for the upper edge roller, select the coordinates of the upper right corner of each upper edge roller as its endpoint coordinates p i(k>0,top) (x max ,y min), the lower edge roller selects the coordinates of the upper left corner of each lower edge roller as its endpoint coordinates p i(k>0,bottom) (x min ,y min ). If the slope k of the conveyor belt 传送带 <0, for the upper edge roller, the coordinates of the upper left corner of the roller are selected as its endpoint coordinates p i(k<0,top) (x min ,y min ), the lower edge roller selects the coordinates of the upper right corner of the roller as its endpoint coordinates p i(k<0,top) (x max ,y min ), and finally two baselines l are fitted through these endpoints. 基准 The bisector of the roller reference line angle obtained by the reference line and the intersection of the reference lines l 基准角平分线 .

[0119] S180, determining the angle deviation angle according to the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line.

[0120] In this embodiment, the angle offset angle refers to the angle between the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line.

[0121] Based on the above, the conveyor belt edge angle bisector l 传送带角平分线 , and the resulting roller reference line angle bisector l 基准角平分线 , calculate the corresponding angle offset angle θ 偏移 .

[0122] S190, determining a global offset area according to the area of ​​a polygonal frame formed by a set of intersection points of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected.

[0123] In this embodiment, the global offset area refers to the area of ​​a polygon formed by a set of intersections of the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line with the edge of the image to be detected.

[0124] In one embodiment, see Figure 6 , the above-mentioned step S190 may include steps S191 to S192.

[0125] S191, determining a set of intersections between the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line and the edge of the image to be detected to obtain a polygonal frame;

[0126] S192. Calculate the area of ​​the polygonal frame to obtain a global offset area.

[0127] Because the angle offset can only reflect the local offset, but if the conveyor belt is offset as a whole, the angle offset will not change. Therefore, the angle bisector of the conveyor belt edge l 传送带角平分线 Angle bisector l with roller reference line 基准角平分线 The area of ​​the polygonal box enclosed by the set of intersection points with the edge of the image to be detected is used to calculate the global offset area s. 偏移 The local offset area here is a closed polygonal box formed by the intersection of two angle bisectors and the edge of the image. The main idea is to divide the polygon into multiple triangles, then add up the areas of all triangles, and calculate the area of ​​the triangle using the cross product of the vector. in, is a vector, which can be obtained from the coordinates of points A and B; |AB| is The modulus of ; sin(BAC) is the sin value of ∠BAC. The advantage of this method is that the existing point coordinates can be directly converted into vectors. Another advantage is that it supports the calculation of the area of ​​not only convex polygons, but also concave polygons.

[0128] S200, determining a conveyor belt deviation degree according to the included angle deviation angle and the global deviation area.

[0129] Specifically, when the global offset area is greater than the offset area threshold and the angle offset angle is less than the angle offset threshold, the degree of conveyor belt offset is determined according to the global offset area; when the global offset area is greater than the offset area threshold and the angle offset angle is greater than the angle offset threshold, the degree of conveyor belt offset is determined according to the angle offset angle; when the global offset area is less than the offset area threshold and the angle offset angle is greater than the angle offset threshold, the degree of conveyor belt offset is determined according to the angle offset angle.

[0130] In this embodiment, the offset area threshold s is set threshold , and the angle offset threshold θ threshold . Conveyor belt deviation degree τ; when s 偏移 >s threshold And θ 偏移 <θ threshold ,but Set λ to s 传送带 is the conveyor belt area; when s 偏移 >s threshold And θ 偏移 >θ threshold ,but set up When 偏移 threshold And θ 偏移 >θ threshold ,but​ set up

[0131] S210: Perform alarm processing according to the degree of deviation of the conveyor belt.

[0132] Set the alarm threshold γ, τ>γ, then the alarm mechanism is triggered. Here, γ=0.2.

[0133] This embodiment provides local and global offset measures for this situation. Here, conveyor belt offset assessment is performed using local angle offset and global area offset strategies, and automatic local angle offset and global area offset values ​​can be provided to measure conveyor belt offset without manually defining a benchmark.

[0134] The above-mentioned method for detecting the degree of conveyor belt deviation, by detecting edge defects in the form of pictures, greatly saves costs and simplifies the deployment steps; semantic segmentation, key point detection, determination of straight line parameter equations, determination of conveyor belt edge angle bisectors, roller detection, detection of roller baseline angle bisectors, determination of angle deviation angles, determination of global deviation area and conveyor belt deviation detection are performed to determine the degree of conveyor belt deviation. Based on semantic segmentation, target detection and straight line detection algorithms, conveyor belt deviation at any angle can be detected to solve the problem that detection cannot be performed due to instrument deployment angle problems. Conveyor belt deviation is evaluated through local angle deviation and global area deviation strategies. There is no need to manually define a benchmark, and early warning of conveyor belt deviation is given. It can also avoid the situation where multiple edges are detected or some edges cannot be detected during conveyor belt edge detection.

[0135] Figure 7 is a schematic block diagram of a conveyor belt deviation degree detection device 300 provided in an embodiment of the present invention. Figure 7 As shown, corresponding to the above conveyor belt deviation degree detection method, the present invention also provides a conveyor belt deviation degree detection device 300. The conveyor belt deviation degree detection device 300 includes a unit for executing the above conveyor belt deviation degree detection method, and the device can be configured in a server. Figure 7 The conveyor belt deviation degree detection device 300 includes a picture acquisition unit 301, a segmentation unit 302, a key point detection unit 303, an equation determination unit 304, a first determination unit 305, a roller detection unit 306, a second determination unit 307, an angle deviation angle determination unit 308, a global deviation area determination unit 309, a degree determination unit 310 and an alarm processing unit 311.

[0136] The image acquisition unit 301 is used to acquire an image of the conveyor belt to obtain an image to be detected; the segmentation unit 302 is used to perform binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing the conveyor belt area and the background; the key point detection unit 303 is used to perform edge key point detection on the image to be detected to obtain edge key points; the equation determination unit 304 is used to determine the parametric equation of the straight line according to the edge key points; the first determination unit 305 is used to determine the conveyor belt edge angle bisector according to the parametric equation of the straight line; the roller detection unit 306 is used to perform roller detection on the image to be detected to obtain the upper edge roller and the lower edge roller; the second determination unit 306 is used to perform roller detection on the image to be detected to obtain the upper edge roller and the lower edge roller; A determination unit 307 is used to determine the roller baseline angle bisector according to the upper edge roller and the lower edge roller; an angle offset angle determination unit 308 is used to determine the angle offset angle according to the conveyor belt edge angle bisector and the roller baseline angle bisector; a global offset area determination unit 309 is used to determine the global offset area according to the area of ​​a polygonal box enclosed by a set of intersection points of the conveyor belt edge angle bisector, the roller baseline angle bisector and the edge of the image to be detected; a degree determination unit 310 is used to determine the degree of conveyor belt offset according to the angle offset angle and the global offset area; an alarm processing unit 311 is used to perform alarm processing according to the degree of conveyor belt offset.

[0137] In one embodiment, if Figure 8 As shown, the segmentation unit 302 includes a semantic segmentation subunit 3021 and a mapping subunit 3022 .

[0138] The semantic segmentation subunit 3021 is used to perform semantic segmentation on the image to be detected using a semantic segmentation network to obtain a segmented image; the mapping subunit 3022 is used to perform black-and-white binary mapping on the segmented image to obtain a semantic segmentation image containing the conveyor belt area and the background.

[0139] In one embodiment, the key point detection unit 303 is used to use a CPN network to identify the coordinates of four key points of the image to be detected to obtain edge key points.

[0140] In one embodiment, if Fig. 9 As shown, the first determination unit 305 includes a slope determination subunit 3051 and a conveyor belt edge angle bisector determination subunit 3052.

[0141] The slope determination subunit 3051 is used to determine the coordinates of the intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt; the conveyor belt edge angle bisector determination subunit 3052 is used to determine the conveyor belt edge angle bisector based on the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

[0142] In one embodiment, if Fig.10 As shown, the roller detection unit 306 includes a target positioning subunit 3061, a center point coordinate calculation subunit 3062 and a distance calculation subunit 3063.

[0143] The target positioning subunit 3061 is used to use the yolov5 target detection model to locate the rollers in the image to be detected, and obtain the image coordinate position of each roller; the center point coordinate calculation subunit 3062 is used to calculate the center point coordinates of each roller according to the image coordinate position of each roller; the distance calculation subunit 3063 is used to calculate the distance from the center point coordinates to the upper edge conveyor belt and the distance from the center point coordinates to the lower edge conveyor belt, so as to determine the upper edge roller and the lower edge roller.

[0144] In one embodiment, if Fig.11 As shown, the global offset area determination unit 309 includes an intersection set determination subunit 3091 and an area calculation subunit 3092 .

[0145] The intersection set determination subunit 3091 is used to determine the intersection set of the edge of the conveyor belt angle bisector and the roller reference line angle bisector with the edge of the image to be detected to obtain a polygonal frame; the area calculation subunit 3092 is used to calculate the area of ​​the polygonal frame to obtain a global offset area.

[0146] In one embodiment, the degree determination unit 310 is used to determine the degree of conveyor belt offset according to the global offset area when the global offset area is greater than the offset area threshold and the angle offset angle is less than the angle offset threshold; determine the degree of conveyor belt offset according to the angle offset angle when the global offset area is greater than the offset area threshold and the angle offset angle is greater than the angle offset threshold; determine the degree of conveyor belt offset according to the angle offset angle when the global offset area is less than the offset area threshold and the angle offset angle is greater than the angle offset threshold.

[0147] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned conveyor belt deviation degree detection device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0148] The conveyor belt deviation degree detection device 300 can be implemented in the form of a computer program. The computer program can be used in Fig.12 Runs on the computer device shown.

[0149] See also Fig.12 , Fig.12 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0150] See also Fig.12 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0151] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a method for detecting a conveyor belt deviation degree.

[0152] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0153] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for detecting the degree of conveyor belt deviation.

[0154] The network interface 505 is used to communicate with other devices over the network. Fig.12 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0156] Acquire a picture of a conveyor belt to obtain a picture to be detected; perform binary semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background; perform edge key point detection on the picture to be detected to obtain edge key points; determine the parametric equation of a straight line based on the edge key points; determine the conveyor belt edge angle bisector based on the parametric equation of the straight line; perform roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; determine the roller reference line angle bisector based on the upper edge roller and the lower edge roller; determine the angle offset angle based on the conveyor belt edge angle bisector and the roller reference line angle bisector; determine the global offset area based on the area of ​​a polygonal box enclosed by a set of intersection points of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the picture to be detected; determine the degree of conveyor belt offset based on the angle offset angle and the global offset area; and perform alarm processing based on the degree of conveyor belt offset.

[0157] In one embodiment, when the processor 502 implements the step of performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background, the processor 502 specifically implements the following steps:

[0158] A semantic segmentation network is used to perform semantic segmentation on the image to be detected to obtain a segmented image; and black-and-white binary mapping is performed on the segmented image to obtain a semantic segmented image containing a conveyor belt area and a background.

[0159] In one embodiment, when the processor 502 implements the step of performing edge key point detection on the image to be detected to obtain edge key points, the processor 502 specifically implements the following steps:

[0160] The CPN network is used to identify the coordinates of four key points of the image to be detected to obtain edge key points.

[0161] In one embodiment, when the processor 502 implements the step of determining the conveyor belt edge angle bisector according to the parametric equation of the straight line, the processor 502 specifically implements the following steps:

[0162] Determine the coordinates of the intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt; determine the conveyor belt edge angle bisector based on the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

[0163] In one embodiment, when the processor 502 implements the step of performing roller detection on the image to be detected to obtain the upper edge roller and the lower edge roller, the processor 502 specifically implements the following steps:

[0164] The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained; the center point coordinates of each roller are calculated according to the image coordinate position of each roller; the distance from the center point coordinates to the upper edge conveyor belt and the distance from the center point coordinates to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

[0165] In one embodiment, when the processor 502 implements the step of determining the global offset area according to the area of ​​a polygonal box formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected, the processor 502 specifically implements the following steps:

[0166] A set of intersection points of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected is determined to obtain a polygonal frame; and the area of ​​the polygonal frame is calculated to obtain a global offset area.

[0167] In one embodiment, when the processor 502 implements the step of determining the degree of conveyor belt deviation according to the included angle deviation angle and the global deviation area, the processor 502 specifically implements the following steps:

[0168] When the global offset area is greater than the offset area threshold and the angle offset angle is less than the angle offset threshold, the degree of conveyor belt offset is determined according to the global offset area; when the global offset area is greater than the offset area threshold and the angle offset angle is greater than the angle offset threshold, the degree of conveyor belt offset is determined according to the angle offset angle; when the global offset area is less than the offset area threshold and the angle offset angle is greater than the angle offset threshold, the degree of conveyor belt offset is determined according to the angle offset angle.

[0169] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0170] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0171] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0172] Acquire a picture of a conveyor belt to obtain a picture to be detected; perform binary semantic segmentation on the picture to be detected to obtain a semantic segmentation picture containing a conveyor belt area and a background; perform edge key point detection on the picture to be detected to obtain edge key points; determine the parametric equation of a straight line based on the edge key points; determine the conveyor belt edge angle bisector based on the parametric equation of the straight line; perform roller detection on the picture to be detected to obtain an upper edge roller and a lower edge roller; determine the roller reference line angle bisector based on the upper edge roller and the lower edge roller; determine the angle offset angle based on the conveyor belt edge angle bisector and the roller reference line angle bisector; determine the global offset area based on the area of ​​a polygonal box enclosed by a set of intersection points of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the picture to be detected; determine the degree of conveyor belt offset based on the angle offset angle and the global offset area; and perform alarm processing based on the degree of conveyor belt offset.

[0173] In one embodiment, when the processor executes the computer program to implement the step of performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background, the processor specifically implements the following steps:

[0174] A semantic segmentation network is used to perform semantic segmentation on the image to be detected to obtain a segmented image; and black-and-white binary mapping is performed on the segmented image to obtain a semantic segmented image containing a conveyor belt area and a background.

[0175] In one embodiment, when the processor executes the computer program to implement the step of performing edge key point detection on the image to be detected to obtain edge key points, the processor specifically implements the following steps:

[0176] The CPN network is used to identify the coordinates of four key points of the image to be detected to obtain edge key points.

[0177] In one embodiment, when the processor executes the computer program to implement the step of using the CPN network to identify the coordinates of four key points of the image to be detected to obtain edge key points, the following steps are specifically implemented:

[0178] Determine the coordinates of the intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt; determine the conveyor belt edge angle bisector based on the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

[0179] In one embodiment, when the processor executes the computer program to implement the step of performing roller detection on the image to be detected to obtain the upper edge roller and the lower edge roller, the processor specifically implements the following steps:

[0180] The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained; the center point coordinates of each roller are calculated according to the image coordinate position of each roller; the distance from the center point coordinates to the upper edge conveyor belt and the distance from the center point coordinates to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

[0181] In one embodiment, when the processor executes the computer program to implement the step of determining the global offset area according to the area of ​​a polygonal box formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected, the processor specifically implements the following steps:

[0182] A set of intersection points of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected is determined to obtain a polygonal frame; and the area of ​​the polygonal frame is calculated to obtain a global offset area.

[0183] In one embodiment, when the processor executes the computer program to implement the step of determining the degree of conveyor belt deviation according to the included angle deviation angle and the global deviation area, the processor specifically implements the following steps:

[0184] When the global offset area is greater than the offset area threshold and the angle offset angle is less than the angle offset threshold, the degree of conveyor belt offset is determined according to the global offset area; when the global offset area is greater than the offset area threshold and the angle offset angle is greater than the angle offset threshold, the degree of conveyor belt offset is determined according to the angle offset angle; when the global offset area is less than the offset area threshold and the angle offset angle is greater than the angle offset threshold, the degree of conveyor belt offset is determined according to the angle offset angle.

[0185] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0186] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0187] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0188] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0190] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. Conveyor belt deviation detection method, It is characterized in that include: Get a picture of the conveyor belt to get the picture to be detected; Performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background; Performing edge key point detection on the image to be detected to obtain edge key points; Determine a parametric equation of a straight line according to the edge key points; Determine the conveyor belt edge angle bisector according to the parametric equation of the straight line; Performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller; Determine the roller reference line angle bisector according to the upper edge roller and the lower edge roller; Determine the angle deviation angle according to the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line; Determine the global offset area according to the area of ​​a polygonal frame formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected; Determining the degree of conveyor belt deviation according to the included angle deviation angle and the global deviation area; Performing alarm processing according to the degree of deviation of the conveyor belt; Among them, the edge key points refer to the starting and ending points of the upper edge of the conveyor belt, and the starting and ending points of the lower edge; the parametric equation of the straight line refers to the parametric equation of the straight line at the edge of the conveyor belt; the parametric equations of the straight lines at the upper and lower conveyor belt edges are determined based on the two key points of the upper conveyor belt edge and the two key points of the lower conveyor belt edge.

2. The method for detecting the degree of conveyor belt deviation according to claim 1, It is characterized in that The binary semantic segmentation of the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background includes: Using a semantic segmentation network to perform semantic segmentation on the image to be detected to obtain a segmented image; The segmented image is subjected to black-and-white binary mapping to obtain a semantic segmented image containing the conveyor belt area and the background.

3. The conveyor belt deviation detection method according to claim 1, It is characterized in that The performing edge key point detection on the image to be detected to obtain edge key points includes: The CPN network is used to identify the coordinates of four key points of the semantic segmentation picture containing the conveyor belt area and the background to obtain edge key points.

4. The method for detecting the degree of conveyor belt deviation according to claim 3, It is characterized in that The step of determining the conveyor belt edge angle bisector according to the parametric equation of the straight line comprises: Determine the coordinates of the intersection of the parametric equation of the straight line at the upper edge of the conveyor belt and the parametric equation of the straight line at the lower edge of the conveyor belt, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt; The conveyor belt edge angle bisector is determined according to the intersection coordinates, the slope of the parametric equation of the straight line at the upper edge of the conveyor belt, and the slope of the parametric equation of the straight line at the lower edge of the conveyor belt.

5. The method for detecting the degree of conveyor belt deviation according to claim 1, It is characterized in that The step of performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller includes: The yolov5 target detection model is used to locate the rollers in the image to be detected, and the image coordinate position of each roller is obtained; Calculate the coordinates of the center point of each roller according to the image coordinate position of each roller; The distance from the center point coordinate to the upper edge conveyor belt and the distance from the center point coordinate to the lower edge conveyor belt are calculated to determine the upper edge roller and the lower edge roller.

6. The method for detecting the degree of conveyor belt deviation according to claim 1, It is characterized in that The determining of the global offset area according to the area of ​​a polygonal frame formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected comprises: Determine a set of intersections between the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line and the edge of the image to be detected to obtain a polygonal frame; The area of ​​the polygonal box is calculated to obtain the global offset area.

7. The method for detecting the degree of conveyor belt deviation according to claim 1, It is characterized in that Determining the conveyor belt deviation degree according to the included angle deviation angle and the global deviation area includes: When the global offset area is greater than the offset area threshold and the included angle offset angle is less than the angle offset threshold, determining the degree of conveyor belt offset according to the global offset area; When the global offset area is greater than the offset area threshold and the included angle offset angle is greater than the angle offset threshold, determining the degree of conveyor belt offset according to the included angle offset angle; When the global offset area is smaller than the offset area threshold and the angle offset angle is larger than the angle offset threshold, the conveyor belt offset degree is determined according to the angle offset angle.

8. Conveyor belt deviation detection device, It is characterized in that include: A picture acquisition unit, used to acquire a picture of the conveyor belt to obtain a picture to be detected; A segmentation unit, used for performing binary semantic segmentation on the image to be detected to obtain a semantic segmentation image containing a conveyor belt area and a background; A key point detection unit, used to perform edge key point detection on the image to be detected to obtain edge key points; An equation determination unit, used for determining a parameter equation of a straight line according to the edge key points; A first determining unit, used for determining the conveyor belt edge angle bisector according to the parametric equation of the straight line; A roller detection unit, used for performing roller detection on the image to be detected to obtain an upper edge roller and a lower edge roller; A second determining unit, configured to determine a roller reference line angle bisector according to an upper edge roller and a lower edge roller; An angle deviation angle determination unit, used for determining the angle deviation angle according to the angle bisector of the conveyor belt edge and the angle bisector of the roller reference line; A global offset area determination unit, used to determine the global offset area according to the area of ​​a polygonal frame formed by a set of intersections of the conveyor belt edge angle bisector, the roller reference line angle bisector and the edge of the image to be detected; A degree determination unit, used to determine the degree of conveyor belt deviation according to the included angle deviation angle and the global deviation area; An alarm processing unit, used for performing alarm processing according to the degree of deviation of the conveyor belt; Among them, the edge key points refer to the starting and ending points of the upper edge of the conveyor belt, and the starting and ending points of the lower edge; the parametric equation of the straight line refers to the parametric equation of the straight line at the edge of the conveyor belt; the parametric equations of the straight lines at the upper and lower conveyor belt edges are determined based on the two key points of the upper conveyor belt edge and the two key points of the lower conveyor belt edge.

9. A computer device, It is characterized in that The computer device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 can be implemented.

Images