Belt and material deviation detection method and device and storage medium

Abstract

The invention relates to the field of belt detection, and discloses a belt and material deviation detection method and device and a storage medium, and the method comprises the steps: employing an industrial camera to obtain belt images of material transportation and belt images without material transportation in different time periods; labeling the acquired images by using a label, and dividing the images into two types of belts and materials to obtain a constructed data set; constructing a belt and material segmentation model, and training the belt and material segmentation model by using the constructed data set to obtain a trained model; and acquiring a real-time image by using an industrial camera, and inputting the real-time image into the trained model to obtain an offset judgment result. The method has the beneficial effects that compared with a traditional belt deviation and material deviation detection method, the detection accuracy is high, and the realizability is good.

Description

Technical Field

[0001] The present invention relates to the field of belt detection, and in particular to a belt and material deviation detection method, equipment and storage medium. Background Art

[0002] Existing belt deviation detection methods are mainly divided into contact and non-contact methods. The contact method usually requires the installation of sensors or devices that directly contact the belt, such as triggers or rollers, in order to detect the movement state, vibration, displacement, etc. of the belt, so as to determine whether deviation occurs. However, this method may increase belt wear and introduce other adverse effects. In contrast, the non-contact method does not require direct contact with the belt, but obtains deviation information through means such as machine vision. In particular, the machine vision-based method captures the belt image through a camera device and uses image processing and analysis algorithms to identify the deviation, avoiding direct contact, helping to reduce wear and interference, but may be affected by the environment, such as background clutter. However, the current machine vision-based methods mainly use traditional image processing technology, so the accuracy is low and there is room for improvement. In terms of material deviation detection, it still mainly relies on manual monitoring and lacks automated and efficient detection methods. Summary of the invention

[0003] The purpose of the present invention is to propose a belt and material deviation detection method, device and storage medium to solve the technical problems that the traditional contact method of belt deviation detection may cause belt wear and adverse effects and the non-contact method has low accuracy and is affected by the environment.

[0004] The present invention provides a belt and material deviation detection method, comprising the following steps: S1. Use industrial cameras to obtain belt images with and without material transportation at different times; S2, label the collected images with labels and divide them into two categories: belts and materials to obtain the constructed data set; S3, constructing a belt and material segmentation model, and using the constructed data set to train the belt and material segmentation model to obtain a trained model; S4. Use an industrial camera to obtain real-time images, input the real-time images into the trained model, and obtain the offset judgment results.

[0005] Furthermore, the belt and material segmentation model processes the real-time image as follows: S41, determining four points from the real-time image, the first two points are used to determine the reference line of the left belt roller, and the last two points are used to determine the reference line of the right belt roller; S42, taking the average value of the y coordinates of the four points, and determining a straight line passing through the entire image; S43, inputting the real-time image into the belt and material segmentation model to obtain a belt and material mask coverage instance; S44, determining the first and last contact points between the mask covering instance and the straight line as the edge points of the belt and the material; S45, comparing the edge point of the belt with the left roller reference line and the right roller reference line to determine whether the belt is deviating; S46, taking the average value of the edge points to obtain the midpoint of the belt and the material; S47. Compare the midpoint of the material with the midpoint of the belt to determine whether the material is offset.

[0006] A storage medium stores instructions and data for implementing a belt and material deviation detection method.

[0007] A belt and material deviation detection device comprises: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a belt and material deviation detection method.

[0008] The beneficial effects provided by the present invention are: a new detection method is proposed. By adopting image segmentation technology based on deep learning and combining it with a machine vision system, accurate monitoring of belt deviation and material offset is achieved without direct contact with the belt, while improving the accuracy and real-time performance of the detection. In addition, the present invention is also committed to solving the problem that material offset detection mainly relies on manual monitoring, and proposes an automated and efficient detection method, thereby realizing intelligent monitoring and management of the production process, and providing important technical support and guarantee for industrial production. Compared with traditional belt deviation and material offset detection methods, this method has high detection accuracy and good feasibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a schematic flow chart of the method of the present invention; Figure 2 It is a belt deviation and material offset detection system based on machine vision; Figure 3 It is a schematic diagram of belt deviation and material offset detection for material transport; Figure 4 The process of selecting the reference point in step S41 is shown; Figure 5 This is a schematic diagram of the belt deviation detection effect without material transportation; Figure 6 This is a schematic diagram of the belt deviation and material offset detection effect of the belt transport; Figure 7 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION

[0010] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0011] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.

[0012] Please refer to Figure 1 , Figure 1 It is a schematic diagram of the process of the present invention.

[0013] The present invention provides a belt and material deviation detection method, comprising the following steps: S1. Use industrial cameras to obtain belt images with and without material transportation at different times; It should be noted that before using industrial cameras for image acquisition, the hardware system for belt deviation and material offset should be built first. Figure 2 , Figure 2 It is a belt deviation and material offset detection system based on machine vision. Figure 2 As shown, the rack for installing the industrial camera spans across the conveyor, and the industrial camera is installed in the middle area of ​​the rack and is connected to the industrial computer.

[0014] S2, label the collected images with labels and divide them into two categories: belts and materials to obtain the constructed data set; It should be noted that the label tag refers to the identifier of the true category or target value of the data sample. In machine learning and deep learning, labels are used to represent the information that the model should learn and predict. The quality and accuracy of the label are crucial to the performance of the model. In the present invention, the main targets are divided into two categories, namely belts and materials, and specific software tools are used for labeling, such as Label Studio, lphotoDraw, etc.

[0015] S3, constructing a belt and material segmentation model, and using the constructed data set to train the belt and material segmentation model to obtain a trained model; It should be noted that the belt and material segmentation model in the present invention can adopt an image segmentation model based on deep learning, such as a segmentation model based on Deeplab-v3+, or other target segmentation models can also be adopted.

[0016] The role of the target segmentation model in the present invention is to obtain the segmented belt and material targets from the image, and the target can be distinguished from the image through the mask covering instance.

[0017] S4. Use an industrial camera to obtain real-time images, input the real-time images into the trained model, and obtain the offset judgment results.

[0018] It should be noted that the belt and material segmentation model processes real-time images as follows: S41, determining four points from the real-time image, the first two points are used to determine the reference line of the left belt roller, and the last two points are used to determine the reference line of the right belt roller; It should be noted that the above four points are manually marked. For example, the coordinates of the four points are ( x l1 , y l1 ),( x l2 , y l2 ),( x r1 , y r1 ),( x r2 , y r2 ).

[0019] S42, taking the average value of the y coordinates of the four points, and determining a straight line passing through the entire image; That is, the coordinates of the point on the straight line are ( x , y ),in x Can be any value within the image size. y =( y l1 + y l2 y r1 + y r2 ) / 4.

[0020] S43, inputting the real-time image into the belt and material segmentation model to obtain a belt and material mask coverage instance; As an example, a real-time image is input into the segmentation model of Deeplab-v3+ to obtain mask coverage examples of belts and materials.

[0021] Among them, Mask coverage instance refers to the use of a binary image (mask) to cover a specific area of ​​the original image in image processing to control the processing process or area. The mask consists of 0 and 1, where 1 represents the area to be processed and 0 represents the masked area. In simple terms, a mask coverage instance is also an area. Of course, in this application, the belt and the material are two targets, so there will be two coverage instances.

[0022] S44, determining the first and last contact points between the mask covering instance and the straight line as the edge points of the belt and the material; It is easy to understand that the mask coverage instance is an area. Since the straight line runs through the entire image, the straight line must intersect with the area to obtain two intersection points, which are also edge points. The mask coverage instance corresponding to the belt is the edge point of the belt, and the mask coverage instance corresponding to the material is the edge point of the material. The edge point of the belt is recorded as ( x p1 , y ),( x p2 , y ), the edge point of the material is recorded as ( x w1 , y ),( x w2 , y ).

[0023] S45, comparing the edge point of the belt with the left roller reference line and the right roller reference line to determine whether the belt is deviating; Specifically, the left edge point of the belt is compared with the left roller baseline to obtain a distance value, and the right edge point is also compared with the right roller baseline to obtain a distance value. The two distance values ​​are compared to obtain a difference. If the difference exceeds a threshold, the belt deviates. The deviation threshold can be dynamically adjusted according to the belt length.

[0024] S46, taking the average value of the edge points to obtain the midpoint of the belt and the material; S47. Compare the midpoint of the material with the midpoint of the belt to determine whether the material is offset.

[0025] It should be noted that because the midpoints of the material and the belt are in a straight line, y The coordinates are equal, so we only need to judge their x Coordinates, if x If the coordinates are different, the material will be offset.

[0026] To understand the above steps, please refer to Figure 3 , Figure 3 It is a schematic diagram of belt deviation and material offset detection for material transportation.

[0027] Figure 3 The left and right roller reference points, belt edge points, and material edge points in the above steps are shown in the figure. The two curved lines in the figure represent the material.

[0028] Please refer to Figure 4 , Figure 4The process of selecting the reference point in step S41 is shown, and the green dots in the figure represent the selected reference points.

[0029] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the belt deviation detection effect without material transportation.

[0030] Please refer to Figure 6 , Figure 6 It is a schematic diagram of the belt deviation and material offset detection effect when transporting materials.

[0031] See also Figure 7 , Figure 7 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: a belt and material deviation detection device 401, a processor 402 and a storage medium 403.

[0032] A belt and material deviation detection device 401: The belt and material deviation detection device 401 implements the belt and material deviation detection method.

[0033] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the belt and material deviation detection method.

[0034] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the belt and material deviation detection method.

[0035] The beneficial effects of the present invention are as follows: a new detection method is proposed. By adopting image segmentation technology based on deep learning and combining it with a machine vision system, accurate monitoring of belt deviation and material offset is achieved without direct contact with the belt, while improving the accuracy and real-time performance of the detection. In addition, the present invention is also committed to solving the problem that material offset detection mainly relies on manual monitoring, and proposes an automated and efficient detection method, thereby realizing intelligent monitoring and management of the production process, and providing important technical support and guarantee for industrial production. Compared with traditional belt deviation and material offset detection methods, this method has high detection accuracy and good feasibility.

[0036] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A belt and material deviation detection method, characterized in that: The method comprises the following steps: S1. Use industrial cameras to obtain belt images with and without material transportation at different times; S2, label the collected images with labels and divide them into two categories: belts and materials to obtain the constructed data set; S3, constructing a belt and material segmentation model, and using the constructed data set to train the belt and material segmentation model to obtain a trained model; S4. Use an industrial camera to obtain real-time images, input the real-time images into the trained model, and obtain the offset judgment results.

2. A belt and material deviation detection method as claimed in claim 1, characterized in that: The belt and material segmentation model processes the real-time image as follows: S41, determining four points from the real-time image, the first two points are used to determine the reference line of the left belt roller, and the last two points are used to determine the reference line of the right belt roller; S42, taking the average value of the y coordinates of the four points, and determining a straight line passing through the entire image; S43, inputting the real-time image into the belt and material segmentation model to obtain a belt and material mask coverage instance; S44, determining the first and last contact points between the mask covering instance and the straight line as the edge points of the belt and the material; S45, comparing the edge point of the belt with the left roller reference line and the right roller reference line to determine whether the belt is deviating; S46, taking the average value of the edge points to obtain the midpoint of the belt and the material; S47. Compare the midpoint of the material with the midpoint of the belt to determine whether the material is offset.

3. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a belt and material deviation detection method as described in any one of claims 1 to 2.

4. A belt and material deviation detection device, characterized in that: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement a belt and material deviation detection method as described in any one of claims 1 to 2.

Images