A belt tear detection method based on a multi-task deep learning network

Abstract

The application discloses a kind of based on multi-task deep learning network's belt tear detection method, first laser is projected on the surface of belt, then utilize camera real-time acquisition belt image data;Image data is augmented and preprocessed, while adding light bar image with noise;With the improved gray gravity center algorithm, the center point of laser stripe is extracted as data label;Improved U-Net network, after encoding and decoding in each layer, increase Dropout layer, then set different weights to carry out network training, obtain improved MU-Net model;The center point of laser line output by fully connected layer is fitted to obtain light stripe center line as the final output result of stripe center line using linear difference method;The stripe center line is analyzed, and the distance between adjacent two pixel points and the included angle of adjacent pixel tangent are judged to identify different defects of belt tear.The application realizes the instant monitoring of belt state, and provides strong support for preventive maintenance of industrial equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and particularly relates to a belt tearing detection method based on a multi-task deep learning network. BACKGROUND

[0002] With the rapid development of industrialization, belt transmission systems have been widely used in various mechanical equipment, including conveyors, cranes, manufacturing production lines, etc. As a transmission medium, the belt plays an important role, and its performance directly affects the running efficiency and safety of the equipment. Due to the high strength load and friction that the belt bears during long-term operation, its physical properties gradually deteriorate, and it is prone to tearing, wear and other faults. This not only may cause the production line to stall, but also may cause serious safety accidents. Therefore, it is particularly important to detect the state of the belt, especially the tearing and damage, in a timely and effective manner.

[0003] Currently, the traditional belt tearing detection methods mainly include mechanical switch detection, binocular vision detection and temperature monitoring.

[0004] Mechanical switch detection is achieved by installing a tearing protection switch under the belt. When the belt tears or material leaks, the switch will be triggered and an alarm will be issued. Although this method is simple to operate, it cannot detect in a timely manner. Binocular vision detection is to install cameras on both sides of the belt to present a full view to enhance the comprehensiveness and accuracy of detection, but it is greatly affected by environmental light. Temperature monitoring monitors the temperature changes of the belt in real time through a thermal sensor. Tearing or friction will cause the temperature to rise, thereby issuing an alarm. Although this method is effective, there may be a risk of false positives in high-temperature environments.

[0005] In addition, there are visual inspection and vibration analysis. Visual inspection can find surface problems to some extent, but it is highly subjective and easily affected by the experience and judgment ability of the operator, leading to missed or false detections. Although vibration analysis can provide some early signals of mechanical failure, its sensitivity to belt tearing is limited, and it is easily disturbed by environmental noise in actual application. These methods not only are time-consuming and labor-intensive, but also have defects such as high subjectivity and low accuracy.

[0006] Currently, the bottom detection scheme is also a common method for belt tear detection, and its advantages include real-time monitoring and high automation. This scheme can monitor the belt status in real time and achieve automatic control through connection with control systems such as PLC, thereby reducing manual intervention. However, the bottom detection scheme also has its shortcomings: first, it can only detect when the belt has a through tear or material leakage, and cannot identify potential defects on the material conveying surface in advance, resulting in early damage being ignored; second, this scheme usually relies on the triggering of an alarm system, which may respond only after the tear occurs, causing a delay in response and potential failure to be discovered in time, increasing the maintenance cost and safety hazards of the equipment. SUMMARY

[0007] 1. Technical problems to be solved by the invention

[0008] The present invention aims to provide a belt tear detection method based on a multi-task deep learning network, which monitors the running state of the belt in real time and identifies the mutation points of the laser projection line on the belt through the combination of deep learning and traditional algorithms. These mutation points are usually precursors of belt tears or damage, and timely detection of these abnormalities helps to provide early warnings, thereby reducing production stoppages and economic losses caused by equipment failures.

[0009] 2. Technical solutions

[0010] To achieve the above-mentioned purposes, the technical solutions provided by the present invention are as follows:

[0011] A belt tear detection method based on a multi-task deep learning network according to the present invention comprises the following steps:

[0012] Step 1: Build a line structured light projection device, collect laser stripe images, and perform image preprocessing;

[0013] Step 2: Extract the laser centerline coordinate points as label data using the improved gray gravity center method for the line structured light projection stripe image obtained after preprocessing;

[0014] Step 3: Create an image dataset for the multi-task network;

[0015] Step 4: Build a multi-task network based on the U-Net network as the benchmark model, and train the network to obtain the extraction results of the line structured light stripe region and the laser centerline;

[0016] Step 5: Analyze the extraction results of the line structured light stripe region and the laser centerline to determine the condition of belt damage or rupture and provide a warning.

[0017] Further, step 1 uses direct laser triangulation to build a line structured light projection device. In the data preprocessing stage, the data set is expanded by various means, including rotation, flipping, and adding different noise line structured light stripe images.

[0018] Further, step 2 first performs median filtering on the line structured light projection stripes in the preprocessed image to eliminate noise. Then, morphological dilation and erosion are used to eliminate noise at the boundaries of the line structured light projection stripes. Each column of the line structured light projection stripe image is scanned to find the pixel position x i and the gray value I(x i ) of the maximum gray value in the column i. Under the premise that there is only one laser line in the image, if there are multiple maximum gray values extracted from the laser line, then x i = (x1-x2) / 2, where x1 is the first position where the maximum gray value appears, and x2 is the second position where the maximum gray value appears. At the same time, a threshold Thresh is set. If I(x i ) is less than the set threshold Thresh, it is considered that the column belongs to the interference at the edge of the laser, and the column is directly skipped to further remove noise.

[0019] Further, after denoising the line structured light projection stripes in step 2, the laser center line coordinates are extracted using an improved gray centroid method. Specifically, a weight factor K is introduced, and K pixel values on both sides of the maximum gray value I(x i ) of each column are weighted and averaged to calculate the centroid position G' from x i -K to x i +K. Specifically:

[0020]

[0021] The extracted line structured light center coordinates G' are saved as the data labels of the multi-task network.

[0022] Further, step 3 takes the extraction of the line structured light stripe region as task one of the multi-task network and takes the extraction of the laser center line as task two of the multi-task network. Lableme is used to label the laser region of the augmented data set in step 1 as the label data for network task one. The center points extracted in step 2 are used as the label data for network task two.

[0023] Further, step 4 uses the U-Net network as the baseline model, adds a dropout layer after each encoding layer and decoding layer, and changes the network output layer from a single layer output to a two-layer output. The multi-task network is built, and the weight ratio of task one and task two is set to 4:6, and the learning rate is set to 0.01.

[0024] Furthermore, step 5 makes a preliminary estimate based on the line structured light stripe area obtained from the output of task one, and makes a preliminary judgment on the areas where the belt may be damaged or broken.

[0025] Furthermore, in step 5, the n stripe center points obtained from Task 2 are fitted using linear interpolation to obtain the laser stripe center line, which is then used as the final laser center line extraction result.

[0026] Furthermore, in step 5, a threshold M is set. First, the Euclidean distance between adjacent centerline pixels is calculated and compared with the threshold M. If the distance between adjacent laser center points is greater than M, it is considered that a break has occurred at that point.

[0027] Furthermore, step 5 detects partial damage to the belt by detecting the angle between the tangents of two adjacent pixels on the laser center line. First, the tangents between adjacent pixels are calculated, then the angle between the adjacent tangents is calculated, and a threshold α is set. If the angle is greater than α, then partial damage is considered to have occurred at that location.

[0028] 3. Beneficial effects

[0029] Compared with existing known technologies, the technical solution provided by this invention has the following significant advantages:

[0030] (1) Automatically label the line structured light projection stripe dataset to reduce interference from ambient light.

[0031] This invention employs an improved grayscale centroid algorithm to quickly extract the coordinate point data of the laser centerline. The extracted dataset is then labeled, which enhances the robustness of detection and reduces the dataset labeling time, thereby improving processing efficiency.

[0032] (2) Automatically extract the laser region and laser stripe center line using an improved multi-task learning framework.

[0033] This invention improves the U-Net network by adding a Dropout layer and an output layer, sharing the learning task to obtain a multi-task learning framework. Combining various laser scenes and shapes, it extracts the laser region and the center line of the laser stripe simultaneously, improving the accuracy and robustness of defect detection, as well as the speed and precision of belt tear detection, while reducing system complexity and maintenance costs. Furthermore, the deep learning-based algorithm can effectively identify minor anomalies in the belt during operation, reducing false alarms and missed alarms.

[0034] (3) Effective belt damage or breakage detection

[0035] This invention can detect belt breakage by analyzing the distance between adjacent pixels on the center line of the laser projection stripe. In addition, this invention innovatively proposes to detect belt defects by judging the change in the angle between the tangents of adjacent pixels. If the change exceeds a certain threshold, then the laser stripe has abruptly changed at that point, that is, the belt has broken. Attached Figure Description

[0036] Figure 1 (a) and (b) in the figure are experimental apparatus diagrams of the present invention;

[0037] Figure 2 This is a data processing flowchart of the present invention;

[0038] Figure 3 This is a schematic diagram of a multi-task neural network structure;

[0039] Figure 4 This is a schematic diagram of the dataset collected in this invention;

[0040] Figure 5 (a) and (b) in the figure are comparison diagrams of the processing results of the improved gray-scale centroid algorithm and the traditional gray-scale centroid algorithm;

[0041] Figure 6 (a) and (b) in the figure are schematic diagrams of the results of Task 1 and Task 2 extracted by the multi-task network;

[0042] Figure 7 Image showing the results of belt defect detection (fracture);

[0043] Figure 8 This is a diagram showing the results of belt defect detection (damage).

[0044] Explanation of the labels in the diagram:

[0045] 1. Support; 2. Laser; 3. Camera; 4. Laser plane; 5. Object to be tested; 6. Reference plane. Detailed Implementation

[0046] This invention proposes a belt tear detection method based on a multi-task deep learning network. This scheme combines deep learning with traditional image processing steps, automatically detecting belt tears through neural network training and learning, thus constructing an intelligent monitoring method. The operational steps include: First, a line laser and a high-speed industrial area array CMOS camera are installed above the belt to acquire real-time images of the laser line projection on the belt, obtaining line laser projection image data of the belt surface; second, the acquired images are preprocessed to augment the data and remove noise, enhancing the applicability to various environments; then, the laser ROI is obtained and focused on a specific area of ​​the belt, using the laser line center point extracted by an improved grayscale centroid method as label data, and the dataset is trained using an improved multi-task neural network to obtain a training model, which is then used to predict the test set and identify potential signs of tearing, such as wear and cracks; finally, a preset alarm mechanism is used to promptly issue an alarm to the operator, thereby achieving accurate monitoring and early warning of the belt condition.

[0047] By employing a multi-task learning framework, not only can the laser region be extracted, but the center line of the line structured light projection fringes can also be quickly obtained. This ensures accuracy while significantly improving operating speed. Furthermore, it provides timely warnings of potential belt tearing, enabling maintenance personnel to better understand the belt's operating status and develop appropriate maintenance plans. Moreover, the technical solution proposed in this invention effectively alleviates the problems of existing methods being sensitive to changes in lighting and experiencing significant noise interference, and can accurately identify defects in complex backgrounds.

[0048] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings and embodiments.

[0049] Example 1

[0050] Step 1: Construct a line structured light projection device using the direct laser triangulation method.

[0051] First, a laser projection system is constructed using the direct-light laser triangulation method. See appendix for details. Figure 1 The laser 2 is fixed to the bracket 1 and placed directly above the reference surface 6 (which coincides with the belt surface in belt tear detection). The camera 3 is located next to the laser 2, and the test object 5 is placed directly below the laser 2. The laser plane 4 acts on the belt surface and the test object 5 of different shapes. This configuration ensures that the laser line is accurately projected onto the belt surface and that the camera captures a clear image, facilitating subsequent analysis.

[0052] Step 2: Acquire laser stripe image

[0053] By illuminating the surface of the belt and the surfaces of objects of different shapes with a line laser, it is easy to collect line structured light stripe images of different materials and shapes as a dataset. When the belt tears or is damaged, the line structured light projection lines will change abruptly. At this time, the industrial camera can collect line structured light stripe images. By processing and analyzing these images, the damage or tearing of the belt can be effectively detected.

[0054] Step 3: Data Preprocessing

[0055] In the data preprocessing stage, the dataset is expanded by methods such as rotation, flipping, and adding line structured light stripe images with different noise levels to enhance the robustness and generalization of the detection results. See the appendix for details. Figure 4 .

[0056] Step 4: Use the coordinates of the laser centerline extracted by the improved grayscale centroid method as the label data.

[0057] (1) Median filtering is applied to the line structured light projection stripes to eliminate noise;

[0058] (2) Morphological dilatational erosion is used to further eliminate noise at the boundaries of linear structured light projection stripes;

[0059] (3) Scan each column of the line structured light projection fringe image and find the position x of the pixel with the maximum gray value in column i. i and grayscale value I(x) i Given an image with only one laser line, if there are multiple maximum grayscale values ​​for a certain column extracted from that laser line, then let x... i = (x1-x2) / 2, where x1 is the first position where the maximum gray value appears, and x2 is the second position where the maximum gray value appears.

[0060] (4) Set the threshold Thresh, if I(x) i If the value is less than the set minimum threshold Thresh, then the column is considered to be interference at the laser edge and is skipped directly to remove noise.

[0061] (5) The specific formula for calculating the centroid is as follows:

[0062]

[0063] Where, ∑ i x i ·I(x i ∑ is the sum of the products of the positions of all pixels with the highest gray values ​​and their corresponding highest gray values, also called the weighted sum. i I(x iThe sum of gray values ​​of all pixels with the highest gray values ​​is denoted as I(x). To further improve accuracy and stability, the centroid formula is improved by introducing a weighting factor K, which is applied to the maximum gray value I(x) of each column. i Take K pixel values ​​from both the top and bottom sides of x and perform a weighted average to calculate the value from x. i -K to x i The centroid position G' of +K is as follows:

[0064]

[0065] In the formula, K=7 and Thresh=100. The extracted laser center line is compared with that of the traditional grayscale algorithm. See the appendix for details. Figure 5 As can be seen, the improved grayscale centroid algorithm has a better extraction effect;

[0066] (6) Save the final line structured light center coordinate point G' extracted using formula (2) to an Excel file as the data label for task 2 in step 5.

[0067] Step 5: Create an image dataset for a multi-task network

[0068] (1) Divide the dataset obtained in step 3 into an 8:1:1 ratio to obtain a training set, a test set, and a validation set;

[0069] (2) The extraction of the laser region is taken as the first task of the multi-task network, and the extraction of the laser center line is taken as the second task of the multi-task network.

[0070] (3) Use Lableme to label the laser regions of the augmented dataset in step 3, and use them as the label data for network task one;

[0071] (4) Use the center points extracted in step 4 as the label data for task 2 of network training;

[0072] Step 6: Improve the multi-task network to obtain the laser centerline extraction result.

[0073] (1) Using the U-Net network as the baseline model, dropout layers are added after each encoding and decoding layer to reduce the number of network parameters and prevent overfitting;

[0074] (2) Modify the network output layer, changing the single-layer output to a two-layer output, thus creating a multi-task training network. See Appendix for details. Figure 3 ;

[0075] (3) Set the weight ratio of Task 1 and Task 2 to 4:6, set the learning rate to 0.01, and use the training set samples as input to train the improved U-Net network to obtain the multi-task network model M-UNet;

[0076] (4) Input the test set into the obtained M-UNet model to obtain the center point detection results of the laser line. The results of Task 1 and Task 2 extracted by the multi-task neural network are shown in the appendix. Figure 6 The left image shows the extracted line structured light region, and the right image shows the result of extracting the center line of the line structured light region.

[0077] Step 7: Perform pre-testing on the extraction results of Task 1

[0078] The linear structured light stripe area obtained from Task 1 is initially estimated to determine the area where the belt may be damaged or broken.

[0079] Step 8: Perform curve fitting on the extraction results of Task 2.

[0080] The n fringe center points obtained from Task 2 are fitted using linear interpolation to obtain the light fringe center line, which is used as the final laser center line extraction result.

[0081] Step 9: Analyze the extraction results to detect belt tears.

[0082] (1) The belt tear is divided into direct breakage and partial damage;

[0083] (2) The direct breakage of the belt is detected by detecting the break point of the laser line. Set a threshold M, first calculate the Euclidean distance between adjacent center line pixels and compare it with the threshold M. If the distance between adjacent laser center points is greater than M, it is considered that a break has occurred at that point, and a box is drawn as a marker.

[0084] (3) Damage to the belt is detected by measuring the angle between the tangents of two adjacent pixels along the laser center line. First, the tangents between adjacent pixels are calculated. Then, the angle between the adjacent tangents is calculated, and a threshold α is set. If the angle is greater than α, partial damage is considered to have occurred at that location, and a box is drawn as a marker. See Appendix for details. Figure 7 , 8 .

[0085] Step 10: Issue an early warning

[0086] Based on the belt tear detected in step 8, an early warning operation will be performed to remind staff to take precautions and reduce potential safety hazards and production losses.

[0087] In summary, the deep learning-based multi-task network method for belt tear detection provided by this invention can combine multi-task learning, data fitting, and break point detection in complex data processing scenarios to quickly respond to abrupt changes, thereby achieving real-time monitoring of belt conditions. This not only improves the safety and reliability of belts but also provides strong support for preventative maintenance of industrial equipment. The widespread adoption of this solution will significantly reduce production stoppages and economic losses caused by equipment failures, thereby improving overall production efficiency.

[0088] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the figures shown are only one embodiment of the present invention; the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.

Claims

1. A belt tear detection method based on a multi-task deep learning network, characterized in that, The steps are as follows: Step 1: Set up a line structured light projection device, acquire laser stripe images, and perform image preprocessing; Step 2: For the preprocessed line structured light projection fringe image, extract the coordinates of the laser centerline as label data using the improved gray-scale centroid method; specifically: introduce a weighting factor. The maximum grayscale value for each column Take K pixel values ​​from both the top and bottom sides and perform a weighted average to calculate the value from... arrive center of gravity , The row coordinates of the pixel corresponding to the maximum grayscale value in each column; specifically: Save the extracted coordinates of the line structured light center point , as data labels for multi-task networks; Step 3: Create the image dataset for the multi-task network; extract the linear structured light stripe region as task one of the multi-task network, and extract the laser center line as task two of the multi-task network; use Labelme to label the laser region of the augmented dataset in step 1 as the label data for network task one; use the center point extracted in step 2 as the label data for network task two. Step 4: Using the U-Net network as the baseline model, add dropout layers after each encoding and decoding layer; at the same time, change the network output layer from a single output layer to a two-layer output layer to build a multi-task network and train the network, setting the weight ratio of Task 1 and Task 2 to 4:6 and the learning rate to 0.01; obtain the extraction results of the line structured light stripe region and the laser center line. Step 5: Analyze the extraction results of the line structured light stripe region and the laser center line to determine the condition of belt damage or breakage, and issue an early warning. Specifically: Based on the linear structured light stripe area obtained from Task 1, a preliminary estimate is made to determine the areas where the belt may be damaged or broken. The n stripe center points obtained from Task 2 are fitted using linear interpolation to obtain the laser stripe center line, which is then used as the final laser center line extraction result. Set a threshold M. First, calculate the Euclidean distance between adjacent centerline pixels and compare it with the threshold M. If the distance between adjacent laser center points is greater than M, it is considered that a break has occurred at that point. The damage to the belt is detected by detecting the angle between the tangents of two adjacent pixels on the laser center line. First, the tangents between adjacent pixels are calculated, then the angle between the adjacent tangents is calculated, and a threshold α is set. If the angle is greater than α, then the belt is considered to be partially damaged.

2. The belt tear detection method based on a multi-task deep learning network according to claim 1, characterized in that: Step 1 uses the direct laser triangulation method to build a line structured light projection device. In the data preprocessing stage, the dataset is expanded through several expansion methods, including rotation, flipping, and adding line structured light stripe images with different noise levels.

3. A belt tear detection method based on a multi-task deep learning network according to claim 1 or 2, characterized in that: Step 2 first applies median filtering to the line structured light projection fringe image region in the preprocessed image to eliminate noise; then, morphological dilatation and erosion are used to eliminate noise at the line structured light projection fringe boundaries; and finally, each column of the line structured light projection fringe image is scanned to find the row coordinates of the pixel corresponding to the maximum gray value in each column. The maximum grayscale value for each column is... This indicates that, where the subscript i represents the column coordinate, if there are multiple maximum grayscale values ​​for a column extracted from an image with only one laser line, then let (the expression is missing from the original text). In the formula This is the first position where the maximum grayscale value appears. This is the second position where the maximum grayscale value appears; simultaneously, a threshold is set. ,like Less than the set threshold If the column is considered to be interference at the laser edge, it is skipped to further remove noise.

Images