Belt tearing detection method based on multi-task deep learning network

Abstract

The invention discloses a belt tearing detection method based on a multi-task deep learning network, and the method comprises the steps: firstly projecting laser to the surface of a belt, and then collecting the image data of the belt in real time through a camera; carrying out augmentation and preprocessing on the image data, and adding a light strip image with noise at the same time; extracting a laser stripe center point as a data label by using an improved gray gravity center algorithm; a U-Net network is improved, a Dropout layer is added after coding and decoding of each layer, then different weights are set for network training, and an improved MU-Net model is obtained; fitting the center point of the laser line output by the full connection layer by using a linear difference method to obtain a light stripe center line, and taking the light stripe center line as a final output result of the stripe center line; and analyzing the center lines of the stripes, and identifying different defects of belt tearing by judging the distance between two adjacent pixel points and the included angle between the tangent lines of the adjacent pixels. According to the invention, real-time monitoring of the state of the belt is realized, and powerful support is provided for preventive maintenance of industrial equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a belt tear detection method based on a multi-task deep learning network. Background Art

[0002] With the rapid development of industrialization, belt drive systems have been widely used in various types of mechanical equipment, including conveyors, cranes, manufacturing production lines, etc. As a transmission medium, the belt plays an important role, and its performance directly affects the operating efficiency and safety of the equipment. Since the belt is subjected to high-intensity loads and friction during long-term operation, its physical properties gradually deteriorate, and it is prone to failures such as tearing and wear. This may not only lead to stagnation of the production line, but also cause serious safety accidents. Therefore, it is particularly important to detect the status of the belt in a timely and effective manner, especially when it is torn and damaged.

[0003] At present, the traditional belt tear detection methods mainly include mechanical switch detection, binocular vision detection and temperature monitoring.

[0004] Mechanical switch detection is achieved by installing a tear protection switch under the belt. When the belt is torn or material leaks, the switch will be triggered and an alarm will sound. Although this method is simple to operate, it cannot detect in time. Binocular visual detection installs cameras on both sides of the belt to present a full picture to enhance the comprehensiveness and accuracy of detection, but it is greatly affected by ambient light. Temperature monitoring uses a thermal sensor to monitor the temperature changes of the belt in real time. Tearing or friction will cause the temperature to rise, thereby sounding an alarm. Although this method is effective, there may be a risk of false alarms in high temperature environments.

[0005] In addition, there are visual inspection and vibration analysis. Although visual inspection can detect surface problems to a certain extent, it is highly subjective and easily affected by the operator's experience and judgment, resulting in missed or false detections. Although vibration analysis can provide some early signals of mechanical failures, its sensitivity to belt tearing is limited, and it is easily disturbed by environmental noise in practical applications. These methods are not only time-consuming and labor-intensive, but also have defects such as strong subjectivity and low accuracy.

[0006] At present, the bottom detection scheme is also a common belt tear detection method, and its advantages include real-time monitoring and high degree of automation. This scheme can monitor the belt status in real time and realize automatic control through connection with control systems such as PLC, thereby reducing manual intervention. However, the bottom detection scheme also has its shortcomings: first, it can only detect when the belt has a through tear or leakage, and cannot identify potential defects on the material conveying surface in advance, resulting in early damage being ignored; second, this scheme usually relies on the triggering of the alarm system, and may respond only after the tear occurs, causing a delay in response, resulting in potential faults not being discovered in time, increasing the maintenance cost and safety hazards of the equipment. Summary of the invention

[0007] 1. Technical problem to be solved by the invention

[0008] The present invention aims to provide a belt tear detection method based on a multi-task deep learning network. The method monitors the running status of the belt in real time and identifies the mutation points of the laser projection line on the belt by combining deep learning with traditional algorithms. Such mutation points are usually a precursor to belt tearing or damage. Timely detection of these anomalies helps to provide early warning, thereby reducing production stagnation and economic losses caused by equipment failure.

[0009] 2. Technical solution

[0010] In order to achieve the above object, the technical solution provided by the present invention is:

[0011] A belt tear detection method based on a multi-task deep learning network of the present invention comprises the following steps:

[0012] Step 1: Build a line structured light projection device, collect laser stripe images, and perform image preprocessing;

[0013] Step 2: For the line structured light projection fringe image obtained by preprocessing, the laser center line coordinate points are extracted as label data using the improved grayscale centroid method;

[0014] Step 3: Create an image dataset for the multi-task network;

[0015] Step 4: Using the U-Net network as the benchmark model, build a multi-task network and train the network to obtain the extraction results of the line structured light stripe area and the laser center line;

[0016] Step 5: Analyze the extraction results of the line structured light stripe area and the laser center line to determine whether the belt is damaged or broken, and issue an early warning.

[0017] Furthermore, in step 1, a line structured light projection device is constructed using a direct laser triangulation method. In the data preprocessing stage, the data set is expanded by a variety of means, including rotation, flipping, and adding line structured light stripe images with different noises.

[0018] Furthermore, in step 2, firstly, median filtering is performed on the line structured light projection stripes in the preprocessed image to eliminate noise; then, morphological expansion and corrosion are used to eliminate the noise at the boundary of the line structured light projection stripes; and each column of the line structured light projection stripe image is scanned to find the pixel position x with the maximum gray value in the column i. i And gray value I(x i ), under the premise that there is only one laser line in an image, if there are multiple maximum grayscale values ​​extracted for a column of the laser line, let x i =(x1-x2) / 2, where x1 is the first position where the maximum gray value appears, and x2 is the second position where the maximum gray value appears; at the same time, set the threshold Thresh, if I(x i ) is less than the set threshold Thresh, the column is considered to be interference at the edge of the laser and is directly skipped to further remove noise.

[0019] Furthermore, in step 2, after denoising the line structured light projection fringes, the improved grayscale centroid method is used to extract the coordinate points of the laser center line. Specifically, a weight factor K is introduced to calculate the maximum grayscale value I(x i ) takes K pixel values ​​on both sides of the upper and lower sides for weighted average, and calculates the i -K to x i The center of gravity position G' of +K is specifically:

[0020]

[0021] Save the extracted line structured light center coordinate point G' as the data label of the multi-task network.

[0022] Furthermore, in step 3, the extraction of the line structured light stripe area is used as task one of the multi-task network, and the extraction of the laser center line is used as task two of the multi-task network; Lableme is used to label the laser area of ​​the data set augmented in step 1 as the label data of network task one; and the center point extracted in step 2 is used as the label data of network task two.

[0023] Furthermore, in step 4, the U-Net network is used as the baseline model, and a dropout layer is added after each encoding layer and decoding layer; at the same time, the network output layer is changed from a single-layer output to a two-layer output, a multi-task network is constructed, and the weight ratio of task one and task two is set to 4:6, and the learning rate is 0.01.

[0024] Furthermore, step 5 makes a preliminary estimate of the line structured light stripe area obtained by the output of task 1, and preliminarily determines the area where the belt may be damaged or broken.

[0025] Furthermore, in step 5, the n stripe center points outputted from task 2 are fitted by linear interpolation to obtain the laser stripe center line as the final laser center line extraction result.

[0026] Furthermore, in step 5, a threshold M is set. First, the Euclidean distance between adjacent center line pixel points is calculated and compared with the threshold M. If the distance between adjacent laser center points is greater than M, it is considered that a break occurs there.

[0027] Furthermore, step 5 detects partial damage to the belt by detecting the angle between the tangents of two adjacent pixel points on the laser center line. First, the tangents between adjacent pixel points are calculated, and then the angle between adjacent tangents is calculated. A threshold α is set. If the angle is greater than α, it is considered that partial damage has occurred there.

[0028] 3. Beneficial effects

[0029] Compared with the existing known technologies, the technical solution provided by the present invention has the following significant effects:

[0030] (1) Automatically label the line structured light projection stripe dataset to reduce the interference of ambient light

[0031] The present invention adopts an improved grayscale centroid algorithm to quickly extract the coordinate point data of the laser centerline and label the extracted data set. This process enhances the robustness of the detection, while reducing the data set labeling time and improving processing efficiency.

[0032] (2) Automatically extract the laser area and laser stripe centerline using an improved multi-task learning framework

[0033] The present invention improves the U-Net network, adds a Dropout layer and an output layer, shares learning tasks, obtains a multi-task learning framework, combines multiple laser scenes and shapes, extracts the laser area and the center line of the laser stripe at the same time, improves the accuracy and robustness of defect detection, and the speed and accuracy of belt tear detection, and reduces system complexity and maintenance costs. The algorithm based on deep learning can effectively identify minor anomalies of the belt during operation and reduce false positives and false negatives.

[0034] (3) Effective belt damage or breakage detection

[0035] The present invention can detect belt breaks by analyzing the distance between adjacent pixel points of the center line of the laser projection stripe. At the same time, the present invention innovatively proposes to detect belt defects by judging the change of the tangent angle of adjacent pixel points. If the change exceeds a certain threshold, the laser stripe has a sudden change at this location, that is, the belt is damaged. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 (a) and (b) are experimental device diagrams of the present invention;

[0037] Figure 2 is a data processing flow chart of the present invention;

[0038] Figure 3 Schematic diagram of the multi-task neural network structure;

[0039] Figure 4 A schematic diagram of a data set collected for the present invention;

[0040] Figure 5 (a) and (b) are comparison diagrams of the processing results of the improved grayscale centroid algorithm and the traditional grayscale centroid algorithm;

[0041] Figure 6 (a) and (b) are schematic diagrams of the results of task one and task two extracted by the multi-task network;

[0042] Figure 7 This is the belt defect detection result diagram (breakage);

[0043] Figure 8 This is the belt defect detection result diagram (damaged).

[0044] Explanation of the symbols in the schematic diagram:

[0045] 1. Bracket; 2. Laser; 3. Camera; 4. Laser plane; 5. Object to be measured; 6. Reference surface. DETAILED DESCRIPTION

[0046] The present invention proposes a belt tear detection method based on a multi-task deep learning network. The scheme combines deep learning with traditional image processing steps, automatically detects belt tear defects through neural network training and learning, and constructs an intelligent monitoring method. The operation steps include: first, a line laser and a high-speed industrial array CMOS camera are installed above the belt to collect the projection image of the laser line on the belt in real time, and obtain the line laser projection line image data of the belt surface; secondly, the collected image is pre-processed, the data is augmented and the noise is removed to enhance the wide range of applicable environments; then, the laser ROI is obtained, focusing on a specific area of ​​the belt, and the center point of the laser line extracted by the improved grayscale centroid method is used as the label data, and the data set is trained using the improved multi-task neural network to obtain the training model, and then the test set is predicted to identify potential signs of tearing, such as wear and cracks; finally, through the preset alarm mechanism, an alarm is issued to the operator in time, so as to achieve accurate monitoring and early warning of the belt status.

[0047] Through the framework of multi-task learning, not only can the laser area be extracted, but also the center line of the line structured light projection stripes can be quickly obtained, which ensures accuracy while greatly improving the running speed. It can also timely warn of possible belt tearing, so that maintenance personnel can better understand the running status of the belt and formulate a reasonable maintenance plan. The technical solution proposed by the present invention can effectively alleviate the problems of the existing methods being sensitive to changes in illumination and large noise interference, and can accurately identify defects in complex backgrounds.

[0048] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments.

[0049] Example 1

[0050] Step 1: Build a line structured light projection device using direct laser triangulation

[0051] First, use the direct laser triangulation method to build a laser projection system. Figure 1 , fix the laser 2 on the bracket 1 and place it just above the reference surface 6 (in belt tear detection, the reference surface and the belt surface coincide), the camera 3 is located next to the laser 2, the object to be tested 5 is placed just below the laser 2, and the laser plane 4 acts on the belt surface and the object to be tested 5 of different shapes. This configuration can ensure that the laser line is accurately projected onto the belt surface and a clear image is captured by the camera for subsequent analysis.

[0052] Step 2: Collect laser stripe images

[0053] Using a line laser to illuminate the surface of the belt and the surfaces of objects of different shapes makes it easy to collect line structured light stripe images of different materials and shapes as a data set. When the belt is torn or damaged, the line structured light projection line will change suddenly. At this time, the industrial camera can collect the line structured light stripe image. By processing and analyzing these images, the damage or tearing of the belt can be effectively detected.

[0054] Step 3: Data Preprocessing

[0055] In the data preprocessing stage, the data set is expanded by rotating, flipping, and adding line structured light stripe images with different noises to enhance the robustness and generalization of the detection results. See the attached Figure 4 .

[0056] Step 4: Use the laser centerline coordinate points extracted by the improved grayscale centroid method as label data

[0057] (1) Median filtering is performed on the line structured light projection stripes to eliminate noise;

[0058] (2) Using morphological dilation and corrosion to further eliminate the noise at the edge of the line structured light projection stripe;

[0059] (3) Scan each column of the line structured light projection stripe image and find the pixel position x with the maximum gray value in column i i And gray value I(x i ), under the premise that there is only one laser line in an image, if there are multiple maximum grayscale values ​​extracted for a column of the laser line, let x i =(x1-x2) / 2, where x1 is the first position where the maximum grayscale value appears, and x2 is the second position where the maximum grayscale value appears.

[0060] (4) Set the threshold Thresh. If I(x i ) is less than the set minimum threshold Thresh, the column is considered to be interference at the edge of the laser, and the column is directly skipped to remove noise;

[0061] (5) The specific formula for calculating the center of gravity is as follows:

[0062]

[0063] Among them, ∑ i x i ·I(x i ) is the sum of the products of all pixel positions with the maximum gray value and the corresponding maximum gray value, also called weighted sum, ∑ i I(x i) is the sum of the grayscale values ​​of all pixels with the maximum grayscale value. In order to further improve the accuracy and stability, the centroid formula is improved by introducing the weight factor K. The maximum grayscale value I(x i ) takes K pixel values ​​on both sides of the upper and lower sides for weighted average, and calculates the i -K to x i The center of gravity position G' of +K is specifically:

[0064]

[0065] In the formula, K = 7, Thresh = 100, and the extracted laser center line is compared with the traditional grayscale algorithm. Figure 5 ,It can be seen that the extraction effect of the improved grayscale centroid algorithm is better;

[0066] (6) The line structured light center coordinate point G' finally extracted by formula (2) is saved in an Excel file as the data label of task 2 in step 5.

[0067] Step 5: Create an image dataset for the multi-task network

[0068] (1) Divide the data set obtained in step 3 into a ratio of 8:1:1 to obtain a training set, a test set, and a validation set;

[0069] (2) Extracting the laser area is used as the first task of the multi-task network, and extracting the laser centerline is used as the second task of the multi-task network;

[0070] (3) Use Lableme to label the laser regions of the augmented dataset in step 3 as label data for network task 1;

[0071] (4) The center point extracted in step 4 is used as the label data for task 2 of network training;

[0072] Step 6: Improve the multi-task network to obtain the laser centerline extraction result

[0073] (1) Using the U-Net network as the benchmark model, a dropout layer is added after each encoding layer and decoding layer to reduce the number of network parameters and prevent overfitting;

[0074] (2) Change the network output layer, change the single-layer output to a two-layer output, and turn it into a multi-task training network. See the attached Figure 3 ;

[0075] (3) Set the weight ratio of task 1 and task 2 to 4:6, set the learning rate to 0.01, use the training set samples as input to train the improved U-Net network, and obtain the multi-task network model M-UNet;

[0076] (4) The test set is input into the obtained M-UNet model to obtain the center point detection results of the laser line. The results of Task 1 and Task 2 extracted by the multi-task neural network are shown in the attached Figure 6 The left picture is the extracted line structured light area, and the right picture is the result of center line extraction of the line structured light area.

[0077] Step 7: Pre-test the extraction results of Task 1

[0078] Make a preliminary estimate of the line structured light stripe area obtained from the output of Task 1 to preliminarily determine the area where the belt may be damaged or broken;

[0079] Step 8: Perform curve fitting on the extraction results of Task 2

[0080] The n stripe center points obtained by the output of task 2 are fitted by linear interpolation method to obtain the light stripe center line as the final laser center line extraction result;

[0081] Step 9: Analyze the extraction results to detect belt tears

[0082] (1) Belt tearing is divided into direct breakage and partial breakage;

[0083] (2) Detect direct belt breakage by detecting the breakpoints of the laser line. Set the threshold M, first calculate the Euclidean distance between adjacent center line pixels, and compare it with the threshold M. If the distance between adjacent laser center points is greater than M, it is considered that a break has occurred there, and a box is drawn as a mark;

[0084] (3) Detect partial belt damage by detecting the angle between the tangents of two adjacent pixels on the laser center line. First, calculate the tangents between adjacent pixels, then calculate the angle between adjacent tangents, set a threshold α, and if the angle is greater than α, it is considered that partial damage has occurred at that location, and a box is drawn as a mark. See the attached figure for details. Figure 7 , 8 .

[0085] Step 10: Take early warning

[0086] According to the belt tearing condition detected in step 8, early warning operation is performed to remind the staff to take precautions to reduce potential safety hazards and production losses.

[0087] In summary, the multi-task network method for belt tear detection based on deep learning provided by the present invention can combine multi-task learning, data fitting and break point detection in complex data processing scenarios, quickly respond to mutation points, and thus achieve real-time monitoring of the belt status, which not only improves the safety and reliability of the belt, but also provides strong support for the preventive maintenance of industrial equipment. The promotion of this solution will greatly reduce production stagnation and economic losses caused by equipment failures, and improve overall production efficiency.

[0088] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A belt tear detection method based on a multi-task deep learning network, characterized in that: The steps are: Step 1: Build a line structured light projection device, collect laser stripe images, and perform image preprocessing; Step 2: For the line structured light projection fringe image obtained by preprocessing, the laser center line coordinate points are extracted as label data using the improved grayscale centroid method; Step 3: Create an image dataset for the multi-task network; Step 4: Using the U-Net network as the benchmark model, build a multi-task network and train the network to obtain the extraction results of the line structured light stripe area and the laser center line; Step 5: Analyze the extraction results of the line structured light stripe area and the laser center line to determine whether the belt is damaged or broken, and issue an early warning.

2. The belt tear detection method based on a multi-task deep learning network according to claim 1 is characterized in that: Step 1 uses direct laser triangulation to build a line structured light projection device. In the data preprocessing stage, the data set is expanded by various means, including rotation, flipping, and adding line structured light stripe images with different noises.

3. A belt tear detection method based on a multi-task deep learning network according to claim 1 or 2, characterized in that: Step 2: First, median filter the line structured light projection stripes in the preprocessed image to eliminate noise; then use morphological expansion and corrosion to eliminate the noise at the edge of the line structured light projection stripes; and scan each column of the line structured light projection stripe image to find the pixel position x with the maximum gray value in column i. i And gray value I(x i ), under the premise that there is only one laser line in an image, if there are multiple maximum grayscale values ​​extracted for a column of the laser line, let x i =(x1-x2) / 2, where x1 is the first position where the maximum gray value appears, and x2 is the second position where the maximum gray value appears; at the same time, set the threshold Thresh, if I(x i ) is less than the set threshold Thresh, the column is considered to be interference at the edge of the laser and is directly skipped to further remove noise.

4. The belt tear detection method based on a multi-task deep learning network according to claim 3 is characterized in that: Step 2: After denoising the line structured light projection fringes, the improved grayscale centroid method is used to extract the coordinate points of the laser center line. Specifically, the weight factor K is introduced to calculate the maximum grayscale value I(x i ) takes K pixel values ​​on both sides of the upper and lower sides for weighted average, and calculates the i -K to x i The center of gravity position G' of +K is specifically: Save the extracted line structured light center coordinate point G' as the data label of the multi-task network.

5. The belt tear detection method based on multi-task deep learning network according to claim 4 is characterized in that: In step 3, the extraction of the line structured light stripe area is used as the first task of the multi-task network, and the extraction of the laser center line is used as the second task of the multi-task network; Labelme is used to label the laser area of ​​the data set augmented in step 1 as the label data of network task one; the center point extracted in step 2 is used as the label data of network task two.

6. The belt tear detection method based on multi-task deep learning network according to claim 5 is characterized in that: In step 4, the U-Net network is used as the baseline model, and a dropout layer is added after each encoding layer and decoding layer. At the same time, the network output layer is changed from a single-layer output to a two-layer output to build a multi-task network, and the weight ratio of task one and task two is set to 4:6, and the learning rate is 0.

01.

7. The belt tear detection method based on multi-task deep learning network according to claim 6 is characterized in that: Step 5 makes a preliminary estimate of the line structured light stripe area obtained from the output of Task 1, and preliminarily determines the area where the belt may be damaged or broken.

8. The belt tear detection method based on multi-task deep learning network according to claim 7 is characterized in that: Step 5: Fit the n stripe center points outputted by Task 2 using a linear interpolation method to obtain the laser stripe center line as the final laser center line extraction result.

9. The belt tear detection method based on multi-task deep learning network according to claim 8, characterized in that: Step 5 sets the threshold M. First, the Euclidean distance between adjacent center line pixel points is calculated and compared with the threshold M. If the distance between adjacent laser center points is greater than M, it is considered that a break occurs there.

10. The belt tear detection method based on multi-task deep learning network according to claim 8, characterized in that: Step 5 detects partial damage of the belt by detecting the angle between the tangents of two adjacent pixels of the laser center line. First, the tangents between adjacent pixels are calculated, and then the angle between adjacent tangents is calculated. A threshold α is set. If the angle is greater than α, it is considered that partial damage has occurred there.

Images