Method for Generating Infrared Image Data of Longitudinal Tearing of Belt Conveyor

By extracting longitudinal tear features on healthy infrared images and combining denoising diffusion probability model, multiple types of longitudinal tear image data are generated, which solves the problem that existing methods are difficult to obtain high-quality longitudinal tear image data sets, and improves longitudinal tear Dataset quality for crack fault diagnosis.

CN118411360BActive Publication Date: 2024-11-08WUHAN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410844635.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-08
Estimated Expiration
2044-06-27

AI Technical Summary

Technical Problem

It is difficult for existing methods to obtain large batches, high-quality longitudinal tear image datasets of belt conveyors, resulting in poor generalization performance of monitoring methods based on infrared imaging and deep learning in actual industrial scenarios.

Method used

By randomly selecting the longitudinal tear occurrence area on healthy infrared images, longitudinal tear features of different sizes are extracted, and repairing and data enhancement are combined with the denoising diffusion probability model to generate multi-type longitudinal tear image data.

Benefits of technology

High-quality and multi-type expansion of longitudinal tear data sets with few samples is achieved, and the quality of longitudinal tear fault diagnosis is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118411360B_ABST
    Figure CN118411360B_ABST
Patent Text Reader

Abstract

The present invention provides a method for generating infrared image data of longitudinal tearing of a belt conveyor, comprising the following steps: step S1: randomly selecting a longitudinal tearing region on a healthy infrared image; step S2: intercepting longitudinal tearing features of different sizes from the longitudinal tearing feature image of the original belt conveyor, and using the longitudinal tearing features to fill the longitudinal tearing region to obtain a synthetic image; step S3: cropping the feature edge of the synthetic image, and repairing it using a denoising diffusion probability model trained with the original longitudinal tearing infrared image of the belt conveyor; step S4: mixing the repaired synthetic image with the original longitudinal tearing infrared image of the belt conveyor, and sampling and data enhancement after secondary training of the denoising diffusion probability model to generate image data. The method realizes high-quality and multi-type expansion of the longitudinal tearing data set under the condition of few samples.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of fault data generation, and in particular to a method for generating longitudinal tear infrared image data of a belt conveyor. Background Art

[0002] Belt conveyors are widely used in ports due to their advantages of high transportation efficiency, low cost and good scalability. They are important equipment for port transshipment and transportation. As the main component, the cost of the conveyor belt accounts for more than half of the total cost. Conveyor belts, especially those operating in harsh environments, are easily affected by various factors. Improper installation and adjustment, damage by foreign objects or obstacles, impact of falling sharp objects, and uneven load distribution may lead to failures, including belt deviation, surface damage and longitudinal tearing. If these faults are not discovered and resolved in time, they may cause damage to equipment, waste of materials, and even cause major safety accidents with great economic losses and casualties. Among them, longitudinal tearing has attracted much attention from enterprises due to its high frequency and serious impact. Therefore, effective online monitoring of the conveyor belt status is of great significance to improving the safety and economic benefits of ports.

[0003] The monitoring method based on infrared imaging and deep learning is currently a more mainstream method, and it has shown excellent performance in the task of monitoring longitudinal tearing of conveyor belts. However, the excellent performance of this method depends to a large extent on large-scale, high-quality image data sets. However, in the actual port industrial environment, managers cannot allow equipment to operate in an abnormal state for a long time, so most of the collected image data are data under healthy conditions, and the amount of image data reflecting the longitudinal tearing of the equipment is small, that is, there is a data imbalance problem. The diagnostic model trained with an unbalanced data set has poor generalization performance and is difficult to apply to actual industrial scenarios. Therefore, studying how to obtain large-scale, high-quality longitudinal tear image data sets is of great significance to promote the application of monitoring methods based on infrared imaging and deep learning in actual scenarios. Summary of the invention

[0004] The present invention proposes a method for generating infrared image data of longitudinal tearing of a belt conveyor, so as to solve the technical problem that it is difficult to obtain large-scale and high-quality longitudinal tearing image data sets in existing methods.

[0005] In order to solve the above technical problems, the present invention provides a method for generating infrared image data of longitudinal tearing of a belt conveyor, comprising the following steps:

[0006] Step S1: randomly selecting a longitudinal tearing region on a healthy infrared image;

[0007] Step S2: extracting longitudinal tear features of different sizes from the original longitudinal tear feature image of the belt conveyor, and using the longitudinal tear features to fill the longitudinal tear occurrence area to obtain a composite image;

[0008] Step S3: cutting the feature edges of the composite image and repairing it using a denoising diffusion probability model trained with the original longitudinally torn infrared image of the belt conveyor;

[0009] Step S4: the repaired synthetic image is mixed with the original longitudinal tear infrared image of the belt conveyor, and the denoising diffusion probability model is trained twice and then sampled for data enhancement to generate image data.

[0010] Preferably, the method for extracting the longitudinal tearing characteristic image of the original belt conveyor comprises:

[0011] Step S21: converting the infrared image into a grayscale image;

[0012] Step S22: applying the LoG edge detection algorithm to the grayscale image to extract an edge image;

[0013] Step S23: acquiring the object contour of the edge image, and selecting the contour with the largest area as the edge contour of the longitudinal tearing area;

[0014] Step S24: creating a mask based on the edge contour, performing a bitwise AND operation on the original belt conveyor longitudinal tear infrared image and the mask to obtain a longitudinal tear feature image.

[0015] Preferably, step S22 includes: extracting local features of the grayscale image to obtain a Gaussian kernel corresponding to each pixel for smoothing; applying the Laplace operator for calculation; detecting zero crossing points in the image after Laplace transformation, and using Hough transformation on the processed edges to connect the edges to obtain a final connected edge image.

[0016] Preferably, gradient amplitude and texture features are selected as local features.

[0017] Preferably, the Gaussian kernel corresponding to each pixel is The expression is:

[0018] ;

[0019] ;

[0020] In the formula, Represents the pixel coordinates, represents the standard deviation considering the gradient amplitude and texture features, represents the gradient amplitude, Represents texture features, , and Indicates adjustment parameters.

[0021] Preferably, when detecting the zero crossing point, threshold processing is performed using the following formula:

[0022] ;

[0023] In the formula, represents the binary edge image, is the set threshold, Represents the image after Laplace transformation.

[0024] Preferably, the threshold This is determined by:

[0025] Calculate the local mean and the local standard deviation , whose expression is:

[0026] ;

[0027] ;

[0028] In the formula, Represents pixel Neighborhood of Represents the number of pixels in the neighborhood. Represents the gray value of the pixel in the area;

[0029] The threshold is calculated by the following formula:

[0030] ;

[0031] In the formula, , and Indicates adjustment parameters.

[0032] Preferably, the loss function of the denoising diffusion probability model is The expression is:

[0033] ;

[0034] In the formula, Indicates The weight of the scale, It is in The image at each scale is in time step The expression, represents the actual noise, Indicates that the model The time step of the scale Predicted noise.

[0035] Preferably, the denoising diffusion probability model in step S5 is iterated during the sampling process using the following formula:

[0036] ;

[0037] In the formula, Represents the time step image data, Represents the time step The scaling factor, Represents the time step The cumulative scaling factor within Represents the model at time step The predicted noise, Represents the time step The standard deviation of the noise, represents noise sampled from a standard Gaussian distribution.

[0038] Preferably, the structural similarity index and the general quality index between the generated infrared image and the original infrared image are calculated, and an infrared image that satisfies the threshold requirements of the two indices is selected as the generated image.

[0039] The beneficial effects of the present invention include at least: the present invention extracts features from a small number of original longitudinal tear images, combines the obtained longitudinal tear features with images under healthy working conditions to obtain multi-type longitudinal tear images, and mixes the obtained multi-type longitudinal tear images with the original longitudinal tear infrared images for training DDPM, so as to achieve high-quality, multi-type expansion of the longitudinal tear data set with few samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a longitudinal tear feature extraction method according to an embodiment of the present invention;

[0042] Figure 3 It is a schematic diagram of a method for synthesizing various longitudinal tear infrared images according to an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of a longitudinal tearing image enhancement method based on a denoising diffusion probability model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a method for generating infrared image data of longitudinal tearing of a belt conveyor, comprising the following steps:

[0046] Step S1: Randomly select a longitudinal tearing area on a healthy infrared image.

[0047] Step S2: extracting longitudinal tear features of different sizes from the original longitudinal tear feature image of the belt conveyor, and using the longitudinal tear features to fill the longitudinal tear occurrence area to obtain a composite image.

[0048] Specifically, the embodiment of the present invention also proposes a method for extracting a longitudinal tearing feature image. Figure 2 As shown, firstly, the original infrared image of the longitudinal tear of the belt conveyor is converted into a grayscale image to facilitate subsequent processing. The embodiment of the present invention assumes that the acquired infrared image is a pseudo-color image and needs to be converted into grayscale. If the infrared image is already a grayscale image, the subsequent steps can be directly performed;

[0049] ;

[0050] In the formula, represents the image after grayscale conversion, , , Represents the red, green, and blue channels of the image in pixels The value at .

[0051] The LoG edge detection algorithm is applied to the grayscale converted image to highlight the longitudinal tearing features in the image. The LoG edge detection algorithm includes the following processes:

[0052] First, the grayscale image is subjected to Gaussian smoothing processing. In order to reduce the noise in the image and better retain the details in the image, the embodiment of the present invention further uses adaptive Gaussian smoothing technology to process the grayscale image, and obtains the Gaussian kernel corresponding to each pixel point according to the local features of the image. When selecting local features, the gradient amplitude and texture features of the image are selected as local features for illustration in the embodiment of the present invention.

[0053] Calculate the gradient amplitude and texture features of each pixel separately:

[0054] Gradient Magnitude: ;

[0055] Texture features are represented by local binary patterns (LBP):

[0056] ;

[0057] In the formula, is a sign function, It's a pixel Corresponding gray value, Represents pixel The gray value of the pixel in the neighborhood, Represents pixel The number of pixels in the neighborhood.

[0058] Calculate the standard deviation by considering the gradient amplitude and texture features :

[0059] ;

[0060] In the formula, , and is the adjustment parameter.

[0061] Then the Gaussian kernel corresponding to each pixel is:

[0062] ;

[0063] ;

[0064] In the formula, represents the image after Gaussian smoothing, represents the adaptive Gaussian filter kernel, represents the standard deviation of the adaptive Gaussian filter kernel, Represents a convolution operation.

[0065] Then the Laplace operator is applied for calculation;

[0066] ;

[0067] In the formula, represents the image after Laplace transformation, represents the Laplace operator.

[0068] Finally, zero crossing points are detected in the Laplace transformed image, and the processed edges are connected using Hough transform to obtain the final connected edge image. In order to enhance the effect of zero crossing points in the embodiment of the present invention, threshold processing is performed on the zero crossing points, where the zero crossing points satisfy the following formula:

[0069] ;

[0070] The threshold processing formula is:

[0071] ;

[0072] In the formula, represents the binary edge image, is the set threshold.

[0073] In order to further improve edge accuracy and more significantly enhance the contrast of edge areas in the embodiment of the present invention, the local mean and standard deviation of the image are selected to determine the threshold:

[0074] Local mean: ;

[0075] Local standard deviation: ;

[0076] In the formula, Represents pixel Neighborhood of Represents the number of pixels in the neighborhood. Represents the grayscale value of the pixel in the area.

[0077] Then use the exponential function to calculate the adaptive threshold:

[0078] ;

[0079] In the formula, , and is the adjustment parameter.

[0080] Call the 'findContours' function in OpenCV to find the object contours in the edge image, and select the contour with the largest area as the edge of the longitudinal tear area;

[0081] Finally, a mask is created and used to extract the longitudinal tear feature image.

[0082] Specifically, a mask is created, that is, a matrix with the same size as the original image is created, and the selected contour and the pixel positions inside the contour are set to 1 in the matrix, and the remaining pixel positions are set to 0; the original longitudinal tearing infrared image and the mask are bitwise ANDed, and the result is an image containing only the longitudinal tearing features, and the expression is:

[0083] ;

[0084] In the formula, represents an image containing only longitudinal tear features, represents the original longitudinally torn infrared image, represents the mask created, Represents a bitwise AND operation.

[0085] Finally, the extracted longitudinal tear features are combined with the collected healthy infrared images to synthesize longitudinal tear infrared images with different sizes and random damaged areas, e.g. Figure 3 shown.

[0086] Specifically, the steps include:

[0087] Step S21: From the image containing only longitudinal tear features The rectangular range is intercepted in the expression:

[0088] ;

[0089] In the formula Indicates the intercepted rectangular range. and They respectively represent the coordinates of the upper left corner and the lower right corner of the intercepted rectangle. The intercepted rectangle should just contain the longitudinal tearing feature area.

[0090] Step S22: In the embodiment of the present invention, a scale factor is introduced and the randomness of the tearing area is considered to obtain various longitudinal tearing images, and the expression is:

[0091] ;

[0092] In the formula, Longitudinal tear characteristics indicating dimensional change, Represents the scale factor.

[0093] Step S23: combining the acquired longitudinal tear features of different sizes with the healthy infrared image of the randomly determined longitudinal tear occurrence area.

[0094] Step S3: The feature edges of the synthesized image are cropped and repaired using a denoising diffusion probability model trained on the original longitudinally torn infrared image of the belt conveyor.

[0095] In the embodiment of the present invention, in order to further improve the overall image generation quality, make the generated image take into account both global structure and local details, and enhance the versatility of the model, a multi-scale diffusion model loss function is designed for the denoising diffusion probability model. Four versions of different scales (original resolution, half, one quarter and one eighth of the original resolution) are generated for each input image. For each scale image, noise is gradually added and denoising prediction is performed according to the steps of the diffusion model. The loss of the denoising process is calculated at each scale to ensure the image generation quality at each scale. The designed multi-scale diffusion model loss function is expressed as follows:

[0096] ;

[0097] In the formula, Indicates The weight of the scale, It is in The image at each scale is in time step The expression, represents the actual noise, Indicates that the model The time step of the scale Predicted noise.

[0098] In order to better fuse the healthy infrared image and the extracted longitudinal tear feature image, the embodiment of the present invention crops the feature edges in the synthesized longitudinal tear infrared image, and then uses the denoising diffusion probability model DDPM to repair the cropped area. Based on DDPM's effective learning of the distribution characteristics of infrared image data, a more fused, diverse and natural synthetic image can be obtained.

[0099] Step S4: The repaired synthetic image is mixed with the original longitudinal tear infrared image of the belt conveyor, and the denoising diffusion probability model is trained twice and then sampled for data enhancement to generate image data.

[0100] Specifically, Figure 4 As shown in the figure, the repaired synthetic image is mixed with the original infrared image, the DDPM is further trained, and then the DDPM is sampled to enhance the infrared image under the required longitudinal tearing state. The iterative process of the DDPM is as follows:

[0101] ;

[0102] In the formula, Represents the time step image data, Represents the time step The scaling factor, Represents the time step The cumulative scaling factor within Represents the model at time step The predicted noise, Represents the time step The standard deviation of the noise, represents noise sampled from a standard Gaussian distribution.

[0103] In order to further improve the data quality of the longitudinal tear fault diagnosis data set in the embodiment of the present invention, the generated infrared images are screened, and the infrared images that meet the screening conditions are included in the final longitudinal tear fault diagnosis data set.

[0104] Specifically, the structural similarity index and general quality index between the infrared image generated by the calculation model and the original infrared image are calculated, and the infrared images that meet the threshold requirements of the two indicators are included in the infrared image enhancement dataset. The screening expression is as follows:

[0105] ;

[0106] ;

[0107] In the formula, represents the original infrared image, represents the infrared image generated by the model, and Represent the mean of the original infrared image and the generated infrared image respectively, and Respectively represent the variance of the original infrared image and the generated infrared image, represents the covariance of the original infrared image and the generated infrared image, and represents a constant, and They represent the thresholds set for the structural similarity index SSIM and the universal quality index UQI respectively.

[0108] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. Only the preferred embodiments of the present invention are expressed. The description is more specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. As long as there is no contradiction in the combination of these technical features, they should be considered as the scope recorded in this specification.

[0109] It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these modifications and improvements all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for generating infrared image data of longitudinal tearing of a belt conveyor, characterized in that: The following steps are involved: Step S1: randomly selecting a longitudinal tearing region on a healthy infrared image; Step S2: extracting longitudinal tear features of different sizes from the original longitudinal tear feature image of the belt conveyor, and using the longitudinal tear features to fill the longitudinal tear occurrence area to obtain a composite image; Step S3: cutting the feature edges of the synthetic image and repairing it using a denoising diffusion probability model trained with the original longitudinally torn infrared image of the belt conveyor; Step S4: the repaired composite image and the original longitudinal tear infrared image of the belt conveyor are mixed, and the denoising diffusion probability model is trained twice and then sampled for data enhancement to generate image data; The method for extracting the longitudinal tearing characteristic image of the original belt conveyor comprises: Step S211: converting the infrared image into a grayscale image; Step S212: applying the LoG edge detection algorithm to the grayscale image to extract an edge image; Step S213: acquiring the object contour of the edge image, and selecting the contour with the largest area as the edge contour of the longitudinal tearing area; Step S214: creating a mask based on the edge contour, performing a bitwise AND operation on the original belt conveyor longitudinal tear infrared image and the mask to obtain a longitudinal tear feature image; The method of intercepting longitudinal tear features of different sizes and using the longitudinal tear features to fill the longitudinal tear occurrence area to obtain a synthetic image comprises the following steps: Step S21: From the image containing only longitudinal tear features The rectangular range is intercepted in the expression: ; In the formula Indicates the intercepted rectangular range. and They represent the coordinates of the upper left corner and the lower right corner of the intercepted rectangle, respectively. The intercepted rectangle should just contain the longitudinal tear feature area. Step S22: introducing a scale factor and considering the randomness of the tearing area to obtain various longitudinal tearing images, the expression is: ; In the formula, Longitudinal tear characteristics indicating dimensional change, represents the scale factor; Step S23: combining the acquired longitudinal tear features of different sizes with the healthy infrared image of the randomly determined longitudinal tear occurrence area to obtain a composite image.

2. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 1, characterized in that: Step S212 includes: extracting local features of the grayscale image to obtain a Gaussian kernel corresponding to each pixel for smoothing; applying the Laplace operator for calculation; detecting zero crossing points in the image after Laplace transformation, using Hough transformation on the processed edges to connect the edges, and obtaining a final connected edge image.

3. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 2, characterized in that: Gradient amplitude and texture features are selected as local features.

4. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 3, characterized in that: The Gaussian kernel corresponding to each pixel The expression is: ; ; In the formula, Represents the pixel coordinates, represents the standard deviation considering the gradient amplitude and texture features, represents the gradient amplitude, Represents texture features, , and Indicates adjustment parameters.

5. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 2, characterized in that: When detecting the zero crossing point, threshold processing is performed using the following formula: ; In the formula, represents the binary edge image, is the set threshold, Represents the image after Laplace transformation.

6. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 5, characterized in that: The threshold This is determined by: Calculate the local mean and the local standard deviation , whose expression is: ; ; In the formula, Represents pixel Neighborhood of Represents the number of pixels in the neighborhood. Represents the gray value of the pixel in the area; The threshold is calculated by the following formula: ; In the formula, , and Indicates adjustment parameters.

7. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 1, characterized in that: The loss function of the denoising diffusion probability model is The expression is: ; In the formula, Indicates The weight of the scale, It is in The image at each scale is in time step The expression, represents the actual noise, Indicates that the model The time step of the scale Predicted noise.

8. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 1, characterized in that: The denoising diffusion probability model in step S4 is iterated during the sampling process using the following formula: ; In the formula, Represents the time step image data, Represents the time step The scaling factor, Represents the time step The cumulative scaling factor within Represents the model at time step The predicted noise, Represents the time step The standard deviation of the noise, represents noise sampled from a standard Gaussian distribution.

9. The method for generating infrared image data of longitudinal tearing of a belt conveyor according to claim 1, characterized in that: The structural similarity index and the general quality index between the image data generated in step S4 and the original infrared image are calculated, and the image data that satisfies the threshold requirements of the two indices are selected as the generated image.