A conveyor belt wear identification method based on contrast learning

Abstract

A kind of conveying belt wear identification method based on contrast learning, comprising the following steps: S1, training sample picture is carried out contrast learning training by N MobileConvNeXt feature extraction module and T SwinTransformer feature extraction module, attention block in series;S2, classification head is increased after attention block AttentionBlock, the classification head includes two layers of MLP, sigmoid function, and the sigmoid function outputs binary classification result;S3, enter the prediction stage, set prediction confidence threshold, the picture and thermal map acquisition equipment are fixed at certain angle, and the device needing to be collected is fixed-point photographed, the RGB three-channel picture of corresponding device and the corresponding single-channel thermal map are acquired, and the fusion picture of two is input into the model obtained in step S2 to carry out gas overflow classification prediction.The method can extract features based on unsupervised mode, reduces the artificial labeling cost, and can extract more rich feature information, and the method based on self-supervised learning can better accelerate the model training convergence speed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of conveyor belt wear identification, and particularly relates to a conveyor belt wear identification method based on contrast learning. BACKGROUND

[0002] During continuous operation of the conveyor belt, different degrees of wear will occur. When the conveyor belt breaks without monitoring, the goods on the conveyor belt will be damaged, and accidents will occur. Therefore, it is urgent to automatically monitor the conveying of the equipment system. The existing technologies are as follows:

[0003] 1. (201510392423.7) A conveyor belt wear identification method based on contrast learning

[0004] The patent discloses a conveyor belt wear identification method based on contrast learning, which comprises the following steps: firstly, collecting real-time images to obtain a gray image, using an edge detector to detect the edges of objects in the gray image, then filtering the edge image based on a shape and direction filter template to obtain a straight edge image in the same direction as the running direction of the conveyor belt, using a one-dimensional vector to mark the edges, and determining whether the conveyor belt is torn according to the known reference width and design width threshold of the conveyor belt when the conveyor belt is not torn. The method detects the tearing of the conveyor belt based on a traditional image detection algorithm. The method only detects the tearing of the conveyor belt at a fixed angle and detects the tearing based on the edges of the conveyor belt, and cannot detect the tearing of the center part of the conveyor belt, which leads to the inability to identify most of the tearing of the center part of the conveyor belt.

[0005] 2. (201510469098.X) Conveyor belt surface damage automatic detection method

[0006] The invention discloses a conveyor belt surface damage automatic detection method, which collects conveyor belt running images using a linear array camera, uses the consistency of the same column of the image to establish a suppression row vector, performs edge fidelity denoising on the mean curve of the row and column vectors, extracts damage feature parameters, and quickly identifies the surface damage. The method uses a traditional image feature extraction method, which can only detect edge damage, cannot detect different conveyor belts at different angles, and has certain limitations.

[0007] 3. (202011171382.9) A method for detecting conveyor belt wear based on a detection system and a conveyor belt

[0008] The method discloses a detection conveying belt wear method based on a detection system and a conveying belt, relates to the technical field of material conveying, and the conveying belt comprises a first conveying device and a second conveying device, and the first conveying device and the second conveying device are reversely connected through a rotating shaft assembly; wherein the first conveying device comprises a first conveying section and a second conveying section, a first protective plate is arranged on the front and rear sidewalls of the first conveying section, the second conveying section is connected with the second conveying device, a second protective plate is arranged on the front and rear sidewalls of the second conveying device, the length of the second protective plate is less than the length of the second conveying section, compared with the prior art, the method in the application can monitor the guide roller in real time, and when the guide roller is stuck or the rotation of the guide roller is abnormal, the staff can find it in time. The application needs to add a protective device, and a large amount of funds and installation costs are needed, which is not suitable for large-scale deployment. SUMMARY

[0009] In order to solve the above technical problems, the application provides a conveying belt wear identification method based on contrast learning, and the specific technical scheme is as follows:

[0010] A conveying belt wear identification method based on contrast learning comprises the following steps:

[0011] S1, training sample pictures are subjected to contrast learning training through N MobileConvNeXt feature extraction modules and T SwinTransformer feature extraction modules and an attention block;

[0012] S2, a classification head is added after the attention block AttentionBlock, the classification head comprises two layers of MLPs and a sigmoid function, the two layers of MLPs are connected through a relu activation function, and the sigmoid function outputs a binary classification result;

[0013] S3, entering a prediction stage, setting a prediction confidence threshold, fixing a picture and a thermal map acquisition device at a certain angle to fixed-point photograph the device to be acquired, acquiring an RGB three-channel picture of the corresponding device and a corresponding single-channel thermal map, and inputting the fusion picture into the model obtained in step S2 to perform gas overflow classification prediction.

[0014] Specifically, step S1 is specifically

[0015] S11, features and are obtained through N MobileConvNeXt feature extraction modules and T SwinTransformer feature extraction modules;

[0016] S12, in the attention block AttentionBlock, the output features and are input into the attention block AttentionBlock. ​​​Features are obtained by merging using the concate module. and features Features are obtained by inputting into a VAN network. .

[0017] Specifically, N cascaded MobileConvNeXt feature extraction modules obtain features. The specific steps are as follows:

[0018] SA111: First, the training sample images are passed through a 3x3 convolutional submodule. Then, the features obtained from the convolution are input into the layernorm submodule to obtain the features. ;

[0019] SA112, will Features are obtained by inputting into a spatially separable convolutional submodule. The spatially separable convolutional submodule has a convolutional kernel of 4 and a stride of 4.

[0020] SA113, Features The feature map is obtained by inputting it into the ASPP submodule. ;feature The feature maps are obtained by inputting the dilated convolutional units with dilation rates of 6, 12, and 18, respectively. , , Meanwhile, features The feature map is obtained by successively passing it through a 1x1 max pooling unit, a first 1x1 convolutional unit, and an upsampling unit. ; Feature map , , and feature map After the first merging unit, the feature map is obtained through the second 1x1 convolution unit. ;

[0021] SA114, Feature Map Features are obtained by inputting into the first depthwise separable convolutional submodule. , will feature The graph is input into a linear submodule for 4-fold dimensionality upscaling to obtain features. Then the features Input to activation function Obtain features Finally, this feature Multiplying by theta yields the features ;

[0022] SA115, Features With features the final output feature of the network after merging by the second merging submodule ;

[0023] SA116, replacing the feature in step SA115 as input, repeating steps SA111-SA115 to obtain the feature SA115, repeating steps SA111-SA115, obtaining the feature , and repeating N times until the feature .

[0024] Specifically, the SwinTransformer feature extraction module comprises a patch partition submodule and T SwinTransformerBlock submodules; the steps are specifically as follows:

[0025] SB111, inputting the training sample picture into the patch partition submodule to obtain picture patches, and then inputting the picture patches into a linear embedding layer to obtain picture token features ;

[0026] SB112, inputting the feature into the T SwinTransformerBlock submodules in series to obtain the feature .

[0027] Specifically, step SB112 is specifically as follows:

[0028] SB1121, taking as input, sequentially passing through a first LN unit and a W-MSA unit to obtain a feature , and merging the feature ;

[0029] SB1122, taking the feature as input, sequentially passing through a second LN unit and a first MLP unit to obtain a feature , and merging the feature ;

[0030] SB1123, taking the feature as input, sequentially passing through a third LN unit and a SW-MSA unit to obtain a feature , and merging the feature ;

[0031] SB1124, taking the feature as input, sequentially passing through a fourth LN unit and a second MLP unit to obtain a feature , and merging the feature as the feature ;

[0032] SB1125, The replacement step SB1121 is input, the repeating steps SB1121-SB1124 are repeated T times, and the last to feature is extracted .

[0033] Specifically, the step S12 is specifically as follows:

[0034] S121, the VAN network inputs the feature through the operation of the downsampling submodule, and the feature is input in sequence through the third 1x1 convolution submodule, the first BatchNormalization submodule and the GELU activation function, and the feature is obtained.

[0035] S122, the feature is input to the LKA submodule to obtain the feature .

[0036] S123, the feature is input in sequence to the second BatchNormalization submodule and the FFN submodule, and the feature obtained by the second BatchNormalization submodule and the FFN submodule is added to the feature , and finally the feature is obtained.

[0037] Specifically, the feature is obtained by the following process: the feature is input to the second deep separable convolution submodule, and the output result of the second deep separable convolution submodule is input to the deep separable dilated convolution submodule, and finally the feature is obtained by inputting the feature to the fourth 1x1 convolution submodule. The feature is multiplied by the input feature , and the feature obtained by the multiplication is added to the input feature to obtain the feature .

[0038] Specifically, the loss function used by the classification model established in the step S1 is an infoNCE loss function.

[0039] Specifically, the loss function used by the model established in the step S2 is a cross-entropy loss function.

[0040] The advantages of the present application are:

[0041] (1) The present application proposes a method of contrast learning for extracting image features, which can extract features based on an unsupervised manner, reduces the cost of manual labeling in the feature extraction stage, and can extract more rich feature information, and the method based on self-supervised learning can better accelerate the model training convergence speed.

[0042] (2) Only based on the shooting information, the conveyor belt at different angles can be detected, without other equipment, the damage of any part of the conveyor belt edge and center can be detected at the same time, and the damage classification can be performed through any damaged place of the conveyor belt.

[0043] (3) The present application discloses a CSNet contrastive neural network based on convolution and transformer combination for feature extraction, and finally accesses a classification head to obtain a prediction model ContrastiveClassificationNet based on the damage identification task. BRIEF DESCRIPTION OF DRAWINGS

[0044] Fig. 1 is a whole flowchart.

[0045] Fig. 2 is a flowchart for obtaining features.

[0046] Fig. 3 is a flowchart for obtaining features. DETAILED DESCRIPTION

[0047] A conveyor belt wear identification method based on contrastive learning includes the following steps:

[0048] S1, using N MobileConvNeXt feature extraction modules and T SwinTransformer feature extraction modules in series and AttentionBlock for contrastive learning training, the specific steps are as follows:

[0049] As shown in Figs. 1-3 , S11, the training sample picture is extracted through N MobileConvNeXt feature extraction modules and T SwinTransformer feature extraction modules in series, and the corresponding features and are obtained.

[0050] The specific steps of N MobileConvNeXt feature extraction modules to obtain features are as follows:

[0051] SA111, the training sample picture is first passed through a 3*3 convolution submodule, and then the features obtained by convolution are input into a layernorm submodule to obtain features .

[0052] SA112, input into a spatial separable convolution submodule to obtain features , the convolution kernel of the spatial separable convolution submodule is 4, and the step is 4.​​

[0053] SA113, the feature input into the ASPP submodule to get the feature map ; the feature respectively input into the hole convolution unit with the hole rate of 6, 12 and 18 respectively to get the feature map , , At the same time, the feature passes through the 1x1 max pooling unit, the first 1x1 convolution unit and the up-sampling unit in turn to get the feature map ; the feature map , , and the feature map are input into the first merging unit, then pass through the second 1x1 convolution unit to finally get the feature map ;

[0054] SA114, the feature map is input into the first depth separable convolution submodule to get the feature , the feature is input into the linear submodule to get the feature , then the feature is input into the activation function to get the feature , finally the feature is multiplied by theta to get the feature ;

[0055] SA115, the feature is merged with the feature through the second merging submodule to get the final output feature of the network ;

[0056] SA116, the feature in step SA115 is replaced with the training sample picture in step SA11 as input, and steps SA111-SA1115 are repeated to get the feature , which is cycled N times until the feature .

[0057] The SwinTransformer feature extraction module includes a patch partition submodule and T SwinTransformerBlock submodules; the steps are as follows:

[0058] SB111, the training sample picture is input into the patch partition submodule to get the picture patch, and then the picture patch is input into the linear embedding layer to get the picture token feature ;

[0059] SB112, replace the feature with the feature ; each of the SwinTransformerBlock sub-modules comprises a first LN unit, a W-MSA unit, a second LN unit, a first MLP unit, a third LN unit, a SW-MSA unit, a fourth LN unit, and a second MLP unit arranged in sequence, and the steps are as follows:

[0060] SB1121, replace the feature with the feature and obtain the feature ;

[0061] SB1122, replace the feature with the feature and obtain the feature ;

[0062] SB1123, replace the feature with the feature and obtain the feature ;

[0063] SB1124, replace the feature with the feature and obtain the feature ;

[0064] SB1125, replace the feature with the feature .

[0065] S12, in the attention block AttentionBlock, replace the feature with the feature to obtain the feature , and input the feature into the VAN network to obtain the feature . Specifically as follows:

[0066] S121, the VAN network replaces the input feature with the feature As input, sequentially through the third 1x1 convolution submodule, the first BatchNormalization submodule and the GELU activation function, the feature ;

[0067] S122, the feature is input to the LKA submodule to obtain the feature ; the feature is input to the second depth separable convolution submodule, and the output result thereof is input to the depth separable dilated convolution submodule, and finally input to the fourth 1x1 convolution submodule to obtain the attention feature ; the attention feature is multiplied by the input feature to obtain the feature , which is added to the input feature ;

[0068] S123, the feature is sequentially input to the second BatchNormalization submodule and the FFN submodule to obtain the feature , which is added to .

[0069] S13, the feature is output after the attention block, with the output dimension being 2x1, to obtain the output of positive and negative classification.

[0070] In the training process, a positive and negative sample pair is randomly selected each time. First, for each image, we set it as an anchor. For the positive sample, the positive sample is generated by data augmentation. The augmentation here includes random cropping, Cutout, color transformation, sobel filtering, random noise, random blur and random rotation. For the negative sample, a picture that is not a positive sample and its randomly augmented picture are selected. In this way, different positive sample pairs or negative sample pairs can be obtained by using the anchor picture and the positive sample or the negative sample. The obtained features are respectively called and features. The loss function used in the training is the infoNCE loss function, which is shown as follows:

[0071]

[0072] is the temperature coefficient.

[0073] S2, a classification head is added after the attention block, the classification head includes two layers of MLP, a sigmoid function, the two layers of MLP are connected through a relu activation function, and the sigmoid function outputs a binary classification result. This step is fine tuning based on the pre-training weight of the contrast learning in step S1, and a final classification model is trained. The loss function used in the model is a cross-entropy loss function.

[0074] S3, a prediction stage; the picture and thermal map acquisition device is fixed at a certain angle to fixed-point shooting of the device to be acquired, RGB three-channel pictures of the corresponding device and the corresponding single-channel thermal map are acquired, the two are fused into a picture, and the model obtained in step S2 is input to perform gas overflow classification prediction, a prediction confidence threshold is set to 0.75, if the confidence of the predicted gas leakage is greater than 0.75, it is considered that the device at the position has gas leakage, and an alarm needs to be sent to remind the staff to check.

[0075] The above is only a preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A conveyor belt wear identification method based on contrastive learning, characterized in that, The method comprises the following steps: S1, training sample pictures are subjected to contrast learning training through N MobileConvNeXt feature extraction modules and T SwinTransformer feature extraction modules and attention blocks in series; S2, a classification head is added after the attention block AttentionBlock, the classification head comprises two layers of MLPs, a sigmoid function, the two layers of MLPs are connected through a relu activation function, and the sigmoid function outputs a binary classification result; S3, in a prediction stage, a prediction confidence threshold is set, a picture and a thermal map acquisition device are fixed at a certain angle to take pictures of the device to be acquired, an RGB three-channel picture of the corresponding device and a corresponding single-channel thermal map are acquired, and the two are fused into a picture to be input into the model obtained in step S2 to perform gas overflow classification prediction; Step S1 is specifically S11, correspondingly obtaining features through N MobileConvNeXt feature extraction modules and T SwinTransformer feature extraction modules in series and features ; S12, in the attention block AttentionBlock, output features and features are merged by the concate module to obtain features , and features are input into the VAN network to obtain features ; N MobileConvNeXt feature extraction modules in series to obtain features The specific steps are as follows: SA111, the training sample picture is firstly passed through a 3*3 convolution submodule, and then the features obtained through convolution are input into a middle layernorm submodule to obtain features ; SA112、will input into a spatial separable convolution submodule to obtain features , the convolution kernel of the spatial separable convolution submodule is 4, and the step is 4; SA113, the feature input into the ASPP submodule to get the feature map ; the feature respectively input into the hollow convolution unit with the hollow rate of 6, 12 and 18 respectively to get the feature map , , at the same time, the feature successively through the 1x1 maximum pooling unit, the first 1x1 convolution unit and the up-sampling unit to get the feature map ; the feature map , , and the feature map after the first merging unit, through the second 1x1 convolution unit, finally get the feature map ; SA114, the feature map input into the first depth separable convolution submodule to obtain the feature , the feature , the feature , then the feature input into the activation function obtain the feature , finally multiply the feature by theta to obtain the feature ; SA115, the feature with the feature merged by the second merging submodule to the network final output feature ; SA116, obtaining the feature in step SA115 as input, repeating steps SA111-SA115 to obtain the feature , replacing the training sample picture in step SA11 as input, repeating steps SA111-SA115 to obtain the feature , repeating N times until the feature is obtained.

2. The conveyor belt wear identification method based on contrastive learning according to claim 1, characterized in that, The SwinTransformer feature extraction module comprises a patch partition sub-module and T SwinTransformerBlock sub-modules, and the steps are specifically as follows: SB111, input the training sample picture into the patch partitioning submodule to obtain a picture patch, and then input the picture patch into a linear embedding layer to obtain picture token features ; SB112, the feature input to the T SwinTransformerBlock sub-modules in series to obtain the feature .

3. The conveyor belt wear identification method based on contrastive learning according to claim 2, characterized in that, Step SB112 is specifically as follows: SB1121, will As input, the features obtained by sequentially passing through the first LN unit, the W-MSA unit, and the second LN unit are merged to obtain the final feature Merging the features obtained by sequentially passing through the first LN unit, the W-MSA unit, and the second LN unit ; SB1122, the features As input, the features obtained by sequentially passing through the second LN unit, the first MLP unit, and the features and obtain ; SB1123, will feature As input, the features obtained by sequentially passing through the third LN unit, the SW-MSA unit, and the features Merging the features ; SB1124, the features as input, through the fourth LN unit, the second MLP unit in turn get the features and features After merging, the features are obtained As features ; SB1125, will Replace step SB1121 with input, repeat steps SB1121 - SB1124, loop T times, extract last to feature .

4. The conveyor belt wear identification method based on contrast learning according to claim 1, characterized in that, Step S12 is specifically as follows: S121、the VAN network passes the input features through the operation of the down-sampling sub-module, and obtains features As input, sequentially pass through the third 1x1 convolution sub-module, the first BatchNormalization sub-module, and the GELU activation function, to obtain features ; S122, obtaining the feature input to the LKA submodule ; S123、the feature is input into the second BatchNormalization submodule, the FFN submodule in sequence, and the features obtained are added to obtain the feature .​ 5. The conveyor belt wear identification method based on contrastive learning according to claim 4, characterized in that, Feature The obtaining process is: feature The output result of the input to the second depth separable convolution submodule is input to the depth separable dilated convolution submodule, and finally input to the fourth 1x1 convolution submodule to obtain the attention feature ; the attention feature is multiplied by the input feature to obtain a feature which is added to the input feature to obtain a feature .

6. The conveyor belt wear identification method based on contrastive learning according to claim 1, characterized in that, The loss function used by the classification model established in step S1 is an infoNCE loss function.

7. The conveyor belt wear identification method based on contrastive learning according to claim 1, characterized in that, The loss function used by the model established in step S2 is a cross-entropy loss function.

Images