Lightweight pitting corrosion intelligent detection method, system and equipment based on binary image fusion and medium

Abstract

The invention discloses a light-weight pitting intelligent detection method, system and equipment based on binary image fusion and a medium. The method comprises the following steps: acquiring a pitting picture of a stainless steel sample to be detected; marking a pitting position on the pitting picture of the stainless steel sample to be detected; inputting the pitting corrosion picture of the to-be-detected sample marked with the pitting corrosion position into a pre-trained lightweight pitting corrosion detection network based on binary image fusion to obtain a pitting corrosion detection result of the to-be-detected stainless steel sample; the pre-trained lightweight pitting corrosion detection network based on binarization image fusion is established by fusing the binarization features of the binarization image into a lightweight reference detection network. Aiming at the practical demand of rapid and accurate pitting detection in engineering, and in combination with the characteristic of obvious comparison after binarization processing of the low-reflectivity corrosion pits, the binarized pitting image is fused into a lightweight detection network, so that the network feature extraction capability can be improved, the calculated amount of binarization processing is very low, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of stainless steel material pitting detection, and specifically relates to a lightweight pitting intelligent detection method, system, equipment and medium based on binary image fusion. Background Art

[0002] Pitting damage often occurs on the surface of stainless steel, especially at welds, seals and other locations. Since most pitting holes are small, it is difficult to detect them. They are usually covered by corrosion products and are not discovered until the equipment is corroded and perforated, which brings huge potential risks to the safe operation of the equipment. Traditional manual visual inspection methods are inefficient and easily affected by subjective factors.

[0003] Image detection technology based on deep learning provides a solution for the detection and analysis of pitting defects. In recent years, many people have applied deep learning image detection technology to the field of pitting defect identification, but these studies have focused on the accuracy of detection and paid little attention to the detection speed. In actual on-site inspections, computing power is often limited, so the automatic detection speed is required to be high.

[0004] Lightweight networks can meet the detection speed requirements under limited computing power, but their feature extraction capabilities are insufficient due to the size of the network. In engineering, there is an urgent need for a new method that can improve the detection accuracy of lightweight networks without significantly increasing the amount of computing power in the network. Summary of the invention

[0005] In order to overcome the problem of low detection accuracy caused by insufficient feature extraction capability of lightweight networks during pitting corrosion detection, the present invention provides a lightweight pitting corrosion intelligent detection method, system, equipment and medium based on binary image fusion. The method integrates binary image fusion features into a lightweight detection network for pitting corrosion detection, which can enhance the feature extraction capability of the lightweight detection network, thereby improving the detection accuracy of pitting corrosion.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows: A lightweight pitting intelligent detection method based on binary image fusion includes the following steps: Obtain pitting pictures of the stainless steel samples to be tested; Mark the pitting position on the pitting picture of the stainless steel sample to be tested; The pitting corrosion image of the sample to be tested with the pitting corrosion position marked is input into a pre-trained lightweight pitting corrosion detection network based on binary image fusion to obtain the pitting corrosion detection result of the stainless steel sample to be tested; Among them, the pre-trained lightweight pitting detection network based on binary image fusion is established to integrate the binary features of the binary image into the lightweight benchmark detection network.

[0007] A further improvement of the present invention is that the pre-trained lightweight pitting detection network based on binary image fusion is established by integrating the binary features of the binary image into the lightweight reference detection network, including the following steps: After annotating the pitting images of the stainless steel samples to be tested, a pitting image dataset is obtained; Binarization is performed on the pitting image in the pitting image data set to obtain a binary image; Extracting binary features of the binary image; The binary features of the binary image are integrated into the lightweight benchmark detection network, and a lightweight pitting detection network based on binary image fusion is established.

[0008] A further improvement of the present invention is that extracting the binary features of the binary image comprises the following steps: Add the input feature to the weighted feature of the binary image to obtain the binary feature of the extracted binary image; The input features are ,in and Represents the height and width of the input feature, Represents the number of channels of the input feature; the weighted feature of the binary image is the product of the weighted weight and the pitting of the binary image.

[0009] A further improvement of the present invention is that the weighted weight is obtained by the following process: a convolution operation is performed on the input feature with a convolution kernel of size 1×1 to obtain the weighted weight.

[0010] A further improvement of the present invention is that the weighted features of the binary image are calculated by the following formula:

[0011] In the formula, the binary image , Binarized image weighted features , is the weighted weight.

[0012] A further improvement of the present invention is that the lightweight benchmark detection network is a YOLO-v5s network.

[0013] A further improvement of the present invention is that the pre-trained lightweight pitting detection network based on binary image fusion is obtained through the following process: S4.1: Obtain a pitting corrosion image of the stainless steel sample to be tested, mark the pitting corrosion position on the pitting corrosion image of the stainless steel sample to be tested, and construct a pitting corrosion image data set; The K-fold cross-validation method is used to divide the pitting image dataset into K parts, each of which is used as a test set in turn, and the remaining parts are used as training sets. The ratio of training set to test set is (K-1):1; S4.2: After image standardization is performed on the training set data, the training set data is processed by random rotation, scaling, translation, cropping and perspective transformation data augmentation; the test set data is processed by image standardization; S4.3: Select the training sets in turn to train the lightweight pitting detection network based on binary image fusion, and use the test sets in turn for cross-validation to obtain the pre-trained lightweight pitting detection network based on binary image fusion.

[0014] A lightweight pitting intelligent detection system based on binary image fusion, comprising: A pitting image acquisition module is used to acquire pitting images of the stainless steel sample to be tested; A pitting position marking module is used to mark the pitting position on the pitting image of the stainless steel sample to be tested; The pitting detection result acquisition module is used to input the pitting corrosion image of the sample to be detected after marking the pitting corrosion position into a pre-trained lightweight pitting corrosion detection network based on binary image fusion to obtain the pitting corrosion detection result of the stainless steel sample to be detected; Among them, the pre-trained lightweight pitting detection network based on binary image fusion is established to integrate the binary features of the binary image into the lightweight benchmark detection network.

[0015] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the lightweight pitting intelligent detection method based on binary image fusion is implemented.

[0016] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the lightweight pitting intelligent detection method based on binary image fusion is implemented.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention combines binary image fusion with a lightweight reference detection network for the first time to form a lightweight pitting detection network based on binary image fusion to detect pitting defects. Compared with the lightweight reference detection network, the present invention has a higher detection accuracy. Since the low reflectivity of pitting leads to low imaging pixel values, the use of binary processing of pitting images can highlight the pitting characteristics, so the binary image fusion can integrate the pitting images after binary processing into the pitting detection network, enhancing the feature extraction capability of the lightweight detection network; in view of the actual demand for fast and accurate pitting detection in engineering, combined with the obvious contrast characteristics of low reflectivity pits after binary processing, the pitting images after binary processing are integrated into the lightweight detection network, which can improve the network feature extraction capability. At the same time, the computational complexity of the binary processing is very low, and the computational complexity of the network will not be greatly increased.

[0018] Furthermore, pitting pits have unique morphological characteristics, and the reflectivity of pitting pits to light is very low, which makes the pixel value of pitting pit imaging very low, while the pixel value of normal metal surface is relatively high. The image binarization adopted by the present invention is a process of presenting the entire image with obvious black and white effect. Using the binarization method to process the pitting image with obvious pixel contrast can significantly highlight the pitting characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a lightweight pitting intelligent detection method based on binary image fusion of the present invention; Figure 2 These are some pictures in the pitting image data set collected by the present invention; wherein (a) is the front corrosion morphology of corrosion test piece 1, (b) is the back corrosion morphology of corrosion test piece 2, (c) is the back corrosion morphology of corrosion test piece 3, and (d) is the front morphology of corrosion test piece 4; Figure 3 It is a comparison diagram of the pitting image before and after binarization of the present invention; wherein (a) is the pitting image before binarization, and (b) is the pitting image after binarization; Figure 4 It is a schematic diagram of the structure of the binary image fusion module of the present invention; Figure 5 It is a schematic diagram of the lightweight pitting intelligent detection system based on binary image fusion of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to explain the present invention to make it clearer and more specific. Without departing from the technical scheme of the present application, the several improvements made and various non-creative labor results all belong to the protection scope of the present application.

[0021] See also Figure 1 The invention discloses a lightweight pitting intelligent detection method based on binary image fusion to detect pitting defects. The specific steps are as follows: S1: Build a pitting optical imaging system, collect pitting images of stainless steel materials, use square boxes to mark the pitting positions, and construct a pitting image dataset; S2: Select a lightweight benchmark detection network A and train it on a pitting image dataset to test the accuracy of the benchmark detection network in detecting pitting; S3: Establish a lightweight pitting detection network based on binary image fusion. The specific steps are as follows: S3.1: Select a suitable binarization threshold to perform image binarization on the pitting image to obtain a binary image , highlighting the pitting characteristics; S3.2: Design a pitting binary image fusion module to further extract the binary features of the pitting image, which specifically includes the following steps: S3.2.1: Input features based on the benchmark detection network A , generate the binarized image weights ; S3.2.2: Weight the binarized image With the binary image After point multiplication, the weighted features of the binary image are obtained. ; S3.2.3: Combine input features with binarized image weighted features Add together to get the output features ; S3.3: Add the binary image fusion module to the lightweight benchmark detection network A to complete the establishment of a lightweight pitting detection network based on binary image fusion: S4: Based on the pitting image data set, the benchmark detection network and the lightweight pitting detection network based on binary image fusion are trained and verified. After the training is completed, a trained lightweight pitting detection network based on binary image fusion is obtained; S4.1: Use K-fold cross-validation method to divide the data set into K parts, each of which is used as a test set, and the rest is used as a training set. The ratio of training set to test set is (K-1):1; S4.2: After performing image standardization operation on the training set data described in step S4.1, a data augmentation method of random rotation, scaling, translation, cropping, and perspective transformation is used; for the test set data described in step S4.1, only image standardization operation is used for processing; S4.3: Select training sets in turn to train the benchmark detection network and the lightweight pitting detection network based on binary image fusion, and use the test sets for cross-validation in turn, and take the average results of multiple training tests as the final result of the model test.

[0022] S5: Test the trained lightweight pitting detection network based on binary image fusion to obtain test results.

[0023] Example 1 S1: Based on the national standard GB / T 17897-2016, a 304 stainless steel pitting sample was prefabricated and an optical imaging system consisting of an industrial camera, an active excitation light source, and a bracket was built. The active excitation light source was set on the bracket, and the industrial camera was set on the side of the active excitation light source. The optical imaging system collected a total of 412 pitting images with a size of 640×640. Some of the images are as follows Figure 2 As shown in (a), (b), (c) and (d), to ensure that the corrosion data is objective and comprehensive, clothing marks are left on both the front and back of the test piece when the corrosion test piece is made. A square frame is used to accurately mark the position of each pitting on each picture to record the position information of each pitting pit. The position information of each pitting pit includes the length and width of the pitting pit, as well as the horizontal and vertical coordinates of the pitting pit on the picture. After all the pitting pictures are labeled, the construction of the pitting image dataset is completed.

[0024] S2: Select the YOLO-v5s network as a lightweight benchmark detection network, and train the benchmark detection network based on the constructed pitting image dataset; S3: Establishing a lightweight pitting detection network based on binary image fusion, specifically including the following steps: S3.1: The binarization threshold is set to 150, and the pitting image in the pitting image dataset is binarized to obtain a binarized image , highlighting the pitting characteristics, the pitting image before and after binarization is compared Figure 3 As shown in (a) and (b); S3.2: further extracting the binary features of the binary image through the pitting binary image fusion module; See also Figure 4 , which specifically includes the following steps: S3.2.1: Input features are ,in and Represents the height and width of the input feature, The value is 128. The value is 128. represents the number of channels of input features, The value is 3. The input feature is convolved with a 1 × 1 convolution kernel to obtain the weighted weight , where the convolution kernel of size 1 × 1 can be expressed as , the number of channels of the first input feature Indicates the number of input channels, the number of channels of the second input feature represents the number of output channels. Therefore, the entire weighted It can be expressed as:

[0025] S3.2.2: Weighting Then multiply it with the pitting of the binary image to get the weighted features of the binary image .

[0026] Binarize the image to Represents that the weighted features of the binary image are Indicates that the weighted features of the binary image can be expressed as:

[0027] S3.2.3: Input features Weighted features of binary images Add together to get the output features , which is the binary feature of the binary image, as shown in the following formula:

[0028] S3.3: Add the pitting binary image fusion module to the fifth and seventh layers of the benchmark detection network YOLO-v5s, so as to integrate the binary features of the binary image into the benchmark detection network YOLO-v5s, and complete the establishment of a lightweight pitting detection network based on binary image fusion; S4: Using the pitting image data set to train the lightweight pitting detection network based on binary image fusion, to obtain a trained lightweight pitting detection network based on binary image fusion, including the following steps: S4.1: The pitting image dataset established in step S1 is evenly divided into 5 parts using K (K=5) fold cross validation, each part is used as a test set in turn, and the remaining parts are used as training sets; S4.2: After performing image standardization operation on the training set data described in step S4.1, the data is processed by random rotation, scaling, translation, cropping and perspective transformation data augmentation; the test set data described in step S4.1 is processed only by image standardization operation; S4.3: Select the training set processed by step 4.1 to train the benchmark detection network and the lightweight pitting detection network based on binary image fusion, and use the test set processed by step 4.2 for cross-validation, and take the average result of multiple training tests as the final result of the model test, that is, obtain the trained lightweight pitting detection network based on binary image fusion; S5: After the training, the test set is tested by the trained lightweight pitting detection network based on binary image fusion. mAP50 is the detection accuracy index. The detection results are shown in Table 1 below: Table 1 Test results

[0029] As can be seen from Table 1, compared with the benchmark detection network YOLO-v5s, the pitting detection accuracy index mAP50 of the lightweight pitting detection network based on binary image fusion has increased by 1.81%. Compared with other methods that increase the accuracy by 1% and increase the computational load by 50%, the present invention only increases the computational load by 1.2% (FLOPs(G)) and increases the accuracy by 1.81%, which is advanced.

[0030] Example 2 See also Figure 5 , a lightweight pitting intelligent detection system based on binary image fusion, including: A pitting image acquisition module is used to acquire pitting images of the stainless steel sample to be tested; A pitting position marking module is used to mark the pitting position on the pitting image of the stainless steel sample to be tested; The pitting detection result acquisition module is used to input the pitting corrosion image of the sample to be detected after marking the pitting corrosion position into a pre-trained lightweight pitting corrosion detection network based on binary image fusion to obtain the pitting corrosion detection result of the stainless steel sample to be detected; Among them, the pre-trained lightweight pitting detection network based on binary image fusion is established to integrate the binary features of the binary image into the lightweight benchmark detection network.

[0031] Example 3 An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the lightweight pitting intelligent detection method based on binary image fusion is implemented.

[0032] Example 4 A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the lightweight pitting intelligent detection method based on binary image fusion is implemented.

[0033] The above description is only for the best embodiment of the present invention, but it should not be understood as limiting the claims. The present invention is not limited to the above embodiments, and its specific structure is allowed to be changed. However, all changes made within the protection scope of the independent claims of the present invention are within the protection scope of the present invention.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

Claims

1. A lightweight pitting intelligent detection method based on binary image fusion, characterized in that: The following steps are involved: Obtain pitting pictures of the stainless steel samples to be tested; Mark the pitting position on the pitting picture of the stainless steel sample to be tested; The pitting corrosion image of the sample to be tested with the pitting corrosion position marked is input into a pre-trained lightweight pitting corrosion detection network based on binary image fusion to obtain the pitting corrosion detection result of the stainless steel sample to be tested; Among them, the pre-trained lightweight pitting detection network based on binary image fusion is established to integrate the binary features of the binary image into the lightweight benchmark detection network.

2. The lightweight pitting intelligent detection method based on binary image fusion according to claim 1 is characterized in that: The pre-trained lightweight pitting detection network based on binary image fusion is established to integrate the binary features of the binary image into the lightweight benchmark detection network, including the following steps: After annotating the pitting images of the stainless steel samples to be tested, a pitting image dataset is obtained; Binarization is performed on the pitting image in the pitting image data set to obtain a binary image; Extracting binary features of the binary image; The binary features of the binary image are integrated into the lightweight benchmark detection network, and a lightweight pitting detection network based on binary image fusion is established.

3. The lightweight pitting intelligent detection method based on binary image fusion according to claim 2 is characterized in that: Extracting the binary features of the binary image includes the following steps: Add the input feature to the weighted feature of the binary image to obtain the binary feature of the extracted binary image; The input features are ,in and Represents the height and width of the input feature, Represents the number of channels of the input feature; the weighted feature of the binary image is the product of the weighted weight and the pitting of the binary image.

4. The lightweight pitting intelligent detection method based on binary image fusion according to claim 3 is characterized in that: The weighted weights are obtained through the following process: convolution operation is performed on the input features with a convolution kernel of size 1 × 1 to obtain the weighted weights.

5. The lightweight pitting intelligent detection method based on binary image fusion according to claim 3 is characterized in that: The weighted features of the binary image are calculated by the following formula: In the formula, the binary image , Binarized image weighted features , is the weighted weight.

6. The lightweight pitting intelligent detection method based on binary image fusion according to claim 3 is characterized in that: The lightweight benchmark detection network is the YOLO-v5s network.

7. The lightweight pitting intelligent detection method based on binary image fusion according to claim 3 is characterized in that: The pre-trained lightweight pitting detection network based on binary image fusion is obtained through the following process: S4.1: Obtain a pitting corrosion image of the stainless steel sample to be tested, mark the pitting corrosion position on the pitting corrosion image of the stainless steel sample to be tested, and construct a pitting corrosion image data set; The K-fold cross-validation method is used to divide the pitting image dataset into K parts on average. Each part is used as a test set in turn, and the remaining parts are used as training sets. The ratio of training set to test set is (K-1):

1. S4.2: After image standardization operation is performed on the training set data, the training set data is processed by data augmentation methods of random rotation, scaling, translation, cropping and perspective transformation; The test set data is processed by image standardization operation; S4.3: Select the training sets in turn to train the lightweight pitting detection network based on binary image fusion, and use the test sets in turn for cross-validation to obtain the pre-trained lightweight pitting detection network based on binary image fusion.

8. A lightweight pitting intelligent detection system based on binary image fusion, characterized in that: include: A pitting image acquisition module is used to acquire pitting images of the stainless steel sample to be tested; A pitting position marking module is used to mark the pitting position on the pitting image of the stainless steel sample to be tested; The pitting detection result acquisition module is used to input the pitting corrosion image of the sample to be detected after marking the pitting corrosion position into a pre-trained lightweight pitting corrosion detection network based on binary image fusion to obtain the pitting corrosion detection result of the stainless steel sample to be detected; Among them, the pre-trained lightweight pitting detection network based on binary image fusion is established to integrate the binary features of the binary image into the lightweight benchmark detection network.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the lightweight pitting intelligent detection method based on binary image fusion as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the lightweight pitting intelligent detection method based on binary image fusion as described in any one of claims 1 to 7 is implemented.

Images