Rust image labeling method and device based on unsupervised semantic segmentation, medium

Abstract

The present application relates to a kind of corrosion image labeling method, equipment, medium based on unsupervised semantic segmentation, wherein method is: corrosion image data set to be labeled is constructed;Unsupervised semantic segmentation model is constructed;Label image is converted into binary image, using boundary tracking algorithm extraction contour, then using Douglas-Poke algorithm to polygon approximation to contour, to extract key data point, obtain label contour represented by point set;Based on the decision of artificial screening, eliminate non-corrosion label, and carry out ray method judgment, after image with uniform label, contour optimization and delete inner contour are carried out, to obtain the image of final only containing corrosion contour.Compared with prior art, the present application infers the semantic information of pixel point of corrosion image by unsupervised semantic segmentation, then the obtained semantic information image is artificially screened, and finally the complete corrosion image labeled is obtained, which greatly saves the time of traditional corrosion image pixel-level labeling.

Description

Technical Field

[0001] This invention relates to the field of mechanical and marine engineering technology, and in particular to a method, device, and medium for annotating rust images based on unsupervised semantic segmentation. Background Technology

[0002] Due to the high temperature, high pressure, and humid working environment, mechanical equipment is highly susceptible to rust damage during its service life, posing a serious challenge to safe production. Currently, machine vision is widely used in rust detection and has some application in actual production. However, machine vision relies on a large number of well-annotated rust images, and rust image annotation, especially pixel-level rust image annotation, is time-consuming and labor-intensive. In traditional rust image annotation methods, not only do annotators need to manually annotate, but professionals also need to inspect and rework the images. This undoubtedly greatly increases the process and time required for rust image annotation. Pixel-level rust image annotation has become one of the obstacles to the further development of rust detection, urgently requiring continuous improvement and enhancement by researchers. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a method, device, and medium for labeling rust images based on unsupervised semantic segmentation. The method infers the semantic information of pixels in the rust image through unsupervised semantic segmentation, and then manually filters the obtained semantic information image to finally obtain a fully labeled rust image, which greatly saves the time of traditional pixel-level labeling of rust images.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] The first aspect of this invention provides a method for annotating rust images based on unsupervised semantic segmentation, comprising the following steps:

[0006] S1: Obtain the corrosion images to be labeled and construct the corrosion image dataset to be labeled;

[0007] S2: Construct an unsupervised semantic segmentation model, which includes constructing an input layer, convolutional layer, batch normalization layer, activation layer, and dropout layer of a convolutional neural network structure, and setting the corresponding adjacency space loss function and hyperparameters;

[0008] S3: Convert the label image into a binary image, use the boundary tracking algorithm to extract the contour, and then use the Douglas-Puk algorithm to approximate the contour with polygons to extract key data points and obtain the label contour represented by a point set.

[0009] S4: Based on the decision made by manual screening, non-rust labels are removed, and the ray tracing method is used to determine the image after unifying the labels. The inner contour is optimized and the inner contour is deleted to obtain the final image containing only the rust contour.

[0010] Furthermore, in S1, all the corrosion images are first cropped to the target size, and then a dataset of corrosion images to be labeled is constructed.

[0011] Furthermore, in S2, the batch normalization layer standardizes the features extracted by the upper convolutional layer by normalizing the distribution of the input data to a distribution with a mean of 0 and a variance of 1, thereby solving the problem of numerical instability in deep neural networks.

[0012] The activation layer uses the ReLU activation function, as shown in the following formula:

[0013] Relu(x) = max(0,x) (1)

[0014] Where x is the input data, max() is the maximum value function, and the activation function is used to guide unsupervised semantic segmentation to learn the deep nonlinear relationship between adjacent colors in the rust image, so as to achieve high-precision recognition of rust edges.

[0015] Furthermore, in S2, the process of constructing the adjacent space loss function includes:

[0016] Construct adjacent spatial labels, which are constructed using formula (2):

[0017]

[0018] Where, loss left To measure the label consistency of the left-side adjacent space of a pixel, the loss function is used. right To measure the label consistency of the right-hand neighboring space of a pixel, the loss function is used. up To measure the label consistency of the adjacent space above a pixel, the loss function is used. down To measure the consistency of labels in the adjacent space below a pixel; pred is the predicted value of the unsupervised semantic model, which is a three-dimensional matrix. In the matrix, ":" represents truncating all data in the current dimension, "1:" represents truncating data from index 1 to the end in the current dimension, and "0:-1" represents truncating all data up to the last index in the current dimension; the target value of the adjacent space label is a matrix of all zeros, so that the unsupervised semantic segmentation model can learn the semantic information of the adjacent space;

[0019] By inputting the target value of the adjacent spatial label and the value of the adjacent spatial label into the loss function, the loss of the current iteration can be obtained;

[0020] Construct the total loss function, and then use the cross-entropy loss function. fn and smoothing L1 loss function Obtained by linear weighted summation:

[0021]

[0022] Here, lhpy is used to measure the smooth L1 loss of a pixel in the horizontal direction, lhpz is used to measure the smooth L1 loss of a pixel in the vertical direction, target is an all-zero matrix, and α and β are the weight coefficients of the cross-entropy loss function and the smooth L1 loss function, respectively.

[0023] Furthermore, in S3, the process of extracting contours using the boundary tracing algorithm includes:

[0024] To extract contours from a binary image, first randomly select any pixel as the starting point, and then combine that point with its top, left, and top-left four points to form a block. There will be a total of 16 shapes for the block.

[0025] The next move strategy for blocks in each form is pre-set;

[0026] The block moves forward continuously according to the movement strategy. When it returns to the starting point, the algorithm ends, and a complete polygon outline is obtained.

[0027] Furthermore, in S3, the process of polygon approximation using the Douglas-Puk algorithm is as follows:

[0028] Draw a straight line connecting the first and last points of the curve. Calculate the distance between all points on the curve and the straight line, and find the maximum distance, dmax. Then compare dmax with a pre-set threshold, and simplify the curve point set based on the comparison result, as shown in the following formula:

[0029]

[0030] The set of curve points is P = {(x1,y1),(x2,y2),…,(x...} n ,y n The function f(P) represents the operation of approximating the point set P as a straight line segment or dividing it into two parts, where P1 and P2 are the two part point sets obtained by dividing according to dmax.

[0031] Furthermore, in S7, the process for removing non-corroded labels is as follows:

[0032] Based on the decision made by manual screening, multiple types of labels are unified into a single label as the rust label, and the S3 process is performed again on the image file with the unified label to achieve the fusion of the label outlines.

[0033] Furthermore, in S7, the process of determining the result using the ray casting method is as follows:

[0034] If all the points that make up the first contour are located inside the second contour, then the first contour can be considered to be contained in the second contour, that is, the set of points that make up the first contour needs to be deleted.

[0035] Whether point P is inside the contour can be determined by the number of intersections between the horizontal ray drawn from any point P and the contour. There are two cases: 1) If the number of intersections is odd, then P is inside the contour; 2) If the number of intersections is even, then P is outside the contour.

[0036] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the processor is configured to execute a program in the memory to implement the above-described method for annotating rust images based on unsupervised semantic segmentation.

[0037] A third aspect of the present invention provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the above-described method for annotating rust images based on unsupervised semantic segmentation.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] Beneficial effect 1:

[0040] To verify the quality of the annotated rust images using the improved method, 1000 annotated rust images underwent expert review. The review showed that 98.5% of the rust images met the quality standards and required no modification, while 1.5% failed to meet the standards due to blurring in the rust areas. This demonstrates that, while ensuring image quality, the proposed unsupervised semantic segmentation-based rapid rust image annotation technology has reached the level required for expert review. Annotating any rust image can be considered to require only manual screening, without the need for expert review.

[0041] In traditional methods for annotating artificial rust images, a complete annotation process involves the following steps: 1) initial annotation by the annotator; 2) expert review; 3) revision by the annotator and submission for further expert review. It is worth noting that step 2 is the most time-consuming part of the annotation process because it requires waiting for expert review. Therefore, this invention can effectively speed up image annotation by reducing the number of expert reviews.

[0042] Traditional annotation processes show that manual annotation of rust images is time-consuming and labor-intensive. This invention proposes a rapid annotation technique for rust images based on unsupervised semantic segmentation. This technique can generate annotation results for rust images using an unsupervised semantic segmentation model and label fusion technology, thus partially simplifying the manual annotation process.

[0043] To more intuitively demonstrate the time efficiency improvement of the improved method compared to the traditional method, time complexity is introduced as an evaluation metric. In the traditional method, the time for the initial annotation by the annotator is t1, the expert review time is t2, the time for the annotator to revise and resubmit for expert review is t3, and the number of resubmissions is n. Its time complexity is as follows:

[0044] T old =O(t1+n×(t2+t3)) (6)

[0045] As can be seen from formula (6), the traditional annotation method T old The time complexity is on the order of O(n). In the improved method, the time for manual label selection is t4, and since expert review is not required, the overall time complexity T of the improved method can be calculated. new The time complexity is derived as follows:

[0046] T new =O(t4) (7)

[0047] As can be seen from formula (7), the improved method T new The time complexity is O(1).

[0048] Therefore, the fast annotation technology for rust images based on unsupervised semantic segmentation proposed in this invention is significantly superior to traditional manual annotation methods in terms of time efficiency, and can effectively solve the problem of time-consuming and labor-intensive manual image annotation in traditional methods.

[0049] Benefit 2:

[0050] This invention proposes an unsupervised semantic segmentation model based on 1x1 convolutional kernels, which can achieve high-precision recognition of rust edges in rust images. Traditional unsupervised semantic segmentation models use combinations of convolutional kernels of different sizes for image recognition. This approach is not suitable for labeling rust images because excessively large convolutional kernels can blur the rust edges in the image, making it difficult to recognize the rust edges and reducing the accuracy of the labeling.

[0051] Benefit 3:

[0052] This invention proposes a novel unsupervised label construction method that constructs learned labels for an unsupervised semantic segmentation model by extracting the neighborhood consistency from the model's predicted values. By examining the neighborhood consistency in the model's predicted values, the identification of rusted regions in images can be improved. This method, combined with a 1x1 convolutional kernel, can simultaneously identify both rusted regions and rusted edges. Essentially, rusted regions in a rusted image represent low-frequency information, while rusted edges represent high-frequency information. Neighbor space consistency is the extraction of low-frequency information, thus helping the unsupervised semantic model in this invention to better identify rusted regions in images. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the convolutional neural network structure design in this invention;

[0054] Figure 2 This is a schematic diagram of the boundary tracking algorithm in this invention;

[0055] Figure 3 This is a diagram illustrating the change of the loss curve during the training process in this invention.

[0056] Figure 4 This is a comparison image of the original image and the generated label image in this invention;

[0057] Figure 5 This is the label outline image in this invention;

[0058] Figure 6 This is the final rust image annotation file required in this invention. Detailed Implementation

[0059] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Component models, material names, connection structures, control methods, algorithms, and other features not explicitly described in this technical solution are considered common technical features disclosed in the prior art.

[0060] Example 1

[0061] Step 1: Construct a dataset of rust images to be labeled

[0062] This invention requires acquiring a sufficient number of rust images to be labeled in this step. To balance rust information, model training speed, and reduced hardware requirements, all rust images are cropped to a size of 224x224.

[0063] Step 2: Construct an unsupervised semantic segmentation model

[0064] like Figure 1 As shown, the convolutional neural network structure designed in this invention consists of the following parts:

[0065] (1) Input layer: Converts the input image from [length, width, number of channels] to [number of channels, length, width].

[0066] (2) Convolutional layer: The image is convolved using a 1x1 convolutional kernel. Using a 1x1 convolutional kernel can maximize the recognition of rust edges in the image, so as to achieve high-precision automatic annotation of rust edges.

[0067] (2) Batch normalization layer: The features extracted by the upper convolutional layer are normalized. By normalizing the distribution of the input data to a distribution with a mean of 0 and a variance of 1, the problem of numerical instability in deep neural networks is solved, so that the distribution of each feature in the same batch is similar, and the network is easier to train.

[0068] (3) Activation Layer: This layer endows the convolutional neural network with the ability to perform non-linear fitting. This invention uses the ReLU activation function, as shown in the following formula:

[0069] Relu(x) = max(0,x) (1)

[0070] Here, x represents the input data, and max() is the maximum value function. The activation function can effectively guide unsupervised semantic segmentation to learn the deep nonlinear relationships between adjacent colors in a rust image, thereby achieving high-precision recognition of rust edges.

[0071] (4) Dropout layer: This layer randomly sets the output values ​​of some parameters in the unsupervised semantic segmentation model to 0, so as to retain or discard each neuron with a certain probability, thereby reducing the number of model parameters, simplifying the model complexity, and thus reducing the hardware requirements of this invention.

[0072] (5) Neighbor Space Loss Function: The loss function is the core of the unsupervised semantic segmentation model. In traditional supervised models, the input variables of the loss function are the true values ​​and the model predictions. However, in unsupervised semantic segmentation models, each iteration is an inference process. Therefore, in addition to the model predictions, the input variables of the loss function also need to be the neighbor space labels derived from the model predictions. By inputting the neighbor space labels and the neighbor space target values ​​into the loss function, the loss value for this iteration can be obtained.

[0073] The core idea of ​​constructing adjacent spatial labels is that, except for the rust edge, adjacent pixels should have the same label, which is essentially the extraction of low-frequency information in the image. Rust areas in the image are low-frequency information. Therefore, constructing adjacent spatial labels can effectively identify rust areas. Adjacent spatial labels can be constructed using formula (2).

[0074]

[0075] Where, lossleft To measure the label consistency of the left-side adjacent space of a pixel, the loss function is used. right To measure the label consistency of the right-hand neighboring space of a pixel, the loss function is used. up To measure the label consistency of the adjacent space above a pixel, the loss function is used. down This measures the label consistency among adjacent pixels. `pred` is the prediction value of the unsupervised semantic model, which is a three-dimensional matrix. In the matrix, ":" represents truncating all data in the current dimension, "1:" represents truncating data from index 1 to the end in the current dimension, and "0:-1" represents truncating all data up to the last index in the current dimension.

[0076] The target value for the neighboring spatial labels is a matrix of all zeros, allowing the unsupervised semantic segmentation model to learn the semantic information of the neighboring spaces. The target value and the neighboring spatial label values ​​are input into the loss function to obtain the loss for this iteration.

[0077] This invention selects the cross-entropy loss function. fn and smoothing L1 loss function The linear weighted summation is used as the total loss function, as shown in the following formula:

[0078]

[0079] Here, lhpy is used to measure the smooth L1 loss of a pixel in the horizontal direction, lhpz is used to measure the smooth L1 loss of a pixel in the vertical direction, target is an all-zero matrix, and α and β are the weight coefficients of the cross-entropy loss function and the smooth L1 loss function, respectively.

[0080] In addition to the components mentioned above, the unsupervised semantic segmentation model also relies on various hyperparameters. This invention will provide a set of hyperparameter settings in specific implementation schemes. After the rust image is processed by the unsupervised semantic segmentation model, a preliminary labeled rust image with well-segmented data can be obtained.

[0081] Step 3: Label Outline Generation

[0082] Step 2 yields an unsupervised semantic segmentation model. Inputting the rust image into the model produces a preliminarily labeled rust image. The unsupervised semantic segmentation model groups similar pixels into one class, thus requiring the generation of label contours for each class. This necessitates performing three steps sequentially: binary image generation, contour extraction, and polygon approximation.

[0083] (1) Binary Image Generation

[0084] In the initially labeled rust image, each pixel is assigned a unique label. In order to better locate the label outline, the label image needs to be converted into several binary images. For label k, its corresponding binary image matrix Binary_Mask can be represented by formula (4):

[0085]

[0086] Here, label(i,j) is the label category corresponding to pixel (i,j). The generated binary image matrix contains only 0 and 1, where 0 represents the background region and 1 represents the foreground region. Subsequent operations only need to label the outline of the foreground region with a point set.

[0087] (2) Contour extraction

[0088] This invention employs a boundary tracing algorithm to extract contours from binary images. The algorithm first randomly selects an arbitrary pixel as a starting point and combines this pixel with its top, left, and top-left points to form a block. There are a total of 16 possible block shapes. The boundary tracing algorithm pre-defines the next movement strategy for each block shape. The block moves forward continuously according to the strategy, and the algorithm terminates when it returns to the starting point, thus obtaining the complete polygonal contour. The boundary tracing algorithm flow is as follows: Figure 2 As shown, (5, 3) is selected as the starting point, and {(4, 2), (4, 3), (5, 2), (5, 3)} are combined into a block. According to the algorithm's preset strategy, the block moves down 1 grid and to the right 1 grid in sequence, and continues to move forward according to the strategy until the block finally returns to the starting position, and the contour extraction is completed.

[0089] (3) Polygon approximation

[0090] The point set obtained after contour extraction contains a large amount of redundancy, which affects computational efficiency. Therefore, an approximation algorithm is needed to approximate the contour into polygons to extract key data points. This invention uses the Douglas-Pock algorithm for polygon approximation, which can minimize the number of data points while maintaining the curve shape. The Douglas-Pock algorithm first draws a straight line connecting the first and last points of the curve, calculates the distance between all points on the curve and the line, and finds the maximum distance, dmax. Then, dmax is compared with a pre-set threshold, and the curve point set is simplified based on the comparison result, as shown in the following formula:

[0091]

[0092] The set of curve points is P = {(x1, y1), (x2, y2), ..., (x...} n y nThe function f(P) represents the operation of approximating the point set P as a straight line segment or dividing it into two parts, where P1 and P2 are the two part point sets obtained by dividing according to dmax.

[0093] Step 4: Contour Blending

[0094] Step 3 represents the label outline using a set of points. Step 4 aims to further optimize the outlines through manual selection to obtain a final image containing only the rust outline. This requires unifying the manually selected labels into a single label as the rust label, and then repeating the process of step 3 on the image file with the unified label to achieve the fusion of the label outlines. However, after fusion, there may be nested outlines, i.e., an inner outline exists within an outer outline, which needs to be deleted.

[0095] The ray method used in this invention can be used to determine whether a point is inside a polygon. If all the points forming the first contour are located inside the second contour, then the first contour can be considered to be contained within the second contour, meaning the set of points forming the first contour needs to be deleted. The ray method determines whether point P is inside the contour by the number of intersections between a horizontal ray drawn from any point P and the contour. There are two cases: 1) If the number of intersections is odd, then P is inside the contour; 2) If the number of intersections is even, then P is outside the contour.

[0096] By performing the above operations, you can obtain the final rust image annotation file.

[0097] This embodiment also proposes a rapid annotation device for rust images based on unsupervised semantic segmentation. The device includes a processor and a memory, which are coupled. The memory stores program instructions, and when these instructions are executed by the processor, the aforementioned task management method is implemented. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The memory may include Random Access Memory (RAM) or Non-Volatile Memory, such as at least one disk storage device. The memory can be an internal memory of the Random Access Memory (RAM) type. The processor and memory can be integrated into one or more independent circuits or hardware, such as an Application Specific Integrated Circuit (ASIC). It should be noted that when the computer program in the aforementioned memory is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0098] This embodiment also proposes a computer-readable storage medium storing computer instructions for instructing a computer to execute the aforementioned method for rapid annotation of rust images based on unsupervised semantic segmentation. The storage medium can be an electronic medium, magnetic medium, optical medium, electromagnetic medium, infrared medium, or a semiconductor system or propagation medium. The storage medium may also include semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory (RAM), read-only memory (ROM), hard disk, and optical disc. Optical discs may include optical disc-read-only memory (CD-ROM), optical disc-read / write (CD-RW), and DVD.

[0099] Application Example 1

[0100] In a specific application, a fast annotation technique for rust images based on unsupervised semantic segmentation is provided, which is implemented through the following steps.

[0101] Step 1

[0102] Images of surface rust on mechanical equipment in operation were collected on a marine platform in service. Images with poor imaging quality were removed through data cleaning. The large-size rust images were randomly divided and cropped into 224x224 non-overlapping smaller images. Images containing rusted areas were selected to form a rust image dataset.

[0103] Step Two

[0104] The rust image obtained in step one is input into the unsupervised semantic segmentation model constructed in step two to obtain a preliminary labeled rust image.

[0105] Convolutional neural networks are constructed according to step 2 of the technical solution. The structural parameters of the convolutional neural network are as follows:

[0106] Table 1 Convolutional Neural Network Structure Parameters

[0107]

[0108] The convolutional neural network (CNN) employs a four-layer structure. The input image is an RGB three-channel color image. Through multiple layers of convolution, batch normalization, and activation operations, it continuously extracts higher-level features from the input image. To balance time efficiency, 300 iterations are chosen. A linear weighted sum of the cross-entropy function and the smoothing L1 function is used as the total loss function. The CNN is built on PyTorch, running on Python 3.8, NumPy 1.24, scipy 1.10.1, pillow 10.0.0, pycocotools 2.0.6, scikit-image 0.21.0, PyTorch 1.8.0, and cuda 11.1. The hardware environment consists of an Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz, an Nvidia 3090, and 128GB DDR4 2400 RAM. The learning rate is 0.1, and the weight coefficients α = 0.1 and β = 0.1. The loss curve changes during training as follows: Figure 3 As shown.

[0109] Step 3

[0110] The labeled images identified by the unsupervised semantic segmentation model contain a total of 23 labels, such as Figure 4 As shown. Following the steps in step 3 of the technical solution, the label images are processed. For each label, the boundary tracing algorithm and the Douglas-Puk algorithm are used sequentially for contour extraction and polygon approximation to generate the corresponding label's contour point set. The label contour image is shown below. Figure 5 As shown.

[0111] Step Four

[0112] Non-rust labels are manually removed from the images obtained in step three. Following the process in step four of the technical solution, the label names are unified to rust labels. The unified rust image label file is then used as a new label file for contour generation to achieve contour fusion. Finally, the inner contours are removed using the ray casting method to obtain the final required rust image annotation file, such as... Figure 6 As shown.

[0113] Step 5

[0114] To quantitatively analyze the advantages of the proposed fast annotation technology for rust images based on unsupervised semantic segmentation compared with traditional manual annotation methods, 100 images were randomly selected from the original rust image dataset for a comparative experiment. The average annotation time was used as an indicator to measure the performance of different annotation methods. Under the premise of no interference from other factors, the final experimental data are shown in Table 2.

[0115] Table 2 Comparison of annotation methods

[0116]

[0117]

[0118] in, This indicates the average annotation time for the initial annotation by the annotators. This indicates the average review time by experts.

[0119] This indicates the average time it takes for the annotator to revise the annotations and submit them to the expert for further review. This represents the average time for manual screening in the improved method. The average number of submissions is represented by the formula (8). The average annotation time for the two methods can be calculated using the formula (8).

[0120]

[0121] As shown in Table 2, the improved annotation method is 93.8% faster than the traditional method, a significant improvement. Furthermore, because the traditional method relies solely on the visual judgment of the annotators, the annotation quality declines noticeably as the sample size increases, leading to a further increase in the number of submissions and annotation time. Therefore, the improved method proposed in this invention is significantly superior to traditional manual annotation methods in both time efficiency and annotation quality. It can quickly and efficiently obtain a large number of rust-annotated images, compensating for the lack of high-quality rust images in the fields of mechanical and marine engineering.

[0122] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. A method for annotating rusted images based on unsupervised semantic segmentation, characterized in that, Includes the following steps: S1: Obtain the corrosion images to be labeled and construct the corrosion image dataset to be labeled; S2: Construct an unsupervised semantic segmentation model, which includes constructing an input layer, a 1x1 convolutional kernel convolutional layer, a batch normalization layer, an activation layer, and a Dropout layer of a convolutional neural network structure, and setting the corresponding adjacency space loss function and hyperparameters; S3: Convert the label image into a binary image, use the boundary tracking algorithm to extract the contour, and then use the Douglas-Puk algorithm to approximate the contour with polygons to extract key data points and obtain the label contour represented by a point set. S4: Based on the decision of manual screening, non-corrosion labels are removed, and the ray method is used for judgment. The image after unifying the labels is optimized and the inner contour is deleted to obtain the final image containing only the rust contour. In S2, the process of constructing the adjacent space loss function includes: Construct adjacent spatial labels, which are constructed using formula (2): in, To measure the label consistency of the left-side adjacent space of a pixel, To measure the label consistency of the adjacent space to the right of a pixel, To measure the label consistency of the adjacent space above a pixel, To measure the label consistency of the adjacent space below a pixel; The predicted values ​​of the unsupervised semantic model are represented by a three-dimensional matrix, in which... This means extracting all data in the current dimension. This means extracting data from the current dimension, starting from index 1 and ending at the end. This represents extracting all data up to the last one in the current dimension; the target value of the adjacent space label is a matrix of all zeros, so that the unsupervised semantic segmentation model can learn the semantic information of the adjacent space; By inputting the target value of the adjacent spatial label and the value of the adjacent spatial label into the loss function, the loss of the current iteration can be obtained; Construct the total loss function, using the cross-entropy loss function. and smooth loss function Obtained by linear weighted summation: in, Used to measure the smoothness of pixels in the horizontal direction loss, Used to measure the smoothness of pixels in the vertical direction. loss, It is a matrix of all zeros. , Cross-entropy loss function and smoothing respectively The weighting coefficients of the loss function.

2. The method for annotating rust images based on unsupervised semantic segmentation according to claim 1, characterized in that, In S1, all the rust images are first cropped to the target size, and then a dataset of rust images to be labeled is constructed.

3. The method for annotating rust images based on unsupervised semantic segmentation according to claim 1, characterized in that, In S2, the batch normalization layer standardizes the features extracted by the upper convolutional layer by normalizing the distribution of the input data to a distribution with a mean of 0 and a variance of 1, thereby solving the problem of numerical instability in deep neural networks. The activation layer uses the ReLU activation function, as shown in the following formula: in, For input data, The activation function is a maximum value function, which is used to guide unsupervised semantic segmentation to learn the deep nonlinear relationship between adjacent colors in the rust image, so as to achieve high-precision recognition of rust edges.

4. The method for annotating rust images based on unsupervised semantic segmentation according to claim 1, characterized in that, In S3, the process of extracting contours using the boundary tracing algorithm includes: To extract contours from a binary image, first randomly select any pixel as the starting point, and then combine that point with its top, left, and top-left four points to form a block. There will be a total of 16 shapes for the block. The next move strategy for blocks in each form is pre-set; The block moves forward continuously according to the movement strategy. When it returns to the starting point, the algorithm ends, and a complete polygon outline is obtained.

5. The method for annotating rust images based on unsupervised semantic segmentation according to claim 1, characterized in that, In S3, the process of polygon approximation using the Douglas-Puk algorithm is as follows: Draw a straight line connecting the first and last points of the curve. Calculate the distance between all points on the curve and the straight line, and find the maximum distance, dmax. Then compare dmax with a pre-set threshold, and simplify the curve point set based on the comparison result, as shown in the following formula: Wherein, the set of curve points is , function This represents the operation of approximating the point set P as a line segment or dividing it into two parts. and It is a set of two points obtained by dividing based on dmax.

6. The method for annotating rust images based on unsupervised semantic segmentation according to claim 1, characterized in that, In S7, the process for removing non-corroded labels is as follows: Based on the decision made by manual screening, multiple types of labels are unified into a single label as the rust label, and the S3 process is performed again on the image file with the unified label to achieve the fusion of the label outlines.

7. The method for annotating rust images based on unsupervised semantic segmentation according to claim 1, characterized in that, In S7, the process of determining the result using the ray casting method is as follows: If all the points that make up the first contour are located inside the second contour, then the first contour can be considered to be contained in the second contour, that is, the set of points that make up the first contour needs to be deleted. Whether point P is inside the contour can be determined by the number of intersections between the horizontal ray drawn from any point P and the contour. There are two cases: 1) If the number of intersections is odd, then P is inside the contour; 2) If the number of intersections is even, then P is outside the contour.

8. An electronic device, comprising a memory and a processor, characterized in that, The processor is used to execute the program in the memory to implement the rust image annotation method based on unsupervised semantic segmentation as described in any one of claims 1 to 7.

9. A storage medium containing computer-executable instructions, characterized in that, When executed by a computer processor, the storage medium of the computer-executable instructions is used to perform the unsupervised semantic segmentation-based rust image annotation method as described in any one of claims 1 to 7.

Images