Coating surface defect grade evaluation method based on interactive segmentation

Abstract

The invention belongs to the technical field of industrial image processing and deep learning crossing, and discloses a coating surface defect grade evaluation method based on interactive segmentation. According to the method, an interactive segmentation model comprising a high-resolution image encoder, a user click input encoder, an MVFM module and a high-resolution decoder is built; the operation efficiency and the detection detail richness are improved through a high-resolution encoder, a user clicks an input encoder to encode positive and negative click points into embedded features, and a high-resolution decoder fuses multi-source features and outputs defect masks in combination with user interaction. According to the method, the detection result can be optimized in real time according to the click of the user, the high-resolution image is adapted, and the adaptability and the detection accuracy of complex and variable defects are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of industrial image processing and deep learning, and specifically relates to a method for evaluating the level of defects on the surface of a coating based on interactive segmentation. Background Technology

[0002] Coatings are functional materials composed of key components such as film-forming substances, pigments, dispersion media, and additives. They are indispensable key functional materials in many sectors of the national economy, providing important services and supporting components. Coating durability is one of its core performance indicators. In practical applications, various defects often appear on the coating surface due to factors such as production processes, changes in environmental temperature and humidity, and oxidation. These defects not only affect the appearance but also impair its function of protecting the substrate. To scientifically evaluate the durability of coatings, the industry generally adopts the "aging test" method, which involves preparing a coating on the surface of a specific substrate and testing it under simulated harsh environments. During this process, the coating may develop complex and diverse defects such as blistering, rusting, cracking, and peeling. These defects are numerous, complex in their causes, and difficult to quantify precisely. Therefore, the assessment of surface defects in coatings relies on the accurate acquisition of quantitative characteristics such as the area, number, and size of the defects. In actual industrial scenarios, the same type of defects may appear on substrates of different materials and colors, resulting in complex and varied visual appearances. Traditional machine vision methods rely heavily on human-preset features, which have poor adaptability and limited generalization ability when dealing with such complex and varied feature extraction tasks.

[0003] In recent years, deep learning-based semantic segmentation methods have revolutionized the field. These methods significantly reduce human intervention and, thanks to their powerful feature learning capabilities, can identify defect regions from images with high accuracy, demonstrating good adaptability to multi-scale defects. However, these methods typically rely on large amounts of labeled data to train the model. When faced with rare defect types that are not sufficiently represented in the training data, their generalization ability drops significantly, making it difficult to effectively detect defects that differ greatly from the training set.

[0004] Therefore, it is of great significance to develop a defect detection and grading method that can balance automatic processing efficiency with good adaptability to complex and varied defects on coating surfaces. Summary of the Invention

[0005] To address at least one of the problems existing in the prior art, this invention provides a coating surface defect level assessment method based on interactive segmentation. This method can optimize the detection results in real time based on user click input during the detection process, thereby significantly improving the adaptability to complex and variable defects and the detection accuracy.

[0006] To achieve the objective of this invention, this invention provides a method for evaluating the surface defect level of a coating based on interactive segmentation. The method involves inputting an image of the coating test panel to be tested into a trained interactive segmentation model to obtain the defect level evaluation result. The interactive segmentation model includes a high-resolution image encoder, a user click input encoder, an MVFM module, and a high-resolution decoder. The high-resolution image encoder comprises a shallow network and a deep network. The shallow network extracts low-level detail features from the image of the coating test panel, while the deep network extracts high-level semantic information from the image, resulting in local and global image embeddings. The user click input encoder uses sine and cosine coding to form user click-coded embedding features representing the target detection region from the user's click input. The MVFM module fuses the local and global image embeddings and performs information interaction, outputting an image embedding with local texture detail information. The high-resolution decoder performs interactive attention calculation and information interaction between the global image embedding and the image embedding output by the MVFM module and the user click-coded embedding features, respectively, to obtain a log-odds map. Based on the log-odds map, a mask map marking the defect region is obtained, and based on the mask map, the defect level assessment result is obtained.

[0007] Furthermore, in the interactive segmentation model training phase: using the coating surface defect detection image dataset and the already constructed interactive segmentation model, modules such as data reading, interactive point simulation, model forward inference, loss function calculation, backpropagation, and data saving are built to realize the complete model training process. During the model training process, hyperparameters are optimized based on the real-time training effect, and finally the interactive segmentation model with the best performance is obtained.

[0008] Furthermore, standard test plate images after aging tests were acquired using industrial cameras, and image annotation software was used to annotate defects in the test plate images and export image labels, thereby constructing a coating surface defect detection image dataset for interactive segmentation model training.

[0009] Furthermore, in the interactive segmentation model, the image is encoded using a high-resolution image encoder, and the encoded output is temporarily stored in the computer's system memory. When the user clicks, the high-resolution decoder combines the output of the high-resolution image encoder with the user's click input, performs inference, and finally outputs a mask image, which contains the location, size, and shape of the defects.

[0010] Furthermore, the high-resolution image encoder is composed of two types of sub-layers stacked together: a window attention module and a query pooling window attention module. The deep network contains all the sub-layers of the high-resolution image encoder, while the first few sub-layers in the high-resolution image encoder are shallow networks. The window attention module is used to divide the image embedding into local windows for efficient in-window self-attention, while the query pooling window attention module is used to achieve efficient downsampling by pooling the query matrix on the basis of window attention, while maintaining cross-scale feature dependencies.

[0011] Furthermore, based on the image of the coating test plate, an image feature vector is obtained. The image feature vector is cropped into multiple local images of equal size, and then downsampled into a global sub-image with the same height and width as the local images. Multiple local images are processed by a shallow network for feature extraction, and multiple local image embeddings are output. The global sub-image is processed by a deep network for semantic information extraction, and a global image embedding is obtained.

[0012] Further, the following operations are performed in the MVFM module: After multiple local image embeddings are stitched together, information is fused through a window self-attention module. The stitched and fused local image embeddings are downsampled, and the global image embeddings are upsampled to obtain multiple image embeddings at two scales. The multiple image embeddings are then fused through a convolution module to obtain a new fused image embedding at two scales. The global image embedding and its upsampled output, the stitched local image embedding and its downsampled output are used as queries to perform different query (Q), key (K), and value (V) calculations to perform information interaction calculations between the global image embedding and the local image embedding, and obtain the image embedding after information interaction; The image embedding after information interaction is fused to obtain an image embedding with local texture detail information.

[0013] Furthermore, in the user click input encoder, the sine and cosine position encoding encodes the user's left-click positive input point and right-click negative input point to form a user click encoding embedding feature.

[0014] Furthermore, in the high-resolution decoder, the following operations are performed: The global image embedding and the image embedding output by the MVFM module are used as inputs to the global and local features, respectively, in two branches. Both branches include an interactive attention module and a shared multilayer perceptron module. In both branches, the global image embedding and the image embedding output by the MVFM module perform multiple interactive attention calculations with the user click-encoded embedding features, and exchange information at the same time. This enables the high-resolution decoder to calculate the correlation information between the global image embedding, the image embedding output by the MVFM module, and the user click-encoded embedding features, so as to adjust the output results according to the user's click input. After the calculations of the two branches, the interactive attention operation is performed again. Then, the two convolutions are multiplied by the output of the multilayer perceptron module to obtain the log-odds graph. Map the log-odds plot to a probability plot; The probability map is binarized, and areas with a probability higher than a set threshold are identified as defect areas, generating a mask map that marks the defect areas. Based on the mask image of the marked defect region, defect calculation is performed to generate the defect level.

[0015] The present invention also provides a coating surface defect level assessment system based on interactive segmentation, the system comprising the following modules: The training module is used to train the interactive segmentation model to obtain an interactive segmentation model for evaluating the defect level of the coating surface. The evaluation module is used to input the image of the coating test panel to be tested into the interactive segmentation model to obtain the defect level evaluation result. The interactive segmentation model includes a high-resolution image encoder, a user click input encoder, an MVFM module, and a high-resolution decoder. The high-resolution image encoder includes shallow and deep networks. The shallow network is used to extract low-level detail features from the image of the coating test panel, and the deep network is used to extract high-level semantic information from the image of the coating test panel, resulting in local and global image embeddings. The user click input encoder uses sine and cosine coding to form user click-coded embedding features representing the target detection region from the user's click input. The MVFM module fuses the local and global image embeddings and performs information interaction, outputting an image embedding with local texture detail information. The high-resolution decoder performs interactive attention calculation and information interaction between the global image embedding and the image embedding output by the MVFM module and the user click-coded embedding features, respectively, to obtain a log-odds map. Based on the log-odds map, a mask map marking the defect region is obtained, and based on the mask map, the defect level evaluation result is obtained.

[0016] The trained interactive segmentation model was converted to ONNX format and then deployed to a C++ runtime environment. Simultaneously, Qt Creator was used to develop software that integrates image acquisition, image preprocessing, interactive user detection, and result calculation and export. The coating surface defect level assessment software includes a camera interface, a routine detection interface, and an interactive detection interface. The camera interface displays real-time camera images, acquires test panel images, and initiates detection. The routine detection interface displays routine detection results and loads and saves the detection results of previously saved images. The interactive detection interface displays interactive detection results, allows users to revert to the previous interaction point, and loads and saves the detection results of previously saved images.

[0017] The defect rating function is compatible with standards GB / T 1766-2008 and ISO 4628-3:2016, and outputs rating results based on mask geometric features.

[0018] The present invention also provides a computer device.

[0019] The present invention also provides a computer-readable storage medium.

[0020] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention proposes an interactive segmentation method for evaluating the level of defects on the coating surface. It can optimize the detection results in real time based on the user's click input during the detection process, and significantly improve the adaptability and detection accuracy to complex and variable defects by relying on the user's judgment experience.

[0021] (2) The present invention designs a high-resolution image encoder and decoder. By processing the image locally after cropping and scaling and using a dual-branch encoding mechanism, the computational efficiency and the richness of detection details are improved. The MVFM module efficiently fuses global and local image embeddings. The dual-branch decoder further fuses user interaction point information and accurately outputs the detection mask. The above design can enable the model to better adapt to the high-resolution test board images collected during the detection process, retaining richer defect details and significantly improving the accuracy of the detection results. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a standard test plate that exhibits different defects after aging tests in an embodiment of the present invention.

[0023] Figure 2 This is a diagram showing the overall structure of the interactive segmentation model in an embodiment of the present invention.

[0024] Figure 3 This is a structural diagram of the MVFM module in an embodiment of the present invention.

[0025] Figure 4This is a schematic diagram of the test plate area image input before detection in an embodiment of the present invention.

[0026] Figure 5 This is the final mask image output after detection in this embodiment of the invention, used for defect level assessment.

[0027] Figure 6 This is a schematic diagram of the interface of the coating surface defect level assessment software in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further explained and described below with reference to specific embodiments.

[0029] This invention provides a method for evaluating the level of defects on coating surfaces based on interactive segmentation. This method constructs a high-resolution interactive segmentation model and combines user click input to optimize the detection results in real time, thereby achieving accurate level evaluation of complex and variable defects on coating surfaces.

[0030] A method for evaluating the surface defect level of a coating based on interactive segmentation specifically includes the following steps: Step 1, Data Preparation Stage: Collect images of coating defects and construct a dataset of images for detecting coating surface defects.

[0031] In one embodiment, such as Figure 1 As shown, images of standard test plates after aging tests are acquired using an industrial camera, including images of various defect types on the test plate surface such as rust, peeling, and blistering. Image annotation software is used to annotate the defects in the standard test plate images and export image labels to construct a coating surface defect detection image dataset for interactive segmentation model training.

[0032] Step 2: Build an interactive segmentation model.

[0033] like Figure 2 As shown, the interactive segmentation model includes a high-resolution image encoder, a user-click input encoder, an MVFM module, and a high-resolution decoder. Figure 4 As shown, this is an image of the defective test panel area on the coating surface. The interactive segmentation model uses the test panel area image, which is divided equally into left and right sides, as the original input. First, the image is encoded by a high-resolution image encoder. Because the high-resolution image encoder is resource-intensive and time-consuming, it is only run once per image, and the encoded output is temporarily stored in the computer's system memory. Whenever the user clicks, the high-resolution decoder, which is resource-efficient and time-saving, combines the output of the high-resolution image encoder with the user's click input in real time, performs inference, and finally outputs a mask image, which contains the location, size, and shape of the defect. This achieves the goal of optimizing the detection results in real time based on the user's click input.

[0034] The specific structure of the interactive segmentation model is explained below: (1) The high-resolution image encoder is obtained by improving the Hiera model with high-resolution adaptation, such as... Figure 2 As shown in the upper right section, the high-resolution image encoder comprises two parts: a shallow network and a deep network, stacked using two sub-layers: a window attention module and a query pooling window attention module. The shallow network quickly extracts low-level detail features such as edges and textures from the image of the coating surface defect test plate region, while the deep network extracts high-level semantic information such as the global context of the image. The window attention module divides the image embedding into local windows for efficient in-window self-attention, while the query pooling window attention module, based on window attention, achieves efficient downsampling through pooling calculations on the query matrix, while maintaining cross-scale feature dependencies. The image embedding does not change in dimension when passing through the window attention module, but its length and width are halved when passing through the query pooling window attention module. In one embodiment, to ensure the applicability of the deep learning model and further improve parallel computing efficiency, the interactive segmentation model fills the input image of the high-resolution image encoder into a square image with equal length and width.

[0035] The shallow network comprises the first eight sub-layers of the high-resolution image encoder, while the deep network comprises all sub-layers. Specifically, in one embodiment, the shallow network is composed of two window attention modules, one query pooled window attention module, two window attention modules, one query pooled window attention module, and two window attention modules stacked sequentially. The deep network is composed of all the sub-layers of the aforementioned shallow network, 13 window attention modules, one query pooled window attention module, and two window attention modules stacked sequentially.

[0036] In one embodiment, the test area image is split horizontally into two sub-images with similar length and width. After being padded to form squares, these sub-images are input into the interactive segmentation model. For example... Figure 2 As shown in the upper left part, the test area image undergoes dimensionality transformation, padding, and numerical normalization before being input into the interactive segmentation model. The resulting image feature vector is... ,in This is the batch dimension, which can take any value; here we take... , For the set of real numbers, , Set the width and height dimensions of the input image. Image feature vectors First, crop the image into four equal parts. , For the first A local image, The index number of the local image is used, and then it is downsampled into a global sub-image with the same height and width as the local image. , , Given the width and length dimensions of the local images, the four local images will only pass through the shallow network, while the global sub-image will pass through all parts of the entire high-resolution encoder, i.e., the deep network. The final output will be four local image embeddings and one global image embedding, resulting in five image embeddings: , For the first Image embedding, This is the index number for the image embedding. (2) The user click input encoder uses sine and cosine position encoding, which is used to encode the user's click input, i.e., the left-click input point. With right-click negative click point Sine and cosine coding are used to form user click coding embedding features to characterize the target detection region. , These are target designation markers for interactive segmentation models, explicitly indicating the detection regions of interest to the user. They guide the interactive segmentation model to focus on the semantic features of these key coordinates, enhancing the identification and localization of the target regions. It is a background exclusion marker for interactive segmentation models, marking non-target regions to help the interactive segmentation model distinguish between the target and the background, reduce interference from non-target regions, and improve the accuracy of target localization. This represents the number of input points. A self-attention calculation is performed before inputting the data to the decoder. This invention's interactive segmentation model incorporates click input, allowing users to adjust and optimize detection results based on mouse clicks.

[0037] Users can select target detection areas in the image of the coating surface defect test plate by clicking with the left and right mouse buttons. These target detection areas are then encoded using sine and cosine coding to form the embedded features of the detection area. Users can manually select target detection areas to adjust and optimize the detection results.

[0038] (3) Figure 3 As shown, the MVFM module, through the window attention module, integrates four local image embeddings with one global image embedding through two branches. The process involves both global and local feature information. Specifically, four local images are first embedded and stitched together, then the information is fused using a window self-attention module (identical to the window attention module in a high-resolution image encoder). Figure 3The middle part downsamples the stitched and merged local image embeddings and upsamples the global image embeddings, forming four image embeddings at two scales, with two image embeddings at each scale. The processing branches at the two scales are... Figure 3 The image displays different shades of color, with lighter colors representing larger-scale branches and darker colors representing smaller-scale branches. These two scales are designed to facilitate better information interaction and fusion between global and local image embeddings at different scales. The convolutional module (including convolutional layers, normalization layers, and activation function layers) fuses the four image embeddings into two new fused image embeddings of two scales, serving as the keys (K) and values ​​(V) for subsequent window attention. The original global image embedding and its upsampled output, along with the concatenated local image embedding and its downsampled output, serve as queries (Q). Different query (Q), key (K), and value (V) calculations are performed to perform information interaction calculations between the global and local image embeddings, fusing global semantic information into local texture information to obtain the image embedding after information interaction. Finally, in the MVFM module, the convolutional module fuses the six image embeddings from three parts, outputting a single image embedding with detailed local texture information. Image embedding A global image embedding with global semantic information output by a deep network of a high-resolution image encoder. This will be used as the input to the high-resolution decoder.

[0039] (4) Figure 2 As shown, the high-resolution decoder includes an interactive attention module, a self-attention module, and a multilayer perceptron module. The self-attention module maps individual input features to queries (Q), keys (K), and values ​​(V), calculates the similarity between Q and K to obtain attention weights, and then sums these weights over V. The interactive attention module maps queries (Q), keys (K), and values ​​(V) using different inputs. In the high-resolution decoder, this mainly includes attention computation from tokens (i.e., the output of self-attention) to images (using the token as Q and the image as K and V) and attention computation from images to tokens (using the image as Q and the token as K and V). The multilayer perceptron (MLP) module is a feedforward neural network consisting of an input layer, several fully connected hidden layers, and an output layer. Through fully connected interlayer connections and nonlinear activation functions, it learns complex nonlinear mappings from input to output.

[0040] Global Image Embedding and image embedding As respectively Figure 2 The global and local features in the input image are divided into two branches, left and right. Both branches include an interactive attention module and a shared multilayer perceptron module. Global image embedding is performed in both branches. and image embedding Embedded features with user click encoding respectively Multiple interactive attention calculations are performed, and information exchange is achieved through fully connected layers and convolutional modules, enabling the high-resolution decoder to perform calculations. , , The system analyzes the correlation information between the two branches and adjusts the output based on the user's click input. After the calculations of the two branches, an interactive attention operation is performed again. Then, the outputs of the two convolutions are multiplied by the matrix of multiple fully connected layers to obtain the final logit map. The logit map is mapped to a probability map using a sigmoid activation function, and then binarized with a threshold of 0.5. Regions with probabilities higher than the threshold are identified as defect regions, thus generating a mask map marking the defect regions, such as... Figure 5 As shown. Based on the mask image of the marked defect areas, the proportion of pixels in the rust defect area and the peeling defect area to the total number of pixels in the image, the actual longest side length of the smallest bounding rectangle of the largest blister defect area, and the number of blister defect areas are calculated respectively. Then, according to the preset numerical range rules of standard GB / T 1766-2008 and ISO 4628-3:2016, the corresponding rust defect level, peeling defect level, blister defect size level, and blister defect quantity level are generated.

[0041] Step 3, Interactive Segmentation Model Training Phase: Using the coating surface defect detection image dataset described in Step 1 and the interactive segmentation model constructed in Step 2, modules such as data reading, interactive point simulation, model forward inference, loss function calculation, backpropagation, and data saving are built to realize the complete model training process. During the training process, hyperparameters are optimized based on real-time training results to obtain the best interactive segmentation model.

[0042] The data reading module is responsible for batch loading image matrices and normalizing their parameters. The interaction point simulation module automatically generates interaction points based on the difference between the labels in the dataset and the model's predicted mask, providing these simulated interaction points as input to the model during training. Gradients are calculated synchronously during forward inference, providing a basis for subsequent backpropagation to update model parameters. The loss function uses a weighted sum of cross-entropy loss, focal loss, and dice loss. The backpropagation stage employs the AdamW optimizer, combined with mixed-precision training to improve efficiency. The data saving module is responsible for exporting hyperparameters, training set and validation set loss and accuracy data tables, and saving the best-performing model by comparing model performance at each stage. These modules implement the complete model training process, optimizing hyperparameters based on real-time training results to ultimately obtain the optimal model.

[0043] By inputting the image of the coating to be tested into the trained interactive segmentation model, the defect level assessment result can be obtained.

[0044] Step 4: Deployment of the Interactive Segmentation Model and Setup of the Coating Surface Defect Level Assessment Software: First, convert the trained interactive segmentation model to ONNX format, then deploy it to a C++ runtime environment; simultaneously, use Qt Creator to develop software, such as... Figure 6 As shown, software with functions including image acquisition, image preprocessing, interactive user detection, and detection result calculation and export was constructed. The coating surface defect level assessment software includes a camera interface, a routine detection interface, and an interactive detection interface; the camera interface is used to display real-time camera images, acquire test panel images, and start detection; the routine detection interface is used to display routine detection results, load and save the detection results of saved images; the interactive detection interface is used to display interactive detection results, undo the user's previous interaction point, and load and save the detection results of saved images.

[0045] In one embodiment, an interactive segmentation-based coating surface defect rating system is provided to implement the method described in the foregoing embodiments. The system includes the following modules: The training module is used to train the interactive segmentation model to obtain an interactive segmentation model for evaluating the defect level of the coating surface. The evaluation module is used to input the image of the coating test panel to be tested into the interactive segmentation model to obtain the defect level evaluation result. The interactive segmentation model includes a high-resolution image encoder, a user click input encoder, an MVFM module, and a high-resolution decoder. The high-resolution image encoder includes shallow and deep networks. The shallow network is used to extract low-level detail features from the image of the coating test panel, and the deep network is used to extract high-level semantic information from the image of the coating test panel, resulting in local and global image embeddings. The user click input encoder uses sine and cosine coding to form user click-coded embedding features representing the target detection region from the user's click input. The MVFM module fuses the local and global image embeddings and performs information interaction, outputting an image embedding with local texture detail information. The high-resolution decoder performs interactive attention calculation and information interaction between the global image embedding and the image embedding output by the MVFM module and the user click-coded embedding features, respectively, to obtain a log-odds map. Based on the log-odds map, a mask map marking the defect region is obtained, and based on the mask map, the defect level evaluation result is obtained.

[0046] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in the foregoing embodiments.

[0047] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described in the foregoing embodiments.

[0048] In one embodiment, the following experiment was conducted to verify the effectiveness of the method proposed in this invention: The dataset used in the experiment is the coating surface defect detection image dataset mentioned above, simulating the detection scenario after a real aging test. This dataset contains 3648 images, covering common defect types such as rust, peeling, and blistering, and is divided into training, testing, and validation sets in a 7:2:1 ratio for training, validating, and testing each model.

[0049] The experimental environment is configured as follows: Operating system: Windows 11.

[0050] Graphics card: NVIDIA L40 GPU.

[0051] Processor: AMD Ryzen 9 9950X processor.

[0052] Deep learning framework: PyTorch 2.3.1.

[0053] Programming environment: Python 3.9.

[0054] The following are the specific steps of the experiment: Using the same dataset, hyperparameters (including learning rate, learning rate adjustment parameters, batch size and number of iterations, regularization parameters and optimizer type), loss function, and other training parameters, common semantic segmentation models and interactive segmentation models were trained. Then, the same test set was used to test them with various evaluation parameters. The interactive segmentation model was simulated using four interactive points. The resulting data is shown in Table 1. The evaluation metric for segmentation accuracy used the mIOU (mean intersection-union ratio) score, commonly used in segmentation tasks. The specific calculation method is as follows: , among which | |To predict the area of ​​the segmentation result,| | represents the area of ​​the actual segmentation result. , There are N categories in total, each for the task to be divided.

[0055] Table 1 Comparison of Model Segmentation Results

[0056] In summary, in tests on coating surface defect detection image datasets, the method of this invention can more effectively complete the segmentation task of coating surface defects, resulting in higher detection accuracy in actual detection processes. Furthermore, this method effectively incorporates the user's interpretation experience into the detection process through mouse clicks, further improving the model's accuracy.

[0057] The preferred embodiments described above are merely illustrative of the principles and practical applications of this invention, to help those skilled in the art understand and utilize it. It should be noted that the scope of this invention is not limited to the specific embodiments described above; any equivalent substitutions, obvious modifications, or alterations made by those skilled in the art under the guidance of this specification fall within the substantive protection scope of this invention and should be protected by it.

Claims

1. A method for evaluating the surface defect level of a coating based on interactive segmentation, characterized in that, The image of the coating to be tested is input into the trained interactive segmentation model to obtain the defect level assessment result; The interactive segmentation model includes a high-resolution image encoder, a user click input encoder, an MVFM module, and a high-resolution decoder. The high-resolution image encoder includes a shallow network and a deep network. The shallow network is used to extract low-level detail features from the coating image under test, and the deep network is used to extract high-level semantic information from the coating image under test, resulting in local image embedding and global image embedding. The user click input encoder is used to form the user's click input into user click encoding embedding features representing the target detection region using sine and cosine coding. The MVFM module is used to fuse local image embeddings and global image embeddings and perform information interaction, outputting image embeddings with local texture detail information; the high-resolution decoder is used to perform interactive attention calculation and information interaction between the global image embedding and the image embedding output by the MVFM module and the user click encoded embedding features, respectively, to obtain a log-odds map, a mask map of the marked defect region is obtained based on the log-odds map, and the defect level assessment result is obtained based on the mask map.

2. The method for evaluating the surface defect level of a coating based on interactive segmentation according to claim 1, characterized in that, The interactive segmentation model is trained based on a coating surface defect detection image dataset, which is constructed through the following steps: Images of standard test plates after aging tests are collected, including images of various defect types. Defects in standard test plate images are labeled and image labels are exported to obtain a dataset of coating surface defect detection images.

3. The method for evaluating the surface defect level of a coating based on interactive segmentation according to claim 1, characterized in that, The high-resolution image encoder is composed of two types of sub-layers stacked together: a window attention module and a query pooling window attention module. The deep network contains all the sub-layers of the high-resolution image encoder, while the first few sub-layers in the high-resolution image encoder are shallow networks. The window attention module is used to divide the image embedding into local windows for efficient in-window self-attention. The query pooling window attention module is used to achieve efficient downsampling by pooling the query matrix on the basis of window attention, while maintaining cross-scale feature dependencies.

4. The method for evaluating the surface defect level of a coating based on interactive segmentation according to claim 1, characterized in that, Image feature vectors are obtained from the images of the coating test plate. The image feature vectors are cropped into multiple local images of equal size, and then downsampled into a global sub-image with the same height and width as the local images. Multiple local images are processed by a shallow network for feature extraction, and multiple local image embeddings are output. The global sub-image is processed by a deep network for semantic information extraction, and the global image embedding is obtained.

5. The method for evaluating the surface defect level of a coating based on interactive segmentation according to claim 4, characterized in that, Perform the following operations in the MVFM module: After multiple local image embeddings are stitched together, information is fused through a window self-attention module. The stitched and fused local image embeddings are downsampled, and the global image embeddings are upsampled to obtain multiple image embeddings at two scales. The multiple image embeddings are then fused through a convolution module to obtain a new fused image embedding at two scales. The global image embedding and its upsampled output, the stitched local image embedding and its downsampled output are used as queries to perform different queries, key and value calculations, so as to perform information interaction calculation between the global image embedding and the local image embedding, and obtain the image embedding after information interaction; The image embedding after information interaction is fused to obtain an image embedding with local texture detail information.

6. The method for evaluating the surface defect level of a coating based on interactive segmentation according to claim 1, characterized in that, In the user click input encoder, the sine and cosine position encoding encodes the user's left-click positive input point and right-click negative input point to form the user click encoding embedding feature.

7. A method for evaluating the surface defect level of a coating based on interactive segmentation according to any one of claims 1-6, characterized in that, In a high-resolution decoder, perform the following operations: The global image embedding and the image embedding output by the MVFM module are used as inputs to the global and local features, respectively, in two branches. Both branches include an interactive attention module and a shared multilayer perceptron module. In both branches, the global image embedding and the image embedding output by the MVFM module perform multiple interactive attention calculations with the user click-encoded embedding features, and exchange information at the same time. This enables the high-resolution decoder to calculate the correlation information between the global image embedding, the image embedding output by the MVFM module, and the user click-encoded embedding features, so as to adjust the output results according to the user's click input. After the calculations of the two branches, the interactive attention operation is performed again. Then, the two convolutions are multiplied by the output of the multilayer perceptron module to obtain the log-odds graph. Map the log-odds plot to a probability plot; The probability map is binarized, and areas with a probability higher than a set threshold are identified as defect areas, generating a mask map that marks the defect areas. Based on the mask image of the marked defect region, defect calculation is performed to generate the defect level.

8. A coating surface defect level assessment system based on interactive segmentation, characterized in that, The system for implementing the method according to any one of claims 1-7, the system comprising the following modules: The training module is used to train the interactive segmentation model to obtain an interactive segmentation model for evaluating the defect level of the coating surface. The evaluation module is used to input the image of the coating test panel to be tested into the interactive segmentation model to obtain the defect level evaluation result. The interactive segmentation model includes a high-resolution image encoder, a user click input encoder, an MVFM module, and a high-resolution decoder. The high-resolution image encoder includes a shallow network and a deep network. The shallow network is used to extract low-level detail features from the image of the coating test panel, and the deep network is used to extract high-level semantic information from the image of the coating test panel, resulting in local image embedding and global image embedding. The user click input encoder is used to form the user's click input into user click encoding embedding features representing the target detection area using sine and cosine coding. The MVFM module is used to fuse local image embeddings and global image embeddings and perform information interaction, outputting image embeddings with local texture detail information; the high-resolution decoder is used to perform interactive attention calculation and information interaction between the global image embedding and the image embedding output by the MVFM module and the user click encoded embedding features, respectively, to obtain a log-odds map, a mask map of the marked defect region is obtained based on the log-odds map, and the defect level assessment result is obtained based on the mask map.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images