Defective data cleaning method, device, storage medium and electronic device

Abstract

The present application relates to a method, device, storage medium, and electronic device for cleaning defect data. The method comprises: obtaining multi-scale context features of the defect data to be cleaned, frequency domain features of the initial defect area in the defect data to be cleaned, and spatial features of the initial defect area, thereby obtaining scene features of the defect data to be cleaned; predicting a cleaning threshold for each pixel in the initial defect area based on the combined features and sensitivity parameters; and cleaning the pixels in the initial defect area based on the cleaning threshold for each pixel to obtain cleaned defect data. The present application solves the technical problem of low cleaning accuracy of data cleaning methods in the prior art.

Description

Technical Field

[0001] The present application relates to the field of defective data cleaning, and in particular to a defective data cleaning method, device, storage medium, and electronic device. Background Art

[0002] In the prior art, image data can be cleaned. The purpose of data cleaning is to delete any data that does not meet the requirements and retain any data that meets the requirements. For example, in the scenario of determining whether a workpiece has surface defects, the image data of the workpiece can be cleaned to remove any incorrectly identified defects, thereby obtaining accurate image data.

[0003] However, the data cleaning methods in the existing technology have coarse cleaning granularity and usually perform image-level screening. For an image that is judged to be "qualified" after screening, if it still contains noise pixels, or the boundaries of the insulator area are unclear, or there are small defective areas that are mistakenly identified as background, the solution cannot handle the details inside these images, cannot refine the information inside the image, and the cleaning is not accurate. Summary of the Invention

[0004] The present application provides a defective data cleaning method, device, storage medium and electronic device to solve the technical problem of low cleaning accuracy of data cleaning methods in the prior art.

[0005] In the first aspect, the present application provides a method for cleaning defect data, including: obtaining multi-scale context features of the defect data to be cleaned, frequency domain features of the initial defect area in the defect data to be cleaned, and spatial features of the initial defect area, wherein the multi-scale context features are context features at different receptive fields and different resolutions in the process of a semantic segmentation model identifying the defect data to be cleaned, the frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect area or its neighborhood, and the spatial features are used to segment the initial defect area from other areas of the defect data to be cleaned; obtaining scene features of the defect data to be cleaned, wherein the scene features are used to indicate the scene when the defect data to be cleaned is generated; predicting a cleaning threshold for each pixel in the initial defect area based on a combined feature and a sensitivity parameter, wherein the combined feature is a feature obtained by combining the multi-scale context features, the frequency domain features, and the spatial features, and the sensitivity parameter is a parameter determined based on the scene features; according to the cleaning threshold of each pixel, the pixels in the initial defect area are cleaned to obtain cleaned defect data.

[0006] In a second aspect, the present application provides a defect data cleaning device, comprising: a first acquisition module, configured to acquire multi-scale context features of the defect data to be cleaned, frequency domain features of an initial defect area in the defect data to be cleaned, and spatial features of the initial defect area, wherein the multi-scale context features are context features at different receptive fields and different resolutions in the process of a semantic segmentation model identifying the defect data to be cleaned, the frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect area or its neighborhood, and the spatial features are used to segment the initial defect area from other areas of the defect data to be cleaned; a second acquisition module, configured to acquire scene features of the defect data to be cleaned, wherein the scene features are used to indicate the scene when the defect data to be cleaned is generated; a prediction module, configured to predict a cleaning threshold for each pixel in the initial defect area based on a combined feature and a sensitivity parameter, wherein the combined feature is a feature obtained by combining the multi-scale context features, the frequency domain features, and the spatial features, and the sensitivity parameter is a parameter determined based on the scene features; a cleaning module, configured to clean the pixels in the initial defect area according to the cleaning threshold of each pixel to obtain cleaned defect data.

[0007] As an optional example, the first acquisition module includes: a first acquisition unit, used to adjust the resolution of the above-mentioned defect data to be cleaned to obtain the above-mentioned defect data to be cleaned with multiple resolutions; inputting the above-mentioned defect data to be cleaned with each resolution into a semantic segmentation model, and the above-mentioned semantic segmentation model uses different receptive fields to extract features from the above-mentioned defect data to be cleaned, and obtain context features corresponding to different receptive fields at each resolution to obtain the above-mentioned multi-scale context features.

[0008] As an optional example, the first acquisition module includes: a second acquisition unit, configured to extract features of the initial defect area or a neighborhood of the initial defect area through wavelet transform to obtain the frequency domain features.

[0009] As an optional example, the first acquisition module includes: a third acquisition unit, used to use a region growing algorithm to segment edge pixels inside or at the edge of the above-mentioned initial defect area; determine the connected domain surrounded by the above-mentioned edge pixels; and aggregate the features of the above-mentioned connected domain to obtain the above-mentioned spatial features.

[0010] As an optional example, the above-mentioned device also includes: a comparison module, which is used to store the multi-scale context features of any defect data in a dynamic feature library after obtaining the multi-scale context features of any defect data, wherein the above-mentioned dynamic feature library stores the multi-scale context features of different defect data; after obtaining the multi-scale context features of the above-mentioned defect data to be cleaned, the frequency domain features of the initial defect area in the above-mentioned defect data to be cleaned, and the spatial features of the above-mentioned initial defect area, the above-mentioned combined features are compared with the features in the above-mentioned dynamic feature library; and the validity or invalidity of the above-mentioned combined features is determined based on the comparison results.

[0011] As an optional example, the prediction module includes: a determination unit, used to determine the lighting conditions of the scene during production and the material type of the defect data to be cleaned based on the above-mentioned scene characteristics; determine a first sensitivity range based on the above-mentioned material type; and determine the above-mentioned sensitivity parameter from the above-mentioned first sensitivity range based on the above-mentioned lighting conditions.

[0012] As an optional example, the above-mentioned prediction module includes: a prediction unit, which is used to input the above-mentioned combined features and the above-mentioned sensitivity parameters into a multi-layer perceptron, and the above-mentioned multi-layer perceptron generates a cleaning threshold for each pixel point that is consistent with the above-mentioned sensitivity parameters.

[0013] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the memory stores a computer program, and the processor is configured to implement any one of the above-mentioned defect data cleaning methods when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute any of the above-mentioned defect data cleaning methods of the present application.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the solution provided by the embodiment of the present application obtains the multi-scale context features of the defect data to be cleaned, the frequency domain features of the initial defect area in the defect data to be cleaned, and the spatial features of the initial defect area, wherein the multi-scale context features are context features at different receptive fields and different resolutions in the process of the semantic segmentation model identifying the defect data to be cleaned, the frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect area or its neighborhood, and the spatial features are used to segment the initial defect area from other areas of the defect data to be cleaned; obtain the scene features of the defect data to be cleaned, wherein the scene features are used to indicate the defect data to be cleaned. Cleaning the scene when defect data is generated; predicting the cleaning threshold of each pixel in the above-mentioned initial defect area based on the combined features and sensitivity parameters, wherein the above-mentioned combined features are features obtained by combining the above-mentioned multi-scale context features, the above-mentioned frequency domain features and the above-mentioned spatial features, and the above-mentioned sensitivity parameters are parameters determined based on the above-mentioned scene features; cleaning the pixels in the above-mentioned initial defect area according to the cleaning threshold of each pixel to obtain cleaned defect data, so that in the process of cleaning the defect data to be cleaned, the overall features of the defect data to be cleaned and the local defect texture and edge frequency characteristics of the initial defect area and the boundary between the initial defect area and other areas can be comprehensively considered to clean the defect data to be cleaned, thereby improving the accuracy of cleaning. Moreover, in this process, the cleaning threshold of each pixel in the cleaning process is accurately determined in combination with the scene, further improving the cleaning accuracy of the defect data to be cleaned. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 A flowchart of a defect data cleaning method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the receptive field of a defect data cleaning method provided in an embodiment of the present application;

[0021] Figure 3 A neighborhood diagram of a defect data cleaning method provided in an embodiment of the present application;

[0022] Figure 4 A neighborhood diagram of another defect data cleaning method provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of a defect data cleaning device provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] The disclosure below provides many different embodiments or examples for implementing different configurations of the present invention. To simplify the disclosure of the present invention, the components and configurations of specific examples are described below. Of course, these are merely examples and are not intended to limit the present invention. In addition, the present invention may repeat reference numerals and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or configurations discussed.

[0027] In order to solve the technical problem of low cleaning accuracy of data cleaning methods in the prior art, the present application provides a defective data cleaning method that can improve the cleaning accuracy of defective data to be cleaned.

[0028] Figure 1 This is a flow chart of a defect data cleaning method provided in an embodiment of the present application. Figure 1 As shown, the above defect data cleaning method includes:

[0029] S101, obtaining multi-scale context features of the defect data to be cleaned, frequency domain features of the initial defect region in the defect data to be cleaned, and spatial features of the initial defect region, wherein the multi-scale context features are context features at different receptive fields and resolutions during the process of the semantic segmentation model identifying the defect data to be cleaned, the frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect region or its neighborhood, and the spatial features are used to segment the initial defect region from other regions of the defect data to be cleaned;

[0030] S102, obtaining a scene feature of the defect data to be cleaned, wherein the scene feature is used to indicate the scene when the defect data to be cleaned is generated;

[0031] S103, predicting a cleaning threshold for each pixel in the initial defect area based on the combined features and sensitivity parameters, where the combined features are features obtained by combining multi-scale context features, frequency domain features, and spatial features, and the sensitivity parameters are parameters determined based on scene features;

[0032] S104 , cleaning the pixels in the initial defect area according to the cleaning threshold of each pixel to obtain cleaned defect data.

[0033] The defect data in the above-mentioned defect data cleaning method may be image data obtained by photographing or by image processing, image generation, etc. The image data includes objects with defects or without defects. For example, if defect data cleaning is performed on a workpiece, the defect data to be cleaned may be an image including the workpiece. If defect data cleaning is performed on a highway, the defect data to be cleaned may be an image including the highway. Depending on the specific usage scenario, the defect data to be cleaned may be an image including different objects. The objects in the image may include defects or may not include defects.

[0034] Taking the application of workpiece image data cleaning as an example, multiple images of the workpiece may be acquired, and the workpiece in the image may or may not contain defects. The cleaning process then involves cleaning out the data from the image containing the workpiece that has been incorrectly identified, such as identifying a non-defective area in the workpiece as a defective area, or identifying a background area as a defective area. For cleaning this data, the defect data cleaning solution of this embodiment can be used. After cleaning, the defect data in the workpiece image is accurately identified and can be used for other processing.

[0035] In this embodiment, if defect data to be cleaned is to be cleaned, multiple features of the defect data to be cleaned are obtained. These features include multi-scale context features, frequency features of the initial defect region in the defect data to be cleaned, and spatial features of the initial defect region in the defect data to be cleaned. The three features obtained are combined to obtain a combined feature. The combined feature and sensitivity parameter are used to determine the cleaning threshold for each pixel in the initial defect region of the defect data to be cleaned. In this way, the cleaning threshold of each pixel is used to determine whether the pixel is defective data. Pixels that do not meet the cleaning threshold are cleaned. This eliminates some incorrectly identified defect data from the initial defect data, resulting in accurate defect data.

[0036] The initial defective area in the defective data to be cleaned may be one or more defective areas in the defective data to be cleaned identified after pre-identification. The defective area may be a whole or multiple parts, and the multiple parts may be connected or disconnected.

[0037] The aforementioned scene features are the scene in which the defect data to be cleaned is generated. For example, if the defect data to be cleaned is a captured image of the workpiece surface, the scene data is the scene in which the workpiece was captured, or the scene in which the workpiece is captured in the captured image. The sensitivity parameter can be determined based on the scene features. The sensitivity parameter varies with different scenes and is used to indicate the sensitivity when determining the cleaning threshold. Factors such as scene lighting, the distance between the camera and the workpiece when capturing the image, and the background all affect the sensitivity parameter.

[0038] This embodiment can predict a cleaning threshold for each pixel point in the initial defect area based on the combined features and sensitivity parameters, and determine whether to clean the pixel point by determining the pixel value of the pixel point in the initial defect area or the difference between the cleaning prediction value of the pixel point and the cleaning threshold.

[0039] The solution provided by the embodiment of the present application obtains multi-scale context features of the defect data to be cleaned, frequency domain features of the initial defect area in the defect data to be cleaned, and spatial features of the initial defect area, wherein the multi-scale context features are context features at different receptive fields and different resolutions during the semantic segmentation model's identification of the defect data to be cleaned, the frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect area or its neighborhood, and the spatial features are used to separate the initial defect area from other areas of the defect data to be cleaned; obtains scene features of the defect data to be cleaned, wherein the scene features are used to indicate the scene when the defect data to be cleaned is generated; predicts a cleaning threshold for each pixel in the initial defect area based on the combined features and sensitivity parameters, wherein the combined features are features obtained by combining the multi-scale context features, the frequency domain features, and the spatial features, and the sensitivity parameters are parameters determined based on the scene features; and cleans the pixels in the initial defect area based on the cleaning threshold of each pixel to obtain cleaned defect data. Therefore, in the process of cleaning the defect data to be cleaned, the overall features of the defect data to be cleaned, the local defect texture and edge frequency characteristics of the initial defect area, and the boundary between the initial defect area and other areas can be comprehensively considered to clean the defect data to be cleaned, thereby improving the accuracy of cleaning. Moreover, during this process, the cleaning threshold of each pixel point in the cleaning process is accurately determined in combination with the scene, further improving the cleaning accuracy of the defective data to be cleaned.

[0040] As an optional example, obtaining multi-scale context features of the defect data to be cleaned includes: adjusting the resolution of the defect data to be cleaned to obtain defect data to be cleaned with multiple resolutions; inputting the defect data to be cleaned with each resolution into a semantic segmentation model, and the semantic segmentation model uses different receptive fields to extract features of the defect data to be cleaned, obtaining context features corresponding to different receptive fields at each resolution, and thus obtaining multi-scale context features.

[0041] In this embodiment, it is an optional way to obtain multi-scale context features of the defect data to be cleaned. When obtaining multi-scale context features, a semantic segmentation model can be used to obtain multi-scale context features of the defect data to be cleaned. If the defect data to be cleaned includes multiple pictures, a semantic segmentation model can be used for each picture to obtain multi-scale context features of each picture. Each picture can be directly input into the semantic segmentation model, and the semantic segmentation model uses different receptive fields to obtain context features and obtain multi-scale context features. In addition, different resolutions can be set for each picture, and pictures of each resolution are input into the semantic segmentation model. The semantic segmentation model uses different receptive fields to extract context features of pictures of each resolution to obtain multi-scale context features.

[0042] The receptive field mentioned above refers to how many pixels in the image are used by the semantic segmentation model as feature points of the extracted context features. For example, 3×3 pixels in the image are extracted as one feature point in the above features, or 4×4 pixels in the image are extracted as one feature point in the context features. For example, Figure 2 As shown, Figure 2 In the image, 3×3 pixels are extracted as a feature point with a 3×3 receptive field.

[0043] Taking the cleaning of defect data on the workpiece surface as an example, for the images of the workpiece surface, the image resolution is adjusted to obtain images of multiple resolutions. The images are input into the semantic segmentation model, and features are extracted using different receptive fields to obtain multi-scale context features of the image.

[0044] In this example, the context features of cleaning defect data are obtained from images of different resolutions in different receptive fields, thereby ensuring the comprehensiveness and accuracy of the context features.

[0045] As an optional example, obtaining the frequency domain features of the initial defect area in the defect data to be cleaned includes: extracting features of the initial defect area or a neighborhood of the initial defect area through wavelet transform to obtain the frequency domain features.

[0046] In this embodiment, when obtaining the frequency characteristics of the initial defect area, the characteristics of the initial defect area itself can be obtained, or the characteristics of the neighborhood of the initial defect area can be used as the frequency characteristics of the initial defect area. The neighborhood of the initial defect area is the area obtained by expanding the outermost pixel point of the initial defect area by one pixel point. When expanding one pixel point outward, it is divided into expanding one pixel point in the horizontal and vertical directions, and expanding one pixel point in the horizontal, vertical and oblique directions. For example, Figure 3 As shown in the figure, the black pixels are the pixels of the initial defect area. If the pixels are expanded horizontally and vertically, then Figure 3 The pixels indicated by the arrows in are the neighborhood pixels. Figure 4 As shown in , if the pixels are expanded in the horizontal, vertical and oblique directions, the pixels in the oblique directions must also be considered as neighboring pixels. Figure 4 The arrows in the middle indicate the pixels in the tilted direction.

[0047] The expanded area is the neighborhood of the initial defect area. When obtaining frequency features, wavelet transform can be used to extract the features of the initial defect area to obtain frequency features, or the features of the neighborhood of the initial defect area can be extracted to obtain frequency features. Extracting the frequency features of the initial defect area can identify the image content of the initial defect area, while identifying the features of the neighborhood can learn the image content of the neighborhood. Choose one of the two to perform frequency feature extraction.

[0048] Taking the cleaning of defect data on the workpiece surface as an example, for the initial defect area in the workpiece image, the wavelet transform method can be used to extract the frequency characteristics of the initial defect area, or the initial defect area can be expanded outward by one pixel, and the expanded area is determined as the neighborhood, and the frequency characteristics of the neighborhood are extracted.

[0049] As an optional example, obtaining the spatial features of the initial defect area includes: using a region growing algorithm to segment edge pixels inside or at the edge of the initial defect area; determining a connected domain surrounded by the edge pixels; and aggregating features of the connected domain to obtain spatial features.

[0050] In this embodiment, the spatial features of the initial defect area are used to represent the boundary between the initial defect area and other areas. For the initial defect area, one or more pixels can be selected from the inside of the initial defect area or the edge of the initial defect area to determine as the initial pixel point, and then expansion starts from the initial pixel point. If it is a pixel point within the selected initial defect area, then expansion starts from the pixel point to the edge of the initial defect area. If the pixel point is a pixel point at the edge of the initial defect area, then expansion starts from the pixel point to the inside of the initial defect area. After the expansion, the pixel point is judged to be an edge pixel point, a pixel point inside the initial defect area, or a pixel point outside the initial defect area, thereby identifying all edge pixel points. All edge pixel points surround the initial defect area and are connected to each other to form a connected domain. The features of the connected domain are aggregated to obtain the spatial features.

[0051] If the initial defect regions consist of multiple disconnected regions in the image, one or more pixels are selected from the interior or edge of each initial defect region to initiate expansion. The expansion method is not described in detail here. After expansion, each of the multiple initial defect regions corresponds to a connected domain, and each connected domain is fused to generate a feature. The features generated by the fusion of each connected domain are collectively used as the spatial feature.

[0052] Taking the cleaning of the defect data to be cleaned on the workpiece surface as an example, for the initial defect area in the workpiece image, a pixel point is selected from the inside of the initial defect area to start expanding toward the edge of the initial defect area. After expansion, the edge pixel points of the initial defect area are determined, and then the edge pixel points are connected to obtain a connected domain. The connected domain is fused to obtain a fusion feature, and the fusion feature is used as a spatial feature.

[0053] This embodiment uses spatial features to learn the boundaries of the initial defect area. In this process, edge pixels are determined by expanding a pixel point, thereby improving the accuracy of determining edge pixels.

[0054] As an optional example, the method also includes: after obtaining the multi-scale context features of arbitrary defect data, storing the multi-scale context features of arbitrary defect data in a dynamic feature library, wherein the dynamic feature library stores the multi-scale context features of different defect data; after obtaining the multi-scale context features of the defect data to be cleaned, the frequency domain features of the initial defect area in the defect data to be cleaned, and the spatial features of the initial defect area, comparing the combined features with the features in the dynamic feature library; and determining whether the combined features are valid or invalid based on the comparison results.

[0055] In this embodiment, a dynamic feature library can be pre-configured to store multi-scale contextual features of any defect data. For example, the multi-scale contextual features identified by the semantic segmentation model for workpiece images can be stored in the dynamic feature library. Alternatively, the multi-scale contextual features identified by the semantic segmentation model for road surface images can also be stored in the dynamic feature library. The dynamic feature library then stores multi-scale contextual features for images of different objects in different scenarios.

[0056] After combining the multi-scale context features, frequency domain features, and spatial features obtained by identifying the defect data to be cleaned to obtain the combined features, they can be compared with the multi-scale context features in the dynamic feature library. The purpose of the comparison is to determine the availability and correctness of the fused features.

[0057] If the comparison includes features similar to or associated with the combined features in the dynamic feature library, the combined features are determined to be available and can be used for subsequent predictions. If it is determined that the dynamic feature library does not include features similar to or associated with the combined features, the combined features are determined to be unavailable.

[0058] After determining the combined features through the above process, the pixel cleaning threshold is predicted. This prediction utilizes not only the combined features but also the scene features of the defect data to be cleaned. These are the characteristics of the scene in which the defect data was generated, such as lighting, background, object material, and color.

[0059] In one example, determining the sensitivity parameter includes: determining the lighting conditions of the production scene and the material type of the defect data to be cleaned based on scene characteristics; determining a first sensitivity range based on the material type; and determining the sensitivity parameter from the first sensitivity range based on the lighting conditions.

[0060] In this embodiment, the sensitivity parameter can be determined based on the material and lighting conditions of the objects in the scene features. Different materials and lighting conditions result in different sensitivity parameters. Different materials can correspond to different sensitivity parameter ranges. For example, metal may correspond to one sensitivity parameter range, while plastic may correspond to another sensitivity parameter range, with the values ​​of the ranges varying. After determining the material, the sensitivity parameter is determined from these ranges based on the lighting conditions. Better lighting conditions can result in smaller sensitivity parameters, indicating higher precision requirements.

[0061] As an optional example, predicting the cleaning threshold of each pixel in the initial defect area based on the combined features and sensitivity parameters includes: inputting the combined features and sensitivity parameters into a multi-layer perceptron, and the multi-layer perceptron generating a cleaning threshold for each pixel that is consistent with the sensitivity parameters.

[0062] In this embodiment, after determining the combined features and sensitivity parameters, the cleaning threshold for each pixel in the initial defect area can be predicted. The combined parameters and sensitivity parameters can be input into a multilayer perceptron, which then determines the pixel cleaning threshold for the combined parameters based on the sensitivity parameter values. After the prediction, each pixel in the initial defect area corresponds to a cleaning threshold.

[0063] For the combined features, recognition continues, determining the defect probability of each pixel and comparing it to the corresponding cleaning threshold. In the comparison results, pixels with a defect probability higher than the cleaning threshold are retained or confirmed as defects. Pixels with a defect probability lower than the cleaning threshold are cleaned and considered non-defective pixels and not part of the initial defect area. After adjustment, some pixels in the initial defect area are cleaned, resulting in a cleaned defect area.

[0064] In this embodiment, when determining pixels that do not belong to the initial defect area, if the pixels that do not belong to the initial defect area are located at the edge of the initial defect area, they can be cleaned. If the pixels that do not belong to the initial defect area are located outside the edge of the initial defect area, further judgment is required to determine whether to clean them. The further judgment is made by checking the difference between the cleaning threshold of the pixel and the cleaning threshold of the surrounding pixels. If the difference is small, the pixel is cleaned. If the difference is large, the pixel threshold of the pixel may be inappropriate, and therefore the pixel can be retained. If the pixels that do not belong to the initial defect area are located outside the edge of the initial defect area, and multiple pixels are connected to form an area, the difference between the pixel threshold of the pixels in the area and the pixel threshold of the pixels in the initial defect area can be determined. First, determine the pixel thresholds of the pixels in the area that differ too much from the pixel thresholds of the pixels in the initial defect area. If the number of pixels with too much difference in pixel thresholds is small, the pixels with too much difference are determined as pixels that are not to be cleaned. In addition, in this area, if the pixel thresholds of a large number of pixels are too different from the pixel thresholds of the pixels in the initial defect area, the area still needs to be cleaned.

[0065] The following describes the above defect data cleaning method in conjunction with a scenario of cleaning defect data of a workpiece.

[0066] Take the defect data as an example of a defect image obtained by photographing the surface of a workpiece.

[0067] First, the initial defect region must be determined and multi-scale contextual features acquired. The defect image is processed using a semantic segmentation network to initially identify and segment the defect region. During the segmentation process, multi-scale contextual feature maps of the defect region are extracted from multiple intermediate layers of the semantic segmentation network (corresponding to different receptive fields and resolutions). These feature maps contain information about the defect's shape, texture, and neighborhood at different spatial scales.

[0068] The extracted high-quality, representative multi-scale contextual features are stored in a dynamic feature library. This library is continuously updated and expanded as more data is processed and feedback on the cleaning effect is received, and is used for subsequent feature comparison and model optimization.

[0069] The initial defect region identified above is the area of ​​the workpiece surface where defects are initially determined in the workpiece image. For example, the area where scratches are located on the workpiece surface. This solution is then used to adjust the image to accurately determine the defect region.

[0070] For the initial defect area, frequency domain features and spatial features are extracted. Frequency domain features are extracted by performing frequency domain analysis on the initial defect area or the neighborhood of the initial defect area that has been initially segmented. Wavelet transform is used to extract wavelet coefficient features that can reflect the texture and edge frequency characteristics of the defect, and distinguish between real defects and high-frequency noise or background texture. Spatial features are extracted by applying a region growing algorithm inside or at the edge of the initial defect area that has been initially segmented. Pixels with a higher probability of being output by the segmentation network are used as seed points, and the algorithm expands outward according to the preset region growing criteria to accurately outline the spatially connected domain of the defect, aggregate the spatially continuous defect parts, and separate the background noise.

[0071] The multi-scale contextual features extracted from the semantic segmentation network, the frequency-domain features obtained from frequency-domain wavelet analysis, and the spatial connectivity features extracted by the region growing algorithm are fused. The fusion strategy includes feature concatenation and a weighted sum attention mechanism to form a more comprehensive and robust integrated feature representation of the initial defect area. The fused features are then compared or correlated with features in the dynamic feature library, leveraging existing knowledge in the library to assist in determining the validity of the current features and further enhance the feature representation.

[0072] For scenes where workpieces are photographed, global or local scene features are obtained (judged through statistical characteristics of the image, metadata, and specific scene recognition modules) to identify scene features of multiple factors such as the material type, lighting conditions, and background of the current production scene.

[0073] Based on the identified scene characteristics, the overall cleaning sensitivity parameters are adjusted. For example, for materials with complex surface textures, sensitivity to subtle fluctuations should be reduced to avoid misidentifying normal textures as defects. For scenes requiring detection of minute defects, sensitivity should be increased. Sensitivity should be increased if background interference is significant, and reduced if lighting conditions are poor. By combining these scene conditions, the sensitivity parameters are determined, such as a weighted sum or weighted average of the above conditions.

[0074] The fused combined features and the adjusted scene sensitivity parameters are fed into a lightweight multilayer perceptron (MLP) to predict the cleaning threshold for each pixel to identify a true defect. Different image regions, different defect types, and even different parts of the same defect may exhibit different feature representations. For example, the edge of a defect may have low contrast, while the center may have high contrast. Using a single threshold may result in loss of edge information or the inclusion of noise near the center. Pixel-level thresholding allows for the application of a judgment criterion best suited to each pixel's local characteristics, thereby more accurately preserving true defects and removing noise. This allows for refined and adaptive data cleaning, overcoming the limitations of global or simple thresholding methods. The MLP model is designed to be lightweight to ensure efficient prediction. It should be noted that in this embodiment, the pixel threshold can be predicted for every pixel in the entire image, for every pixel in the initial defect area, or for every pixel within a predetermined range including the initial defect area and the beginning and end.

[0075] Using the predicted pixel-level cleaning threshold, the initial defect areas from the initial segmentation are adjusted. For each pixel, the predicted defect probability is compared with the corresponding threshold. Pixels below the threshold are considered noise or background and can be removed, marked as non-defective, or smoothed. Pixels above the threshold are retained or confirmed as defects. This process allows for precise correction of the original segmentation results, removing false-positive pixels while preserving true defect details.

[0076] The cleaned data is output, thus obtaining an image with an accurate defect mask.

[0077] In addition, the cleaned high-quality data is fed back to the semantic segmentation network for incremental training of the model, enabling the segmentation model to learn from cleaner data and improve the accuracy of its subsequent segmentation.

[0078] Feature modeling parameter optimization: Prepare an independent validation dataset. Use the current parameter configuration (including feature extraction method, fusion strategy, MLP model parameters, sensitivity adjustment logic, etc.) to process the validation set data and use the cleaned results to train / evaluate the segmentation model. Calculate the performance metrics of the segmentation model on the validation set, particularly the intersection over union (IoU). Based on the performance of the IoU metric, use the backpropagation algorithm to optimize key parameters that affect the data cleaning effect. These parameters include: feature fusion weights, region growing criterion parameters, MLP model weights, and which features are involved in modeling. Through this closed-loop feedback mechanism, the system can automatically adjust feature modeling and cleaning strategies based on the actual cleaning results, continuously improving the accuracy and adaptability of cleaning.

[0079] This technical solution, through dynamic feature modeling, scene recognition, and adaptive sensitivity adjustment, can automatically adapt to complex and changing industrial scenarios, such as varying material types, lighting variations, and defect morphologies, without the need for frequent manual parameter adjustments. By combining the multi-scale contextual understanding capabilities of deep learning with the strengths of traditional image processing methods (wavelet analysis and region growing) in texture and spatial detail, it achieves comprehensive and accurate characterization of defect features. Furthermore, pixel-level threshold prediction further enhances cleaning precision. A lightweight MLP method is used for threshold prediction, ensuring processing speed. An automated closed-loop feedback optimization mechanism reduces manual intervention and improves overall efficiency. By feeding cleaned data back into the segmentation model for training and reversely optimizing feature modeling parameters based on the validation set's Intersection of Union (IoU), the system continuously learns and improves during use, achieving self-improvement in performance. The introduction of a dynamic feature library and multi-source feature fusion enhance the system's resistance to noise and interference.

[0080] Figure 5 This is a schematic diagram of the structure of a defect data cleaning device provided in an embodiment of the present application. Figure 5 As shown, the above defect data cleaning device includes:

[0081] The first acquisition module 501 is used to obtain multi-scale context features of the defect data to be cleaned, frequency domain features of the initial defect area in the defect data to be cleaned, and spatial features of the initial defect area. The multi-scale context features are context features at different receptive fields and resolutions used by the semantic segmentation model to identify the defect data to be cleaned. The frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect area or its neighborhood. The spatial features are used to segment the initial defect area from other areas of the defect data to be cleaned.

[0082] The second acquisition module 502 is used to acquire scene features of the defect data to be cleaned, wherein the scene features are used to indicate the scene when the defect data to be cleaned is generated;

[0083] Prediction module 503, configured to predict a cleaning threshold for each pixel in the initial defect area based on the combined features and sensitivity parameters, where the combined features are features obtained by combining multi-scale context features, frequency domain features, and spatial features, and the sensitivity parameters are parameters determined based on scene features;

[0084] The cleaning module 504 is configured to clean the pixels in the initial defect area according to a cleaning threshold of each pixel to obtain cleaned defect data.

[0085] The defect data mentioned in the above-mentioned defect data cleaning device can be image data obtained by shooting or through image processing, image generation and other means. The image data includes objects with defects or without defects. For example, if the defect data cleaning is performed on a workpiece, the defect data to be cleaned can be an image including the workpiece. If the defect data cleaning is performed on a highway, the defect data to be cleaned can be an image including the highway. Depending on the specific usage scenario, the defect data to be cleaned can be an image including different objects. The objects in the image may include defects or may not include defects.

[0086] Taking the application of workpiece image data cleaning as an example, multiple images of the workpiece may be acquired, and the workpiece in the image may or may not contain defects. The cleaning process then involves cleaning out the data from the image containing the workpiece that has been incorrectly identified, such as identifying a non-defective area in the workpiece as a defective area, or identifying a background area as a defective area. For cleaning this data, the defect data cleaning solution of this embodiment can be used. After cleaning, the defect data in the workpiece image is accurately identified and can be used for other processing.

[0087] In this embodiment, if defect data to be cleaned is to be cleaned, multiple features of the defect data to be cleaned are obtained. These features include multi-scale context features, frequency features of the initial defect region in the defect data to be cleaned, and spatial features of the initial defect region in the defect data to be cleaned. The three features obtained are combined to obtain a combined feature. The combined feature and sensitivity parameter are used to determine the cleaning threshold for each pixel in the initial defect region of the defect data to be cleaned. In this way, the cleaning threshold of each pixel is used to determine whether the pixel is defective data. Pixels that do not meet the cleaning threshold are cleaned. This eliminates some incorrectly identified defect data from the initial defect data, resulting in accurate defect data.

[0088] The initial defective area in the defective data to be cleaned may be one or more defective areas in the defective data to be cleaned identified after pre-identification. The defective area may be a whole or multiple parts, and the multiple parts may be connected or disconnected.

[0089] The aforementioned scene features are the scene in which the defect data to be cleaned is generated. For example, if the defect data to be cleaned is a captured image of the workpiece surface, the scene data is the scene in which the workpiece was captured, or the scene in which the workpiece is captured in the captured image. The sensitivity parameter can be determined based on the scene features. The sensitivity parameter varies with different scenes and is used to indicate the sensitivity when determining the cleaning threshold. Factors such as scene lighting, the distance between the camera and the workpiece when capturing the image, and the background all affect the sensitivity parameter.

[0090] This embodiment can predict a cleaning threshold for each pixel point in the initial defect area based on the combined features and sensitivity parameters, and determine whether to clean the pixel point by determining the pixel value of the pixel point in the initial defect area or the difference between the cleaning prediction value of the pixel point and the cleaning threshold.

[0091] The solution provided by the embodiment of the present application obtains multi-scale context features of the defect data to be cleaned, frequency domain features of the initial defect area in the defect data to be cleaned, and spatial features of the initial defect area, wherein the multi-scale context features are context features at different receptive fields and different resolutions during the semantic segmentation model's identification of the defect data to be cleaned, the frequency domain features are used to reflect the defect texture and edge frequency characteristics of the initial defect area or its neighborhood, and the spatial features are used to separate the initial defect area from other areas of the defect data to be cleaned; obtains scene features of the defect data to be cleaned, wherein the scene features are used to indicate the scene when the defect data to be cleaned is generated; predicts a cleaning threshold for each pixel in the initial defect area based on the combined features and sensitivity parameters, wherein the combined features are features obtained by combining the multi-scale context features, the frequency domain features, and the spatial features, and the sensitivity parameters are parameters determined based on the scene features; and cleans the pixels in the initial defect area based on the cleaning threshold of each pixel to obtain cleaned defect data. Therefore, in the process of cleaning the defect data to be cleaned, the overall features of the defect data to be cleaned, the local defect texture and edge frequency characteristics of the initial defect area, and the boundary between the initial defect area and other areas can be comprehensively considered to clean the defect data to be cleaned, thereby improving the accuracy of cleaning. Moreover, during this process, the cleaning threshold of each pixel point in the cleaning process is accurately determined in combination with the scene, further improving the cleaning accuracy of the defective data to be cleaned.

[0092] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0093] This embodiment also provides a defect data cleaning system, which may include the defect data cleaning device described above, for cleaning defect data to be cleaned. Specific examples can be found in the above description, which will not be repeated here.

[0094] like Figure 6 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0095] Memory 113, for storing computer programs;

[0096] In one embodiment of the present application, the processor 111 is configured to implement the defect data cleaning method provided by any one of the aforementioned method embodiments when executing a program stored in the memory 113 .

[0097] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the defect data cleaning method provided in any of the aforementioned method embodiments is implemented.

[0098] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0099] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0100] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0101] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for cleaning defective data, characterized in that: include: Obtaining multi-scale context features of defect data to be cleaned, frequency domain features of an initial defect region in the defect data to be cleaned, and spatial features of the initial defect region, wherein the multi-scale context features are context features at different receptive fields and different resolutions during the process of a semantic segmentation model identifying the defect data to be cleaned, the frequency domain features are used to reflect defect texture and edge frequency characteristics of the initial defect region or its neighborhood, and the spatial features are used to segment the initial defect region from other regions of the defect data to be cleaned; Acquiring a scene feature of the defect data to be cleaned, wherein the scene feature is used to indicate a scene when the defect data to be cleaned is generated; Predicting a cleaning threshold for each pixel in the initial defect area based on the combined feature and the sensitivity parameter, wherein the combined feature is a feature obtained by combining the multi-scale context feature, the frequency domain feature, and the spatial feature, and the sensitivity parameter is a parameter determined based on the scene feature; The pixels in the initial defect area are cleaned according to the cleaning threshold of each pixel to obtain cleaned defect data.

2. The method according to claim 1, characterized in that The multi-scale context features for obtaining defect data to be cleaned include: Adjusting the resolution of the defect data to be cleaned to obtain defect data to be cleaned with multiple resolutions; The defect data to be cleaned at each resolution is input into a semantic segmentation model, and the semantic segmentation model uses different receptive fields to extract features from the defect data to be cleaned, thereby obtaining context features corresponding to different receptive fields at each resolution to obtain the multi-scale context features.

3. The method according to claim 1, characterized in that Obtaining the frequency domain features of the initial defect area in the defect data to be cleaned includes: The frequency domain features are obtained by extracting the features of the initial defect area or the neighborhood of the initial defect area through wavelet transformation.

4. The method according to claim 1, wherein Acquiring the spatial characteristics of the initial defect area includes: Using a region growing algorithm to segment edge pixels inside or at the edge of the initial defect area; Determine a connected domain surrounded by the edge pixels; The features of the connected domain are aggregated to obtain the spatial features.

5. The method according to claim 1, wherein The method further comprises: After obtaining the multi-scale context features of any defect data, the multi-scale context features of the any defect data are stored in a dynamic feature library, wherein the dynamic feature library stores multi-scale context features of different defect data; After obtaining the multi-scale context features of the defect data to be cleaned, the frequency domain features of the initial defect area in the defect data to be cleaned, and the spatial features of the initial defect area, the combined features are compared with the features in the dynamic feature library; The validity or invalidity of the combined feature is determined based on the comparison result.

6. The method according to claim 1, characterized in that Determining the sensitivity parameter includes: Determining the lighting conditions of the production scene and the material type of the defect data to be cleaned according to the scene characteristics; Determining a first sensitivity range according to the material type; The sensitivity parameter is determined from the first sensitivity range according to the lighting condition.

7. The method according to claim 1, characterized in that The step of predicting the cleaning threshold of each pixel in the initial defect area based on the combined features and the sensitivity parameter includes: The combined feature and the sensitivity parameter are input into a multi-layer perceptron, and the multi-layer perceptron generates a cleaning threshold for each pixel point that is consistent with the sensitivity parameter.

8. A defect data cleaning device, characterized in that: include: A first acquisition module is configured to acquire multi-scale context features of defect data to be cleaned, frequency domain features of an initial defect region in the defect data to be cleaned, and spatial features of the initial defect region, wherein the multi-scale context features are context features at different receptive fields and resolutions during identification of the defect data to be cleaned by a semantic segmentation model; the frequency domain features are used to reflect defect texture and edge frequency characteristics of the initial defect region or its neighborhood; and the spatial features are used to segment the initial defect region from other regions of the defect data to be cleaned; A second acquisition module is used to acquire a scene feature of the defect data to be cleaned, wherein the scene feature is used to indicate a scene when the defect data to be cleaned is generated; a prediction module, configured to predict a cleaning threshold for each pixel in the initial defect area based on a combined feature and a sensitivity parameter, wherein the combined feature is a feature obtained by combining the multi-scale context feature, the frequency domain feature, and the spatial feature, and the sensitivity parameter is a parameter determined based on the scene feature; The cleaning module is used to clean the pixels in the initial defect area according to the cleaning threshold of each pixel to obtain cleaned defect data.

9. An electronic device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; At least one memory connected to the at least one bus, wherein the memory stores a computer program, and when the processor executes the computer program, the defect data cleaning method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the defect data cleaning method according to any one of claims 1 to 7 of the present application.

Images