Data cleaning method and device based on interactive segmentation model, equipment and medium

Abstract

This invention discloses a data cleaning method, apparatus, device, and medium based on an interactive segmentation model. The method includes: acquiring an image annotation dataset comprising multiple original defect images and corresponding labeled mask images; performing single-target segmentation based on the labeled mask images to determine the target mask region and the corresponding original target region; using an interactive segmentation model to predict the defect location of the original target region based on initial prediction results and initial cue points to obtain a target prediction result; determining target cue points based on the difference between the target prediction result and the target mask region; cyclically executing the defect location prediction process based on the target prediction result and the target cue points; and ending the defect location prediction process when the loop termination condition is met, and determining the data filtering result of the image annotation dataset. This solution utilizes an intelligent interactive segmentation model to construct a fully automated data cleaning process, employing different intersection-union (IU) thresholds to measure different categories, enabling rapid and accurate screening of abnormal annotations.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a data cleaning method, apparatus, device, and medium based on an interactive segmentation model. Background Technology

[0002] In the training process of industrial vision inspection models, high-quality labeled data is crucial to ensuring model performance. Currently, the main technical solutions for cleaning and screening labeled data quality are as follows: 1. Manual review: Data cleaners examine each labeled image individually, relying on their eyes to judge whether the labeled mask accurately covers the target; 2. Automatic filtering based on a fixed intersection-union ratio (IU) threshold: A baseline model is used to predict the data, and the IU ratio between the predicted result and the manually labeled data is calculated. Samples with an IU ratio lower than the fixed IU threshold (e.g., 0.5) are discarded as dirty data; 3. Rule-based morphological screening: Image processing techniques (such as OpenCV) are used to calculate the geometric features of the labeled region, such as area, aspect ratio, and edge smoothness, and a series of hard rules (e.g., too small an area, too discrete shape) are set to screen for abnormal labels.

[0003] The aforementioned existing technical solutions have significant drawbacks when dealing with large-scale (e.g., 100,000+) industrial defect data. Solution 1 is inefficient and costly: manual review requires substantial human resources and time, and prolonged work can lead to fatigue, resulting in missed and false positives, and inconsistent cleaning standards are difficult to maintain. Solution 2 has rigid threshold settings and poor adaptability: the visual characteristics of different categories of industrial defects (e.g., fine scratches versus large patches) vary greatly. For defects with complex edges, even with correct labeling, the intersection-union ratio (IU) may be low; for simple shapes, even with labeling errors, the IU may be high. Using a uniform IU threshold for all categories leads to numerous false negatives or missed screenings. Solution 3 lacks semantic understanding: geometric rule-based filtering often only focuses on the shape of the mask, without understanding the semantic content of the image itself. If the labeled mask shape is normal but the position is completely wrong, rule-based methods cannot identify such errors, leading to inaccurate data filtering. Summary of the Invention

[0004] This invention provides a data cleaning method, apparatus, device, and medium based on an interactive segmentation model. It constructs a fully automated data cleaning process by using an intelligent interactive segmentation model and sets different cross-union ratio thresholds for different defect categories, which can quickly and accurately screen abnormal labeled data.

[0005] According to one aspect of the present invention, a data cleaning method based on an interactive segmentation model is provided, the method comprising: An image annotation dataset is obtained, comprising multiple original defect images and corresponding annotated mask images. Single-target segmentation is performed based on the annotated mask images to determine the target mask region and the corresponding original target region. The annotated mask images are used to describe the defect location and defect category. The target prediction result is obtained by predicting the defect location of the original target area using an interactive segmentation model based on the initial prediction result and the initial prompt point. Based on the difference between the target prediction result and the target mask area, a target cue point is determined, and based on the target prediction result and the target cue point, the process of predicting the defect location of the original target area through an interactive segmentation model is executed cyclically. When the loop stopping condition is met, the defect location prediction process ends, and the data filtering result of the image annotation dataset is determined; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio (IU) of the target prediction result and the target mask region reaches the IU threshold of the defect category to which the target mask region belongs.

[0006] According to another aspect of the present invention, a data cleaning apparatus based on an interactive segmentation model is provided, the apparatus comprising: The single-target segmentation module is used to acquire an image annotation dataset including multiple original defect images and corresponding labeled mask images, and to perform single-target segmentation based on the labeled mask images to determine the target mask region and the corresponding original target region; wherein, the labeled mask images are used to describe the defect location and defect category; The initial prediction module is used to predict the defect location of the original target region based on the initial prediction result and the initial prompt point through an interactive segmentation model to obtain the target prediction result. The cyclic prediction module is used to determine the target cue point based on the difference between the target prediction result and the target mask area, and to cyclically execute the process of predicting the defect location of the original target area through an interactive segmentation model based on the target prediction result and the target cue point. The data filtering module is used to end the defect location prediction process and determine the data filtering result of the image annotation dataset when the loop stopping condition is met; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio (IU) of the target prediction result and the target mask region reaches the IU threshold of the defect category to which the target mask region belongs.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data cleaning method based on the interactive segmentation model according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data cleaning method based on an interactive segmentation model as described in any embodiment of the present invention.

[0009] The technical solution of this invention involves acquiring an image annotation dataset including multiple original defect images and corresponding annotated mask images. Single-target segmentation is performed based on the annotated mask images to determine the target mask region and the corresponding original target region. The annotated mask images describe the defect location and defect category. An interactive segmentation model predicts the defect location of the original target region based on initial prediction results and initial cue points to obtain a target prediction result. Target cue points are determined based on the difference between the target prediction result and the target mask region. The process of predicting the defect location of the original target region using the interactive segmentation model is repeated based on the target prediction result and target cue points. When a loop stopping condition is met, the defect location prediction process ends, and the data filtering result of the image annotation dataset is determined. The loop stopping condition is reaching the maximum number of loops or the intersection-union (IU) ratio between the target prediction result and the target mask region reaching the IU threshold for the defect category to which the target mask region belongs. This technical solution leverages the precise segmentation capabilities of an intelligent interactive segmentation model to construct a fully automated data cleaning process. It can obtain semantically relevant prediction results unaffected by coarse annotation results and sets different IU thresholds for different defect categories, thereby enabling rapid and accurate screening of abnormal annotated data and improving the efficiency and accuracy of data cleaning.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a data cleaning method based on an interactive segmentation model provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a process for obtaining a target prompt point according to an embodiment of the present invention; Figure 3 This is a flowchart of another data cleaning method based on an interactive segmentation model provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of the overall process of a data cleaning method based on an interactive segmentation model according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a data cleaning device based on an interactive segmentation model according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device that implements a data cleaning method based on an interactive segmentation model according to an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Example 1 Figure 1This is a flowchart of a data cleaning method based on an interactive segmentation model provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where image annotation data is cleaned quickly and accurately using an interactive segmentation model. The method can be executed by a data cleaning device based on an interactive segmentation model, which can be implemented in hardware and / or software and can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes: S110: Obtain an image annotation dataset including multiple original defect images and corresponding annotation mask images; perform single-target splitting based on the annotation mask images to determine the target mask region and the corresponding original target region; wherein, the annotation mask images are used to describe the defect location and defect category.

[0016] The original defect image refers to an original image containing at least one defect. For example, the original defect image can be an industrial defect image, such as those containing minor scratches or large patches. The labeled mask image refers to a mask image obtained by manually or using a basic segmentation model (such as U-Net) to annotate the defects in the original defect image. The image annotation dataset stores the image annotation data, specifically including multiple original defect images and their corresponding labeled mask images. It should be noted that the labeled mask images may contain erroneous or coarse annotations. This invention aims to automatically filter out abnormal annotation data from the labeled mask images, thereby achieving efficient and accurate cleaning of the annotation data. Furthermore, this invention employs a blob (connected region) level analysis strategy. Therefore, when multiple defect targets appear in the original defect image, all defect targets need to be segmented based on the labeled mask image; conversely, when only one defect target appears in the original defect image, to reduce the impact of an excessively large background area, it is also necessary to locate the individual defect target based on the labeled mask image. The target mask region can refer to the local labeled mask image region corresponding to a single defect target, and the original target region can refer to the local image region in the original defect image corresponding to the position of the target mask region. It can be understood that the original defect image and its corresponding labeled mask image have the same size, the target mask region and its corresponding original target region also have the same size, and the size of the target mask region and the original target region is smaller than the size of the original defect image and the labeled mask image.

[0017] In this embodiment, an image annotation dataset is first obtained, comprising multiple original defect images and corresponding labeled mask images. This image annotation dataset is the data set that needs to be cleaned. Then, single-target segmentation is performed based on the labeled mask images to determine one or more target mask regions and corresponding original target regions. Optionally, determining the target mask region and corresponding original target region based on single-target segmentation of the labeled mask images includes: determining the bounding rectangle of each connected region in the labeled mask image as a first border; expanding the area enclosed by the first border by a first preset multiple to obtain a second border; determining the image region enclosed by the second border in the labeled mask image as the target mask region; and determining the original target region from the original defect images based on the position information of the target mask region in the labeled mask image.

[0018] Specifically, all connected regions in the labeled mask image can be segmented, and a bounding rectangle (i.e., the minimum bounding rectangle) can be determined for each connected region as the first bounding box. Considering the special case where the image region enclosed by the first bounding box only contains foreground (i.e., the defective target) and not background (i.e., non-defective targets), and to reduce the impact of an excessively large background region, the enclosing area of ​​the first bounding box can be expanded by a first preset multiple to obtain the second bounding box. For example, the first preset multiple can be set to a value between 0 and 1 (e.g., 0.4). The image region enclosed by the second bounding box in the labeled mask image is used as the target mask region, and the corresponding position region is cropped from the original defect image based on the position information of the target mask region in the labeled mask image as the target original region.

[0019] S120, the target prediction result is obtained by predicting the defect location of the original target area based on the initial prediction result and the initial prompt point through the interactive segmentation model.

[0020] The interactive segmentation model can segment objects based on cue points. These cue points can include positive points (indicating foreground) and negative points (indicating background). The object prediction result refers to the object segmentation result output by the interactive segmentation model.

[0021] In this embodiment, before predicting the defect location of the original target region using the interactive segmentation model, an initialization operation is first required. This includes initializing the segmentation results and prompt points, thereby obtaining the initial prediction results and initial prompt points. For example, a new single-channel all-black image (corresponding to the background) with the same size as the target mask region can be created as the initial prediction result. Simultaneously, based on an automatic point sampling strategy, one positive point is sampled from the center point of the labeled mask in the target mask region as the initial prompt point. After the initialization operation is completed, the initial prediction results, initial prompt points, and the original target region are input into the interactive segmentation model. The interactive segmentation model then performs the first defect location prediction on the original target region to obtain the target prediction result.

[0022] S130: Determine the target cue point based on the difference between the target prediction result and the target mask area, and iteratively execute the process of predicting the defect location of the original target area through the interactive segmentation model based on the target prediction result and the target cue point.

[0023] In this embodiment, after obtaining the target prediction result output by the interactive segmentation model, error regions (including missed detection mask regions and false detection mask regions) can be identified based on the difference between the target prediction result and the target mask region, and target cue points can be automatically determined based on the error regions. Target cue points can refer to cue points that the interactive segmentation model needs to use in subsequent predictions. Missed detection (FN) mask regions refer to regions where the actual value is positive and the prediction is negative (corresponding to undetected targets); false detection (FP) mask regions refer to regions where the actual value is negative and the prediction is positive (corresponding to falsely detected background). Optionally, determining target cue points based on the difference between the target prediction result and the target mask region includes: determining missed detection mask regions and false detection mask regions based on the difference between the target prediction result and the target mask region; performing 8-connectivity analysis on the missed detection mask regions and false detection mask regions respectively to determine the area and location information of all connected regions in the missed detection mask regions and false detection mask regions; determining the largest connected region in the missed detection mask regions and false detection mask regions as candidate connected regions; and determining target cue points based on the area and location information of the candidate connected regions.

[0024] In this embodiment, an interactive click generation algorithm based on maximum area decision-making is designed for target prompt point determination. This algorithm aims to automatically identify and repair difficult-to-distinguish regions in the target prediction results by simulating human interaction logic. Specifically, as shown... Figure 2 As shown, the target prediction result (i.e., the prediction mask) is first compared pixel-by-pixel with the target mask region (i.e., the ground truth mask). Based on the difference between the target prediction result and the target mask region, the set of pixels that are actually positive and predicted as negative is identified as the missed detection mask region (FN Mask), and the set of pixels that are actually negative and predicted as positive is identified as the false detection mask region (FPMask). Furthermore, to ensure that the connectivity analysis at the boundaries is not limited by image edges, a constant padding of 1 pixel width is applied to both the FN Mask and FPMask. Then, 8-connectivity analysis is performed on the constant-padding missed detection mask region and the false detection mask region, extracting the area and position information of all connected regions in both regions. The connected region with the largest area in both regions is then identified as the candidate connected region, and the target cue point is determined based on the area and position information of the candidate connected regions.

[0025] This solution, through this setting, only retains the largest connected regions in the missed detection mask area and the false detection mask area, ignoring minor noise interference. This ensures that each click applies to the most serious error area, thereby improving the repair efficiency of a single interaction.

[0026] In this embodiment, optionally, determining the target prompt point based on the area and location information of the candidate connected regions includes: determining a hybrid scoring map of the candidate connected regions based on the location information of the candidate connected regions; wherein the hybrid scoring map is constructed based on distance transformation, mass balance, and cross constraints; determining the prompt point type based on the area information of the candidate connected regions and the hybrid scoring map; wherein the prompt point type is a positive point or a negative point, with positive points used to characterize the repair of missed detections and negative points used to characterize the repair of false detections; determining the prompt point location based on the hybrid scoring map of the candidate connected regions corresponding to the prompt point type; and determining the target prompt point based on the prompt point type and the prompt point location.

[0027] In this embodiment, a multi-constraint fusion click coordinate generation method is adopted. This method uses a fusion algorithm combining distance transformation, quality balance, and cross constraints to find the optimal click location, and then uses an area-first decision strategy to determine the click type. Specifically, firstly, based on the location information of candidate connected regions, a hybrid scoring map of the candidate connected regions is constructed using distance transformation, quality balance, and cross constraints. This hybrid scoring map describes the comprehensive score corresponding to the selectable prompt point location under distance transformation, quality balance, and cross constraints. In other words, the hybrid scoring map includes multiple scores, and the higher the score, the better the quality of the corresponding prompt point location (the higher the score, the better the quality of the corresponding prompt point location). Optionally, a hybrid scoring map of the candidate connected regions is determined based on the location information of the candidate connected regions, including: determining a distance transformation scoring map based on the shortest straight-line distance from each pixel in the candidate connected region to the boundary of the candidate connected region; determining a cumulative distribution function based on the row and column pixel distribution of the candidate connected region, and determining a quality balance scoring map based on the cumulative distribution function; constructing a cross-shaped candidate band based on the size information of the candidate connected region and using the center of the bounding rectangle of the candidate connected region as a reference, and determining a cross-shaped constraint scoring map based on the cross-shaped candidate band; and fusing the distance transformation scoring map, the quality balance scoring map, and the cross-shaped constraint scoring map to obtain a hybrid scoring map of the candidate connected regions.

[0028] When determining the mixed score map of candidate connected regions, the shortest straight-line distance from each pixel in the candidate connected region to the boundary of the candidate connected region is first calculated based on L2 distance and normalized. The normalized shortest straight-line distance corresponding to each pixel is used as the distance transformation score to construct the distance transformation score map, making the scores converge towards the target center. Then, the cumulative distribution function is determined according to the row and column pixel distribution of the candidate connected region. The quality balance score is determined by comparing the difference between the cumulative distribution value (the cumulative number of pixels in a certain row / column) and half of the total number of pixels, thus constructing a quality balance score map to find the point where the geometric center and the quality center are balanced. Next, using the center of the bounding rectangle of the candidate connected region as a reference, horizontal and vertical cross-shaped candidate bands are constructed according to the size information of the candidate connected region. This constraint can force the prompt point to fall in the "main trunk" area of ​​the target and avoid clicking on irregular edge branches. For example, the horizontal half-width and vertical half-width of the cross-shaped candidate band are set to one-tenth of the width and height of the candidate connected region, respectively, and the minimum half-width of the horizontal half-width and vertical half-width is set to 2 pixels, thereby preventing the cross-shaped candidate band from disappearing when the target is too small. Pixels falling within the cross-shaped candidate band are then scored as 1 (to retain distance transform and quality balance scores), while pixels falling outside the cross-shaped candidate band are scored as 0 (to exclude them from selection), thus constructing a cross-constraint score map. Finally, the distance transform score map, quality balance score map, and cross-constraint score map can be fused at the pixel level. For example, the scores at corresponding positions in the three score maps can be directly multiplied to obtain a mixed score map of candidate connected regions, thereby fusing features of different dimensions (distance from the center, quality balance, and morphological constraints) into a final score map.

[0029] Furthermore, such as Figure 2As shown, after determining the mixed score map of candidate connected regions, post-processing operations are required, including removing constant padding and excluding clicked areas. Excluding clicked areas prevents the same location from being selected repeatedly, thus ensuring the uniqueness and validity of the prompt points. After post-processing, the prompt point type can be determined based on the area information of the candidate connected regions and the mixed score map, clarifying whether the current task is to correct missed detections or false detections. Optionally, the prompt point type is determined based on the area information of the candidate connected regions and the mixed scoring map, including: taking the largest connected regions in the missed detection mask region and the false detection mask region as the first region and the second region, respectively; taking the largest scores in the mixed scoring maps corresponding to the first region and the second region as the first mixed score and the second mixed score, respectively; if the area of ​​the first region is greater than or equal to a second preset multiple of the area of ​​the second region, the prompt point type is determined to be a positive point; wherein, the second preset multiple is less than 1; if the area of ​​the first region is less than a second preset multiple of the area of ​​the second region, it is determined whether the product of the first mixed score and the preset deviation coefficient is less than the second mixed score; wherein, the preset deviation coefficient is greater than 1; if so, the prompt point type is determined to be a negative point; otherwise, the prompt point type is determined to be a positive point.

[0030] Specifically, such as Figure 2As shown, the largest connected region in the missed detection mask region and the false detection mask region is designated as the first region and the second region, respectively. The difference between the area of ​​the first region (FN Max Area) and the area of ​​the second region (FP Max Area) is then determined. If the area of ​​the first region is greater than or equal to a second preset multiple (e.g., 0.5) of the area of ​​the second region, i.e., FN Max Area ≥ (FP Max Area × 0.5), then the difference between the areas of the first and second regions is significant, and the prompt point type can be directly determined as a positive point. If the area of ​​the first region is less than a second preset multiple of the area of ​​the second region, i.e., FN Max Area < (FP Max Area × 0.5), then the difference between the areas of the first and second regions is not significant. To effectively prevent frequent jitter caused by minor fluctuations in the false detection mask region, a preset deviation coefficient p_bias (e.g., 1.2) can be introduced. Further judgment is made by comparing the maximum score in the mixed scoring map corresponding to the first and second regions. The maximum scores in the mixed score maps corresponding to the first and second regions are respectively used as the first mixed score (FN Max Score) and the second mixed score (FP Max Score). It is then determined whether the product of the first mixed score and a preset deviation coefficient is less than the second mixed score. If so, i.e., (FN Max Score × p_bias) ≥ FP Max Score, the prompt point type is determined to be negative; otherwise, i.e., (FN Max Score × p_bias) < FP Max Score, the prompt point type is determined to be positive.

[0031] In this embodiment, after determining the mixed score map of the candidate connected regions and the cue point type, it is also necessary to determine the cue point position based on the mixed score map of the candidate connected regions corresponding to the cue point type. Optionally, determining the cue point position based on the mixed score map of the candidate connected regions corresponding to the cue point type includes: if the cue point type is a positive point, then the pixel position corresponding to the first mixed score is determined as the cue point position; if the cue point type is a negative point, then the pixel position corresponding to the second mixed score is determined as the cue point position.

[0032] It should be noted that the candidate connected regions corresponding to positive and negative points are the first region and the second region, respectively. In this embodiment, the mixed score map of the candidate connected regions corresponding to the prompt point type can be used as the target score map, and the maximum score is found in the target score map. The coordinates of the pixel point corresponding to the maximum score are used as the prompt point location. Specifically, if the prompt point type is positive, the maximum score is found in the mixed score map (FNScore Map) of the first region as the first mixed score, and the coordinates of the pixel point corresponding to the first mixed score are determined as the prompt point location; if the prompt point type is negative, the maximum score is found in the mixed score map (FP Score Map) of the second region as the second mixed score, and the coordinates of the pixel point corresponding to the second mixed score are determined as the prompt point location.

[0033] After determining the cue point type and location, the target cue point can be determined based on these criteria. Then, the target prediction result, the target cue point, and the original target region are input into the interactive segmentation model. The interactive segmentation model predicts the defect location in the original target region to obtain a new target prediction result. A new target cue point is then determined based on the difference between the new target prediction result and the target mask region. The previous target prediction result and target cue point are updated using the new target prediction result and the new target cue point. Based on the updated target prediction result and target cue point, the process of predicting the defect location in the original target region using the interactive segmentation model is repeated.

[0034] S140, when the loop stopping condition is met, the defect location prediction process ends and the data filtering results of the image annotation dataset are determined; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio (IU) between the target prediction result and the target mask region reaches the IU threshold of the defect category to which the target mask region belongs.

[0035] In this embodiment, after each defect location prediction, it is necessary to determine whether the loop stopping condition is met. The loop stopping condition is either that the current loop count reaches the maximum loop count (which can be flexibly set according to actual needs) or that the intersection-union ratio (IU) of the predicted target and the target mask region in the current loop reaches the IU threshold for the defect category to which the target mask region belongs. The IU threshold can be used as a criterion for evaluating the quality of the labeled data. It should be noted that due to the common inter-class differences in industrial defect targets, using a uniform IU threshold to measure different defect categories is unreasonable. Therefore, this invention sets different IU thresholds for each defect category, i.e., one IU threshold corresponds to one defect category, to achieve differentiated and accurate determination of different defect categories. If the loop stopping condition is not met, the defect location prediction process continues; if the loop stopping condition is met, the defect location prediction process ends, and the data filtering results of the image annotation dataset are determined.

[0036] In this embodiment, optionally, determining the data filtering result of the image annotation dataset includes: if the maximum number of iterations is not reached at the end of the defect location prediction process, the target mask area is determined to be qualified annotation data; if the intersection-union ratio (IU) between the target prediction result and the target mask area does not reach the IU threshold under the defect category to which the target mask area belongs at the end of the defect location prediction process, the target mask area is determined to be unqualified annotation data.

[0037] In this embodiment, the data filtering results of the image annotation dataset can be determined based on the number of iterations and the cross-union ratio (CUP) at the end of the defect location prediction process. Specifically, if the maximum number of iterations is not reached at the end of the defect location prediction process, it indicates that the CUP between the target prediction result and the target mask region before reaching the maximum number of iterations reaches (i.e., greater than or equal to) the CUP threshold for the defect category to which the target mask region belongs. In this case, the target mask region can be identified as qualified annotation data (i.e., high-quality annotation data, which can be regarded as normal annotation data). If the CUP between the target prediction result and the target mask region does not reach the CUP threshold for the defect category to which the target mask region belongs at the end of the defect location prediction process, it indicates that the CUP between the target prediction result and the target mask region is still less than the CUP threshold for the defect category to which the target mask region belongs when the maximum number of iterations is reached. That is, the annotation mask of the defect target is excessively expanded or shrunken. In this case, the target mask region can be identified as unqualified annotation data (i.e., low-quality annotation data, which can be regarded as abnormal annotation data).

[0038] Furthermore, after determining the data filtering results of the image annotation dataset, in order to obtain more usable annotation data from the unqualified annotation data, the intersection-union ratios (IURs) of the unqualified annotation data at the end of the defect location prediction process can be sorted in descending order, and the unqualified annotation data with the highest IUR ranking can be manually filtered and repaired.

[0039] The technical solution of this invention involves acquiring an image annotation dataset including multiple original defect images and corresponding annotated mask images. Single-target segmentation is performed based on the annotated mask images to determine the target mask region and the corresponding original target region. The annotated mask images describe the defect location and defect category. An interactive segmentation model predicts the defect location of the original target region based on initial prediction results and initial cue points to obtain a target prediction result. Target cue points are determined based on the difference between the target prediction result and the target mask region. The process of predicting the defect location of the original target region using the interactive segmentation model is repeated based on the target prediction result and target cue points. When a loop stopping condition is met, the defect location prediction process ends, and the data filtering result of the image annotation dataset is determined. The loop stopping condition is reaching the maximum number of loops or the intersection-union (IU) ratio between the target prediction result and the target mask region reaching the IU threshold for the defect category to which the target mask region belongs. This technical solution leverages the precise segmentation capabilities of an intelligent interactive segmentation model to construct a fully automated data cleaning process. It can obtain semantically relevant prediction results unaffected by coarse annotation results and sets different IU thresholds for different defect categories, thereby enabling rapid and accurate screening of abnormal annotated data and improving the efficiency and accuracy of data cleaning.

[0040] Example 2 Figure 3 This is a flowchart of a data cleaning method based on an interactive segmentation model provided in Embodiment 2 of the present invention. This embodiment is an optimization based on the above embodiment. Specifically, the optimization includes: after obtaining an image annotation dataset including multiple original defect images and corresponding labeled mask images, the method further includes: performing a dilation operation on each labeled mask image by a preset number of pixels to obtain a dilated mask image; determining the intersection-union ratio (CUI) of each labeled mask image and the corresponding dilated mask image under each defect category; and determining the average CUI under each defect category as the CUI threshold for the corresponding defect category.

[0041] like Figure 3 As shown, the method in this embodiment specifically includes the following steps: S210, obtain an image annotation dataset including multiple original defect images and corresponding annotation mask images, and perform single-target splitting based on the annotation mask images to determine the target mask region and the corresponding original target region; wherein, the annotation mask images are used to describe the defect location and defect category.

[0042] S220, perform a dilation operation on each labeled mask image by a preset number of outward expansion pixels to obtain a dilated mask image.

[0043] In this embodiment, the intersection-union (IU) threshold for defect categories can be determined based on each labeled mask image in the image annotation dataset. Specifically, firstly, a preset number of pixels for expansion can be set according to the annotation accuracy requirements. For example, since it is generally considered that a difference of less than 3 pixels between the labeled mask and the target boundary is a fine annotation, the preset number of pixels for expansion can be set to 2 based on the most extreme pixel difference, thereby meeting the conventional annotation accuracy requirements. Then, a dilation operator is used to dilate the labeled masks in each labeled mask image by the preset number of pixels for expansion, and the result after the dilation operation is used as the dilated mask image.

[0044] S230, determine the cross-union ratio of each labeled mask image and the corresponding dilated mask image under each defect category.

[0045] After obtaining the dilated mask images, for each labeled mask image, the intersection-over-union (IoU) ratio of the labeled mask image and its corresponding dilated mask image is calculated for each defect category. Each defect category in a labeled mask image corresponds to one IoU ratio. Specifically, for a given defect category in a labeled mask image, all labeled mask regions and dilated mask regions corresponding to that defect category are selected from both the labeled mask image and its corresponding dilated mask image. The IoU ratio of the selected labeled mask regions (including one or more target mask regions) and dilated mask regions is then calculated to obtain the IoU ratio of the labeled mask image for that defect category. Repeating this process yields the IoU ratio for each labeled mask image under each defect category.

[0046] S240, the mean cross-union ratio for each defect category is determined as the cross-union ratio threshold for the corresponding defect category.

[0047] After obtaining the cross-union ratio (CUP) of each labeled mask image under each defect category, the average CUP of each defect category can be calculated to obtain the mean CUP of each defect category, and the mean CUP of each defect category can be determined as the CUP threshold of the corresponding defect category.

[0048] S250 uses an interactive segmentation model to predict the location of defects in the original target area based on the initial prediction results and initial prompts, thus obtaining the target prediction result.

[0049] S260, determine the target cue point based on the difference between the target prediction result and the target mask area, and cyclically execute the process of predicting the defect location of the original target area through the interactive segmentation model based on the target prediction result and the target cue point.

[0050] S270, when the loop stopping condition is met, the defect location prediction process ends and the data filtering results of the image annotation dataset are determined; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio (IU) between the target prediction result and the target mask region reaches the IU threshold of the defect category to which the target mask region belongs.

[0051] Figure 4 This paper presents a schematic diagram of the overall workflow of a data cleaning method based on an interactive segmentation model. The data to be filtered refers to a set of image annotation data from an image annotation dataset. The upper and lower images in the data to be filtered represent the original defect image and its corresponding annotation mask image, respectively. The truncation threshold (IoU_T) is the intersection-union (IoU) threshold. Specifically, as shown... Figure 4As shown, firstly, an image annotation dataset including multiple original defect images and corresponding labeled mask images is acquired. On one hand, single-target segmentation is performed based on the labeled mask images to obtain target mask regions, and the corresponding ROIs are obtained from the original defect images as the original target regions based on the single-target segmentation results. On the other hand, the labeled masks in each labeled mask image are expanded by 2 pixels to obtain expanded mask images, and the truncation threshold (IoU_T) for each defect category is determined based on the average intersection-union ratio (IoU) of the labeled mask images and the expanded mask images for each defect category. Then, initial point sampling is performed on each target mask region. One point is sampled from the center point of the labeled mask of the target mask region as an initial cue point, thus ensuring that the cue point is within the labeled mask. At the same time, a completely black image with the same size as the target mask region is initialized as the initial segmentation result. Next, the initial cue point, the initial segmentation result, and the target original regions corresponding to the target mask regions are input into the intelligent interactive segmentation model for the first inference to obtain the target segmentation result, and the intersection-union ratio (IoU) between the target segmentation result and the target mask region is calculated. The IoU is then compared with the IoU_T of the corresponding defect category. If IoU ≥ IoU_T, the model inference process terminates, and the target mask region is classified as high-quality data. Otherwise, positive or negative points are sampled (selected from redundant prediction results to avoid duplicate prompts) as target prompts, and the model inference process is iterated based on the target prompts and the target prediction results until the loop termination condition is met. The loop termination condition is reaching the maximum number of iterations or the intersection-union ratio (IoU) between the target prediction result and the target mask region reaches the IoU threshold of the defect category to which the target mask region belongs. If IoU < IoU_T after reaching the maximum number of iterations N, the target mask region is classified as low-quality data, and the top 20% of low-quality data with the largest IoU in each category are manually checked.

[0052] The technical solution of this invention, after acquiring an image annotation dataset including multiple original defect images and corresponding annotated mask images, performs a dilation operation on each annotated mask image by a preset number of pixels to obtain an dilated mask image; determines the intersection-over-union (IoU) ratio of each annotated mask image and its corresponding dilated mask image under each defect category; and determines the average IoU ratio under each defect category as the IoU threshold for that defect category. This technical solution can automatically obtain an adaptive IoU threshold for each defect category through the dilation operation, achieving differentiated and accurate judgment of defects of different shapes and sizes, thereby overcoming the shortcomings of traditional fixed IoU thresholds that cannot adapt to multiple categories of industrial defects.

[0053] Example 3 Figure 5This is a schematic diagram of a data cleaning device based on an interactive segmentation model provided in Embodiment 3 of the present invention. This device can execute the data cleaning method based on an interactive segmentation model provided in any embodiment of the present invention, and possesses the corresponding functional modules and beneficial effects of the method. For example... Figure 5 As shown, the device includes: The single-target segmentation module 310 is used to acquire an image annotation dataset including multiple original defect images and corresponding annotation mask images, and to perform single-target segmentation based on the annotation mask images to determine the target mask region and the corresponding original target region; wherein, the annotation mask images are used to describe the defect location and defect category; The initial prediction module 320 is used to predict the defect location of the original target region based on the initial prediction result and the initial prompt point through an interactive segmentation model to obtain the target prediction result. The cyclic prediction module 330 is used to determine the target cue point based on the difference between the target prediction result and the target mask area, and to cyclically execute the process of predicting the defect location of the original target area through an interactive segmentation model based on the target prediction result and the target cue point. The data filtering module 340 is used to end the defect location prediction process and determine the data filtering result of the image annotation dataset when the loop stopping condition is met; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio of the target prediction result and the target mask region reaches the intersection-union ratio threshold of the defect category to which the target mask region belongs.

[0054] Optionally, the single-target splitting module 310 is specifically used for: The bounding rectangles of each connected region in the labeled mask image are determined as the first border, and the area enclosed by the first border is expanded by a first preset multiple to obtain the second border. The image region enclosed by the second border in the labeled mask image is determined as the target mask region; The original target region is determined from the original defect image based on the position information of the target mask region in the labeled mask image.

[0055] Optionally, the cyclic prediction module 330 is used for: The missed detection mask region and the false detection mask region are determined based on the difference between the target prediction result and the target mask region; Perform 8-connectivity analysis on the missed detection mask region and the false detection mask region respectively to determine the area and location information of all connected regions in the missed detection mask region and the false detection mask region; The largest connected region in both the missed detection mask region and the false detection mask region is determined as the candidate connected region. The target prompt point is determined based on the area and location information of the candidate connected regions.

[0056] Optionally, the cyclic prediction module 330 is further configured to: A hybrid scoring map of the candidate connected regions is determined based on the location information of the candidate connected regions; wherein, the hybrid scoring map is constructed based on distance transformation, quality balance and cross constraint; The type of prompt point is determined based on the area information of the candidate connected regions and the mixed scoring map; wherein, the type of prompt point is a positive point or a negative point, a positive point is used to indicate the repair of missed detections, and a negative point is used to indicate the repair of false detections; The location of the prompt point is determined based on the mixed scoring map of the candidate connected regions corresponding to the prompt point type; The target prompt point is determined based on the prompt point type and the prompt point location.

[0057] Optionally, the cyclic prediction module 330 is further configured to: The distance transformation scoring map is determined based on the shortest straight-line distance from each pixel in the candidate connected region to the boundary of the candidate connected region. The cumulative distribution function is determined based on the row and column pixel distribution of the candidate connected regions, and the quality balance score map is determined based on the cumulative distribution function. Based on the size information of the candidate connected regions, a cross-shaped candidate band is constructed with the center of the outer rectangle of the candidate connected regions as the reference, and a cross-shaped constraint scoring map is determined based on the cross-shaped candidate band; The distance transformation score map, the mass balance score map, and the cross constraint score map are fused to obtain the hybrid score map of the candidate connected region.

[0058] Optionally, the cyclic prediction module 330 is further configured to: The largest connected region in the missed detection mask region and the largest connected region in the false detection mask region are respectively designated as the first region and the second region. The largest scores in the mixed scoring maps corresponding to the first region and the second region are respectively designated as the first mixed score and the second mixed score. If the area of ​​the first region is greater than or equal to a second preset multiple of the area of ​​the second region, then the prompt point type is determined to be a positive point; wherein, the second preset multiple is less than 1; If the area of ​​the first region is less than a second preset multiple of the area of ​​the second region, then determine whether the product of the first mixed score and the preset deviation coefficient is less than the second mixed score; wherein the preset deviation coefficient is greater than 1; If so, then the type of the prompt point will be determined as a negative point; Otherwise, the prompt point type will be set to "positive point".

[0059] Optionally, the cyclic prediction module 330 is further configured to: If the prompt point type is positive, then the pixel position corresponding to the first mixed score is determined as the prompt point position; If the prompt point type is negative, then the pixel position corresponding to the second mixed score is determined as the prompt point position.

[0060] Optionally, the data filtering module 340 is specifically used for: If the maximum number of iterations is not reached by the end of the defect location prediction process, the target mask area is determined to be qualified annotation data. If the cross-union ratio (CUB) between the target prediction result and the target mask region does not reach the CUB threshold for the defect category to which the target mask region belongs at the end of the defect location prediction process, then the target mask region is determined to be unqualified labeled data.

[0061] Optionally, the apparatus further includes: an intersection-to-union ratio threshold determination module, used for: After obtaining an image annotation dataset including multiple original defect images and corresponding annotation mask images, an inflated mask image is obtained by performing an inflated operation on each of the annotation mask images by a preset number of outward expansion pixels. Determine the intersection-union ratio (IoU) of each of the labeled mask images and the corresponding dilated mask images under each defect category; The mean cross-union ratio for each defect category is determined as the cross-union ratio threshold for the corresponding defect category.

[0062] The data cleaning apparatus based on an interactive segmentation model provided in this embodiment of the invention can execute the data cleaning method based on an interactive segmentation model provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0063] Example 4 Figure 6 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0064] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0065] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0066] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data cleaning methods based on interactive segmentation models.

[0067] In some embodiments, the data cleaning method based on the interactive segmentation model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data cleaning method based on the interactive segmentation model described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the data cleaning method based on the interactive segmentation model by any other suitable means (e.g., by means of firmware).

[0068] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0070] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0071] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0072] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0073] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0074] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0075] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data cleaning method based on an interactive segmentation model, characterized in that, The method includes: An image annotation dataset is obtained, comprising multiple original defect images and corresponding annotated mask images. Single-target segmentation is performed based on the annotated mask images to determine the target mask region and the corresponding original target region. The annotated mask images are used to describe the defect location and defect category. The target prediction result is obtained by predicting the defect location of the original target area using an interactive segmentation model based on the initial prediction result and the initial prompt point. Based on the difference between the target prediction result and the target mask area, a target cue point is determined, and based on the target prediction result and the target cue point, the process of predicting the defect location of the original target area through an interactive segmentation model is executed cyclically. When the loop stopping condition is met, the defect location prediction process ends, and the data filtering result of the image annotation dataset is determined; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio (IU) of the target prediction result and the target mask region reaches the IU threshold of the defect category to which the target mask region belongs.

2. The method according to claim 1, characterized in that, Based on the labeled mask image, single-target segmentation is performed to determine the target mask region and the corresponding original target region, including: The bounding rectangles of each connected region in the labeled mask image are determined as the first border, and the area enclosed by the first border is expanded by a first preset multiple to obtain the second border. The image region enclosed by the second border in the labeled mask image is determined as the target mask region; The original target region is determined from the original defect image based on the position information of the target mask region in the labeled mask image.

3. The method according to claim 1, characterized in that, Determining target cue points based on the difference between the target prediction result and the target mask area includes: The missed detection mask region and the false detection mask region are determined based on the difference between the target prediction result and the target mask region; Perform 8-connectivity analysis on the missed detection mask region and the false detection mask region respectively to determine the area and location information of all connected regions in the missed detection mask region and the false detection mask region; The largest connected region in both the missed detection mask region and the false detection mask region is determined as the candidate connected region. The target prompt point is determined based on the area and location information of the candidate connected regions.

4. The method according to claim 3, characterized in that, Determining the target prompt point based on the area and location information of the candidate connected regions includes: A hybrid scoring map of the candidate connected regions is determined based on the location information of the candidate connected regions; wherein, the hybrid scoring map is constructed based on distance transformation, quality balance and cross constraint; The type of prompt point is determined based on the area information of the candidate connected regions and the mixed scoring map; wherein, the type of prompt point is a positive point or a negative point, a positive point is used to indicate the repair of missed detections, and a negative point is used to indicate the repair of false detections; The location of the prompt point is determined based on the mixed scoring map of the candidate connected regions corresponding to the prompt point type; The target prompt point is determined based on the prompt point type and the prompt point location.

5. The method according to claim 4, characterized in that, Determining the hybrid scoring map of the candidate connected regions based on their location information includes: The distance transformation scoring map is determined based on the shortest straight-line distance from each pixel in the candidate connected region to the boundary of the candidate connected region. The cumulative distribution function is determined based on the row and column pixel distribution of the candidate connected regions, and the quality balance score map is determined based on the cumulative distribution function. Based on the size information of the candidate connected regions, a cross-shaped candidate band is constructed with the center of the outer rectangle of the candidate connected regions as the reference, and a cross-shaped constraint scoring map is determined based on the cross-shaped candidate band; The distance transformation score map, the mass balance score map, and the cross constraint score map are fused to obtain the hybrid score map of the candidate connected region.

6. The method according to claim 4 or 5, characterized in that, The type of prompt point is determined based on the area information of the candidate connected regions and the mixed scoring map, including: The largest connected region in the missed detection mask region and the largest connected region in the false detection mask region are respectively designated as the first region and the second region. The largest scores in the mixed scoring maps corresponding to the first region and the second region are respectively designated as the first mixed score and the second mixed score. If the area of ​​the first region is greater than or equal to a second preset multiple of the area of ​​the second region, then the prompt point type is determined to be a positive point; wherein, the second preset multiple is less than 1; If the area of ​​the first region is less than a second preset multiple of the area of ​​the second region, then determine whether the product of the first mixed score and the preset deviation coefficient is less than the second mixed score; wherein the preset deviation coefficient is greater than 1; If so, then the type of the prompt point will be determined as a negative point; Otherwise, the prompt point type will be set to "positive point".

7. The method according to claim 6, characterized in that, Determining the location of a prompt point based on the mixed scoring map of the candidate connected regions corresponding to the prompt point type includes: If the prompt point type is positive, then the pixel position corresponding to the first mixed score is determined as the prompt point position; If the prompt point type is negative, then the pixel position corresponding to the second mixed score is determined as the prompt point position.

8. The method according to claim 1, characterized in that, Determining the data filtering results of the image annotation dataset includes: If the maximum number of iterations is not reached by the end of the defect location prediction process, the target mask area is determined to be qualified annotation data. If the cross-union ratio (CUB) between the target prediction result and the target mask region does not reach the CUB threshold for the defect category to which the target mask region belongs at the end of the defect location prediction process, then the target mask region is determined to be unqualified labeled data.

9. The method according to claim 1, characterized in that, After obtaining the image annotation dataset, which includes multiple original defect images and corresponding annotated mask images, the following steps are also included: Each of the labeled mask images is subjected to a dilation operation with a preset number of outward expansion pixels to obtain a dilated mask image; Determine the intersection-union ratio (IoU) of each of the labeled mask images and the corresponding dilated mask images under each defect category; The mean cross-union ratio for each defect category is determined as the cross-union ratio threshold for the corresponding defect category.

10. A data cleaning device based on an interactive segmentation model, characterized in that, The device includes: The single-target segmentation module is used to acquire an image annotation dataset including multiple original defect images and corresponding labeled mask images, and to perform single-target segmentation based on the labeled mask images to determine the target mask region and the corresponding original target region; wherein, the labeled mask images are used to describe the defect location and defect category; The initial prediction module is used to predict the defect location of the original target region based on the initial prediction result and the initial prompt point using an interactive segmentation model to obtain the target prediction result. The cyclic prediction module is used to determine the target cue point based on the difference between the target prediction result and the target mask area, and to cyclically execute the process of predicting the defect location of the original target area through an interactive segmentation model based on the target prediction result and the target cue point. The data filtering module is used to end the defect location prediction process and determine the data filtering result of the image annotation dataset when the loop stopping condition is met; wherein, the loop stopping condition is reaching the maximum number of loops or the intersection-union ratio (IU) of the target prediction result and the target mask region reaches the IU threshold of the defect category to which the target mask region belongs.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data cleaning method based on the interactive segmentation model according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data cleaning method based on the interactive segmentation model as described in any one of claims 1-9.

Images