Semantic segmentation model evaluation method and device, electronic equipment and medium
By introducing morphological dilation processing to generate edge tolerance regions in the semantic segmentation model evaluation, the problem of traditional evaluation indicators being overly sensitive to edge errors is solved, achieving a more accurate evaluation of industrial micro-defect detection models and improving the fairness and adaptability of the evaluation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-03
AI Technical Summary
In industrial micro-defect detection scenarios, existing semantic segmentation models are overly sensitive to edge errors in traditional evaluation metrics, leading to an underestimation of model performance and an inability to accurately reflect the model's actual detection effectiveness.
Morphological dilation is introduced to generate an edge tolerance zone. False positives that fall into the tolerance zone are eliminated, and only false positives outside the edge tolerance zone are retained for evaluation. The edge of the defect area is processed by an adaptive dilation kernel to simulate the subjective judgment logic of manual quality inspection, which only checks whether something exists.
It effectively reduces the interference of annotation subjectivity on algorithm scoring, improves the evaluation effect of industrial micro-defect detection models, objectively reflects the model's true detection capability, and meets the business needs of industrial quality inspection.
Smart Images
Figure CN121788941A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a semantic segmentation model evaluation method, apparatus, electronic device, and medium. Background Technology
[0002] With the development of intelligent manufacturing, semantic segmentation technology based on deep learning (such as U-Net, DDR-Net, etc.) has been widely used in the detection of surface defects (such as scratches, cracks, oxidation points, etc.) in industrial products. These technologies locate and segment defect areas through pixel-level classification, and have become an important means to replace manual visual inspection.
[0003] In existing technologies, most semantic segmentation models originate from natural image segmentation (such as vehicle and pedestrian segmentation in autonomous driving). In these scenarios, object boundaries are relatively clear, and pixel-level accuracy is sought. Therefore, the mainstream evaluation metrics for semantic segmentation model performance are pixel-level Intersection over Union (IOU) and pixel-level precision / recall. When calculating these evaluation metrics, the model's predicted binary mask is typically compared pixel-by-pixel with the manually labeled ground truth mask. If the values of two pixels are the same (both 1 or both 0), it is considered correct; otherwise, it is considered incorrect.
[0004] However, the edges of minor industrial defects (such as scratches and oil stains) are often blurred, typically exhibiting characteristics of indistinct boundaries but relatively defined core areas. Manual annotation is subject to significant subjective errors, and industrial quality inspection usually focuses on the presence of defects and whether the defect area exceeds the standard, rather than simply whether pixels are completely overlapping. Directly applying evaluation standards from general fields ignores the tolerance requirements for edge errors in industrial scenarios, leading to oversensitivity of evaluation results to edge errors. This results in distorted detection results (i.e., the model's actual detection performance is good, but conventional metrics such as IOU are low). For example, if the model correctly predicts the defect location, but the edge is a few pixels more or less than the annotation, the traditional IOU metric will drop significantly, causing the model's performance to be severely underestimated (e.g., the detection appears accurate to the naked eye, but the IOU is only 0.4). Summary of the Invention
[0005] In view of this, it is necessary to provide a semantic segmentation model evaluation method, device, electronic device and medium to solve the technical problem of distortion of existing semantic segmentation evaluation indicators in industrial micro-defect detection scenarios.
[0006] To address the aforementioned problems, in a first aspect, the present invention provides a semantic segmentation model evaluation method, comprising: Obtain the binarized model prediction map and the manually labeled map, wherein the model prediction map is obtained by the semantic segmentation model to be evaluated; Extract the defect region from the manually annotated image, and perform morphological dilation processing on the edge of the defect region to generate an edge tolerance region; The semantic segmentation model is evaluated by calculating evaluation metrics using only the model prediction map as defective, the manually labeled map as defect-free, and pixels outside the edge tolerance zone as false positives.
[0007] In one possible implementation, defect regions are extracted from the manually annotated image, and morphological dilation is performed on the edges of the defect regions to generate edge-tolerant regions, including: Extract the defect areas and their corresponding dimensional features from the manually annotated diagram; Based on the size feature, the size of the expansion kernel is determined, and morphological expansion processing is performed on the edge of the defect region based on the expansion kernel to generate an edge tolerance region. The larger the size feature, the larger the expansion kernel.
[0008] In one possible implementation, extracting the defect region and its corresponding size features from the manually annotated image includes: Connectivity analysis was performed on the manually annotated map to extract several independent defect regions and their corresponding size features; Based on the aforementioned size characteristics, the size of the expansion kernel is determined. Then, morphological expansion processing is performed on the edge of the defect region based on the expansion kernel to generate an edge tolerance zone, including: Based on the size characteristics, the size of the expansion kernel is determined, and morphological expansion processing is performed on the edge of the corresponding defect region based on the expansion kernel to obtain the local tolerance region. All local tolerance regions are merged to generate the final edge tolerance region.
[0009] In one possible implementation, the size feature is pixel area.
[0010] In one possible implementation, the size of the expanded kernel is not less than a preset minimum kernel size and not more than a preset maximum kernel size.
[0011] In one possible implementation, the size of the expanding kernel In the formula, Indicates the size of the expansion core, Indicates the scaling factor. Represents pixel area. Indicates the minimum kernel size. This indicates the maximum kernel size.
[0012] In one possible implementation, the evaluation metrics include at least one or two of pixel-level precision and pixel-level intersection-over-union ratio, wherein pixel-level precision... pixel-level intersection ratio In the formula, Indicates pixel-level precision. Indicates the pixel-level intersection-union ratio. Indicates a real example, Indicates a false positive example. This indicates a false negative.
[0013] Secondly, the present invention also provides a semantic segmentation model evaluation system, comprising: The data acquisition module is used to acquire the initial model probability prediction map and the manually labeled ground truth mask. The model probability prediction map is obtained by predicting the semantic segmentation model to be evaluated. The preprocessing module is used to binarize the model probability prediction map and the manually labeled ground truth mask to obtain the binarized model prediction map and the manually labeled map. The tolerance generation module is used to extract the defect region from the manually annotated map, perform morphological dilation processing on the edge of the defect region, and generate an edge tolerance region. The metric evaluation module is used to calculate evaluation metrics by only indicating defects in the model prediction map, indicating no defects in the manually labeled map, and using pixels outside the edge tolerance zone as false positives, in order to evaluate the semantic segmentation model.
[0014] Thirdly, the present invention also provides an electronic device, including a memory and a processor; The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the semantic segmentation model evaluation method described above.
[0015] Fourthly, the present invention also provides a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the semantic segmentation model evaluation method described above.
[0016] The beneficial effects of this invention are as follows: The semantic segmentation model evaluation method provided by this invention introduces a tolerance mechanism to generate an edge tolerance zone for defect areas in manually annotated images through morphological dilation. Traditional false positives falling into the edge tolerance zone are eliminated, and only false positives falling outside the edge tolerance zone are retained to calculate evaluation indicators. This innovatively applies morphological dilation to the back-end evaluation system, using morphological characteristics to simulate the subjective judgment logic of manual quality inspection, which only looks at the presence or absence of defects and not the edges. This effectively reduces the interference of annotation subjectivity on algorithm scoring, improves the evaluation effect on the detection capability of industrial micro-defect detection models, avoids the oversensitivity of traditional indicators to edge errors, and effectively filters out real invalid false alarms. It fits the business needs of industrial quality inspection and provides an evaluation method that can objectively reflect the model's true detection capability, resist edge interference, and resist prediction breakage, assisting in model selection and threshold optimization in industrial quality inspection systems. Attached Figure Description
[0017] Figure 1 A schematic flowchart of an embodiment of the semantic segmentation model evaluation method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of step S12; Figure 3 A schematic diagram of an embodiment of the semantic segmentation model evaluation system provided by the present invention; Figure 4 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0019] It should be understood that the illustrative drawings are not drawn to scale. The flowcharts used in this invention illustrate operations implemented according to some embodiments of the invention. It should be understood that the operations in the flowcharts may be implemented out of order, and steps without logical contextual relationships may be reversed or performed simultaneously. Furthermore, those skilled in the art, guided by the content of this invention, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.
[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. "And / or" describes the relationship between related objects, indicating that three relationships may exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone.
[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0022] This invention provides a semantic segmentation model evaluation method, apparatus, electronic device, and medium, which are described below.
[0023] Figure 1 A schematic diagram of an embodiment of the semantic segmentation model evaluation method provided by the present invention is shown below. Figure 1 As shown, semantic segmentation model evaluation methods include: S11. Obtain the binarized model prediction map and the manually labeled map. The model prediction map is obtained by the semantic segmentation model to be evaluated.
[0024] It should be noted that the prediction map of the model to be evaluated and the manually labeled ground truth need to be binarized before they can be used. In some embodiments of this application, the binarization method specifically includes: based on a preset confidence threshold... (e.g., 0.25), predict the probability map of the model to be evaluated. Converted to binarized model prediction graph The conversion formula is as follows: Convert the manually annotated ground truth mask into a binarized manually annotated image (foreground is 1, background is 0).
[0025] It should be noted that the binarization method of this application is not limited to the above method. In other embodiments of this application, fixed threshold segmentation, Otsu's method, adaptive thresholding method and other techniques can also be used to generate binary masks, which will not be described in detail here.
[0026] S12. Extract the defect area from the manually annotated map, perform morphological dilation on the edge of the defect area, and generate an edge tolerance zone.
[0027] Considering that existing technologies typically use globally fixed parameters (e.g., IOU > 0.5 for the entire image is considered acceptable) when determining thresholds or parameters, while in industrial defect detection, the defect scale range is extremely wide (from a few pixels to thousands of pixels). The fixed parameters in existing technologies can lead to excessively high tolerances for small defects, causing them to be drowned out by background noise, or excessively low tolerances for large defects, leading to false alarms. Therefore, to avoid the problem of adjacent small defects being incorrectly merged or misjudged due to a fixed high tolerance, in some embodiments of this invention, an adaptive expansion kernel is introduced, such as... Figure 2 As shown, step S12 includes: S121. Extract the defect areas and their corresponding dimensional features from the manually annotated drawings.
[0028] Conventional methods for extracting defect regions include threshold segmentation, edge detection, and other algorithms. This application considers that industrial images typically contain multiple independent defects, requiring the use of an adaptive dilation kernel to calculate the area and match the dilation kernel for each independent defect. In some embodiments of this invention, step S121 includes: performing connected component analysis on the manually annotated image to extract several independent defect regions and their corresponding size features; thereby, through connected component analysis, independent defects are accurately separated, separating all annotated defects into... A single connected object It directly outputs the features of the defect area, such as area and centroid.
[0029] Conventional dimensional features include perimeter and the length of the major axis of the circumscribed rectangle. This application takes into account the following: First, industrial quality inspection typically focuses on the presence of defects and whether the defect area exceeds the standard; that is, area is the most direct and core judgment indicator in industrial quality inspection. Large defects are matched with large expansion kernels (high tolerance), and small defects are matched with small expansion kernels (low tolerance). Second, industrial defects often have blurred edges, and manual annotation introduces subjective errors. These errors significantly affect dimensional features such as perimeter and the length of the major axis of the circumscribed rectangle, while area features are insensitive to these errors. Third, area calculation is simple and low-cost. Fourth, industrial micro-defects have diverse shapes, and area features can stably adapt to defects of various shapes. Therefore, in some embodiments of this invention, the dimensional feature uses pixel area, traversing each connected object. Calculate its size characteristics, i.e., pixel area. .
[0030] S122. Based on the size characteristics, determine the size of the expansion kernel, and perform morphological expansion processing on the edge of the defect region based on the expansion kernel to generate an edge tolerance zone. The larger the size characteristics, the larger the expansion kernel.
[0031] To avoid unlimited expansion, in some embodiments of the invention, strict upper and lower limits are set, and the size of the expansion kernel is not lower than the preset minimum kernel size and not higher than the preset maximum kernel size.
[0032] Based on the above considerations, in some embodiments of the invention, a nonlinear mapping function is constructed. This results in large defects corresponding to larger expansion nuclei, and small defects corresponding to smaller expansion nuclei, with the size of the expansion nuclei... The calculation formula is: In the formula, Indicates the size of the expansion core, This represents the scaling factor (which can be 0.5). Represents pixel area. This indicates that the minimum kernel size is 0. This indicates a maximum kernel size of 15 (with an outward expansion radius of 7 pixels), using Limit the calculation results to In this way, we can avoid unlimited expansion.
[0033] It should be noted that the formula for calculating the expansion kernel size in this application is not limited to the aforementioned nonlinear mapping function. In other embodiments of this application, piecewise functions and other methods can also be used to calculate and determine the expansion kernel size, which will not be elaborated here. Among them, replacing the aforementioned nonlinear mapping function with a piecewise function is used. In step S122, the size of the expansion kernel is determined based on the size characteristics, including: determining the size of the expansion kernel based on the size characteristics and a preset piecewise function, and adopting a corresponding fixed kernel size within different area ranges.
[0034] Correspondingly, step S122 includes: determining the size of the expansion kernel based on size characteristics, and based on the expansion kernel... Morphological dilation is performed on the edges of the corresponding defect region to obtain the local tolerance region. All local tolerance regions are merged to generate the final edge tolerance region, i.e., the global tolerance region mask. .
[0035] It should be noted that conventional methods tend to simplify the evaluation process to increase speed. This proposal introduces connected component analysis and feature computation as preliminary steps in the evaluation, increasing computational complexity but in exchange for fairness in the evaluation.
[0036] In other embodiments, max pooling can be used instead of standard morphological dilation. Step S12 includes: extracting defect regions from the manually annotated image, performing max pooling on the edges of the defect regions, and generating edge tolerance regions. Mathematically, performing max pooling on a binary image with a kernel size of K and a stride of 1 is equivalent to morphological dilation. This is more efficient to implement in deep learning frameworks (such as PyTorch) and supports GPU acceleration.
[0037] S13. Only pixels that are indicated as defects in the model prediction map and as no defects in the manually labeled map, and pixels outside the edge tolerance zone are used as false positives to calculate the evaluation index for evaluating the semantic segmentation model.
[0038] It should be noted that the upper-level evaluation metrics are calculated based on the basic elements of the confusion matrix: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). In this invention, TP, TN, FP, and FN specifically refer to: Traversal model prediction graph Each positive pixel (value = 1) is classified based on its spatial location: Case 1: If the pixel In manually annotated maps The corresponding value is 1, that is If it is detected, it is marked as a valid detection. A correct prediction is a true example.
[0039] Scenario 2: If the pixel is in the manually annotated image It is 0, but in the tolerance zone mask. The value is 1 (i.e., within the tolerance zone), which means... If it is, it will be marked as a benign false alarm. Such pixels are also considered correct predictions and are not counted as errors; they are removed from the final prediction.
[0040] Scenario 3: If the pixel is in the manually annotated image The value is 0, within the tolerance zone mask. The value is also 0 (i.e., outside the tolerance zone), that is If it is, it is marked as a strict false alarm. These pixels represent background noise and are included in the error statistics.
[0041] for Negative pixels (value = 0) are processed normally; if the pixel... If the value corresponding to the manually labeled image is 1, then the prediction error is a false negative. Conversely, a true negative example is an example of a negative example. .
[0042] Based on the above classification statistics, benign false positives were eliminated. Retain strict false alarms Calculate the corrected evaluation metrics, including: pixel-level accuracy: Pixel-level intersection-over-union ratio: .
[0043] It should be noted that existing technologies employ a rigid ground truth principle. If a predicted result exceeds the defect range of the manually labeled image by even one pixel, it is considered a false positive and included in the error statistics. This invention generates an edge tolerance zone around the defect in the manually labeled image through morphological dilation. Predicted pixels falling within the edge tolerance zone are marked as benign false positives and are intentionally removed when calculating precision and intersection-over-union (IOU), and are not included in the penalty denominator. To improve recall, a conventional approach is to lower the IOU threshold (e.g., from 0.5 to 0.1). However, simply lowering the threshold leads to a large number of background noise points being misclassified as correct detections, making it impossible to distinguish between false positives due to positional deviations and benign errors due to edge overflow. This proposal, by constructing a tolerance zone, accurately "forgives" edge errors while severely penalizing noise points far from the defect, achieving a more scientific evaluation that better aligns with human visual perception.
[0044] Compared with existing technologies, this invention introduces a tolerance mechanism to generate an edge tolerance zone for defect areas in manually annotated images through morphological dilation. Traditional false positives falling into the edge tolerance zone are eliminated, and only false positives falling outside the edge tolerance zone are retained for calculating evaluation indicators. This invention creatively applies morphological dilation to the back-end evaluation system, using morphological characteristics to simulate the subjective judgment logic of manual quality inspection, which only looks at the presence or absence of defects and ignores the edges. This effectively reduces the interference of annotation subjectivity on algorithm scoring, improves the evaluation effect on the detection capability of industrial micro-defect detection models, avoids the oversensitivity of traditional indicators to edge errors, and effectively filters out genuine invalid false alarms. It aligns with the business needs of industrial quality inspection and provides an evaluation method that can objectively reflect the model's true detection capability, resist edge interference, and resist prediction breakage, assisting in model selection and threshold optimization in industrial quality inspection systems.
[0045] To verify the technical effects of the present invention, the following verification experiments were also conducted: Application scenario: Defect detection on the end face of electrode caps in automotive welding production lines.
[0046] Specific implementation process: Input data: An image containing an electrode cap defect. The area of a defect in the manually annotated image is also included. It is 1200 pixels.
[0047] Model prediction: The deep learning semantic segmentation model output the prediction results, but due to the influence of lighting, the edge of the predicted region overflows by about 6 pixels compared to the manually labeled area.
[0048] Adaptive parameter calculation: Calculate the theoretical kernel size according to the formula: Assumptions ,but .because The system forcibly sets the expansion kernel size to [value]. (That is, the maximum allowable expansion radius is 7 pixels).
[0049] Tolerance zone generation: for defective regions The expansion operation is performed outwards, and the resulting edge tolerance zone covers the defect area. Extend outwards by 7 pixels.
[0050] Judgment and Evaluation: The model predicted an overflow of 6 pixels, which is less than the allowed upper limit of 7 pixels. These overflow pixels fall within the edge tolerance zone and are judged by the system as benign false positives. These pixels are discarded when calculating accuracy metrics and are not included in the penalty.
[0051] Comparison conclusion: Improved evaluation accuracy: When evaluating the UNet (ResNet-34) model using the traditional standard pixel-level IOU, the model score is 0.776. Using the pixel-level IOU (Kernel=41) evaluation method of this invention, the model score is improved to 0.881; this 10.5% improvement does not mean that the model has become better, but rather that we have eliminated those false positives and restored the model's true capabilities.
[0052] The false positive rate is significantly reduced: the traditional method has 180,111 false positive pixels (FP). The method of this invention identifies 140,873 of these as benign false positives. The corrected number of strict false positives is only 39,238; this means that 78% of the errors in the traditional metric are actually acceptable marginal errors. This invention accurately isolates this portion of error, avoiding the false positives of good models.
[0053] Furthermore, without an upper limit constraint, an excessively large expansion kernel (such as 35) may devour surrounding small background noise, leading to an inflated evaluation; while limiting it to 7 pixels accommodates reasonable edge errors while maintaining sensitivity to background noise far from the edges.
[0054] To better implement the semantic segmentation model evaluation method in this embodiment of the invention, based on a semantic segmentation model evaluation method, correspondingly, as follows: Figure 3 As shown, this embodiment of the invention also provides a semantic segmentation model evaluation system 30, comprising: The data acquisition module 31 is used to acquire the initial model probability prediction map and the manually labeled ground truth mask. The model probability prediction map is obtained by predicting the semantic segmentation model to be evaluated.
[0055] It should be noted that the data acquisition module 31 is used to acquire image data of the industrial products to be evaluated. This module is connected to the preprocessing module, which sends the original model prediction probability map and the manually labeled ground truth to it.
[0056] The preprocessing module 32 is used to binarize the model probability prediction map and the manually labeled ground truth mask to obtain the binarized model prediction map and manually labeled map.
[0057] It should be noted that the preprocessing module 32 is connected to the data acquisition module 31 and is used to perform binarization processing on the input raw image data. This module contains two data output terminals: The first output terminal is used to output the binarized model prediction map, which is directly connected to the index evaluation module 34. The second output terminal is used to output the binarized manually labeled image, and is also connected to the index evaluation module 34 and the tolerance generation module 33.
[0058] The tolerance generation module 33 is used to extract the defect area in the manually annotated map, perform morphological dilation processing on the edge of the defect area, and generate the edge tolerance area. It should be noted that the tolerance generation module 33 is the core logical unit of this system, used to dynamically generate edge tolerance regions based on real labeled data. This module further includes: Feature analysis unit: used to receive manually annotated maps, extract independent defect connected components and calculate their size features (such as area); Parameter calculation unit: connected to the feature analysis unit, used to adaptively map morphological expansion parameters (expansion kernel size) based on size features. Morphological processing unit: Connected to the parameter calculation unit, it is used to perform local morphological dilation operations and merge them to generate a globally tolerable region mask.
[0059] The output of this module is connected to the index evaluation module 34, which sends mask data containing edge tolerance zone information to it.
[0060] The metric evaluation module 34 is used to calculate evaluation metrics only for pixels that are indicated as defects in the model prediction map, as no defects in the manually labeled map, and outside the edge tolerance zone as false positives, so as to evaluate the semantic segmentation model.
[0061] It should be noted that the indicator evaluation module 34 further includes: Pixel classification and determination unit: This is a multi-channel data aggregation node, receiving model prediction map data and manually labeled map data from preprocessing module 32, as well as edge tolerance region masks from tolerance generation module 33. This module is used to determine the spatial location of each pixel in the image and classify the prediction results into valid detections (…). Strict false alarms () ) or benign false alarm ( ).
[0062] The index calculation unit, connected to the pixel classification and determination module, is used to calculate the tolerance evaluation index based on the classification statistics. This module is configured with filtering logic to automatically remove data marked as benign false positives during the calculation process.
[0063] Output display unit: Connected to the indicator calculation module, it is used to generate and display the final evaluation report, visual comparison charts, and values of various performance indicators.
[0064] Through the collaborative work of the above modules, this system can automatically identify and tolerate non-principle errors caused by edge blurring in industrial defect detection, thereby outputting an evaluation result that objectively reflects the model's detection capability.
[0065] The semantic segmentation model evaluation system 30 provided in the above embodiments can implement the technical solutions described in the above semantic segmentation model evaluation method embodiments. The specific implementation principles of each unit can be found in the corresponding content in the above semantic segmentation model evaluation method embodiments, and will not be repeated here.
[0066] like Figure 4 As shown, the present invention also provides an electronic device 40. The electronic device 40 includes a processor 41, a memory 42, and a display 43. Figure 4 Only some of the components of the electronic device 40 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.
[0067] In some embodiments, memory 42 may be an internal storage unit of electronic device 40, such as a hard disk or memory of electronic device 40. In other embodiments, memory 42 may also be an external storage device of electronic device 40, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 40.
[0068] Furthermore, the memory 42 may include both internal storage units of the electronic device 40 and external storage devices. The memory 42 is used to store application software and various types of data installed on the electronic device 40.
[0069] In some embodiments, processor 41 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 42 or process data, such as the semantic segmentation model evaluation method of the present invention.
[0070] In some embodiments, display 43 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 43 is used to display information from electronic device 40 and to display a visual user interface. Components 41-43 of electronic device 40 communicate with each other via a system bus.
[0071] In some embodiments of the present invention, when the processor 41 executes the semantic segmentation model evaluation program in the memory 42, the following steps may be performed: Obtain the binarized model prediction map and the manually labeled map. The model prediction map is obtained by the semantic segmentation model to be evaluated. Extract the defect area from the manually annotated image, perform morphological dilation on the edge of the defect area to generate an edge tolerance zone; The evaluation metric is calculated using only pixels that are indicated as defects in the model prediction map, as no defects in the manually labeled map, and outside the edge tolerance zone as false positives, in order to evaluate the semantic segmentation model.
[0072] It should be understood that when the processor 41 executes the semantic segmentation model evaluation program in the memory 42, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.
[0073] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 40 mentioned. Electronic device 40 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 40 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).
[0074] Accordingly, embodiments of the present invention also provide a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the semantic segmentation model evaluation methods provided in the above-described method embodiments.
[0075] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.
[0076] The semantic segmentation model evaluation method provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
[0077] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A semantic segmentation model evaluation method, characterized in that, include: Obtain the binarized model prediction map and the manually labeled map, wherein the model prediction map is obtained by the semantic segmentation model to be evaluated; Extract the defect region from the manually annotated image, and perform morphological dilation processing on the edge of the defect region to generate an edge tolerance region; The semantic segmentation model is evaluated by calculating evaluation metrics using only the model prediction map as defective, the manually labeled map as defect-free, and pixels outside the edge tolerance zone as false positives.
2. The semantic segmentation model evaluation method according to claim 1, characterized in that, Extract the defect region from the manually annotated image, perform morphological dilation on the edges of the defect region to generate an edge tolerance region, including: Extract the defect areas and their corresponding dimensional features from the manually annotated diagram; Based on the size feature, the size of the expansion kernel is determined, and morphological expansion processing is performed on the edge of the defect region based on the expansion kernel to generate an edge tolerance region. The larger the size feature, the larger the expansion kernel.
3. The semantic segmentation model evaluation method according to claim 2, characterized in that, Extracting the defect areas and their corresponding dimensional features from the manually annotated image includes: Connectivity analysis was performed on the manually annotated map to extract several independent defect regions and their corresponding size features; Based on the aforementioned size characteristics, the size of the expansion kernel is determined. Then, morphological expansion processing is performed on the edge of the defect region based on the expansion kernel to generate an edge tolerance zone, including: Based on the size characteristics, the size of the expansion kernel is determined, and morphological expansion processing is performed on the edge of the corresponding defect region based on the expansion kernel to obtain the local tolerance region. All local tolerance regions are merged to generate the final edge tolerance region.
4. The semantic segmentation model evaluation method according to claim 2, characterized in that, The size feature is the pixel area.
5. The semantic segmentation model evaluation method according to claim 2, characterized in that, The size of the expansion core is not less than the preset minimum core size and not more than the preset maximum core size.
6. The semantic segmentation model evaluation method according to claim 2, characterized in that, The size of the expansion core In the formula, Indicates the size of the expansion core, Indicates the scaling factor. Represents pixel area. Indicates the minimum kernel size. This indicates the maximum kernel size.
7. The semantic segmentation model evaluation method according to claim 1, characterized in that, The evaluation metrics include at least one or two of pixel-level accuracy and pixel-level intersection-over-union ratio, wherein pixel-level accuracy pixel-level intersection ratio In the formula, Indicates pixel-level precision. Indicates the pixel-level intersection-union ratio. Indicates a real example, Indicates a false positive example. This indicates a false negative.
8. A semantic segmentation model evaluation system, characterized in that, include: The data acquisition module is used to acquire the initial model probability prediction map and the manually labeled ground truth mask. The model probability prediction map is obtained by predicting the semantic segmentation model to be evaluated. The preprocessing module is used to binarize the model probability prediction map and the manually labeled ground truth mask to obtain the binarized model prediction map and manually labeled map. The tolerance generation module is used to extract the defect region from the manually annotated map, perform morphological dilation processing on the edge of the defect region, and generate an edge tolerance region. The metric evaluation module is used to calculate evaluation metrics by only indicating defects in the model prediction map, indicating no defects in the manually labeled map, and using pixels outside the edge tolerance zone as false positives, in order to evaluate the semantic segmentation model.
9. An electronic device, characterized in that, Including memory and processor; The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the semantic segmentation model evaluation method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the semantic segmentation model evaluation method according to any one of claims 1 to 7.