Model management method and device for re-judgment model, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a model management method and device for a re-judgment model, electronic equipment and a storage medium. The method comprises the steps of displaying a labeling window of a target re-judgment model in response to a labeling instruction which is input by a user and aims at the target re-judgment model; displaying a sample defect detection result of the sample product in a labeling window, wherein the sample defect detection result comprises at least one defect record; and in response to a labeling operation of a user for at least part of defect records in the sample defect detection result, determining category labels corresponding to the at least part of defect records, and storing the at least part of defect records and the corresponding category labels as training data in a sample library, the training data being used for training the re-judgment model. According to the scheme, the user is allowed to manage the re-judgment model and customize the category label of the re-judgment model, so that the management autonomy of the re-judgment model can be greatly improved, the use scene of the re-judgment model is expanded, the development workload and cost of the re-judgment model are reduced, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of product testing technology, and more specifically to a model management method and apparatus, electronic device and storage medium for a re-evaluation model. Background Technology

[0002] Product defect detection is a core aspect of quality control in manufacturing, aiming to identify various defects on or inside workpieces through automated means to ensure high yield rates. The following uses wafer defect detection as an example to describe existing problems in the technology. It should be understood that wafers are merely an example; other similar products requiring defect detection (such as LCD screen panels and lithium battery electrodes) also face the same issues. In semiconductor manufacturing processes, microscopic defects on the wafer surface directly affect chip yield and reliability. Wafer defect detection is a critical quality control step in semiconductor manufacturing, aiming to identify and locate minute defects on or inside wafers to ensure chip yield and reliability. The wafer detection process typically includes high-resolution imaging, image processing, and intelligent analysis. For example, high-contrast images of the wafer surface can be acquired, and defect features can be captured using line-scan or area-scan cameras under different lighting conditions. Subsequently, deep learning models or traditional image processing defect detection algorithms can be combined to classify and identify defects in the images.

[0003] In wafer inspection scenarios, various chip (die) designs and defect types are encountered. Traditional defect detection algorithms suffer from over-detection or under-detection due to factors such as the varying axis movement speed of imaging equipment. In other words, defects are diverse and variable, making accurate differentiation difficult using conventional algorithms. In such cases, a re-judgment model (e.g., an artificial intelligence model, or AI model) can be used to correct over-detection and identify new defects. Of course, besides wafers, other similar products may also require re-judgment based on their defect detection results.

[0004] In existing technical solutions, the product's re-judgment model cannot be managed and controlled by the user (such as the product manufacturer). When the existing category labels of the re-judgment model cannot meet the user's classification needs, the re-judgment model designer can be contacted to adjust the re-judgment model. Summary of the Invention

[0005] The present invention is proposed in view of the above-mentioned problems. The present invention provides a model management method, a model management device, an electronic device, and a storage medium for a re-judgment model.

[0006] According to one aspect of the present invention, a model management method for a review model is provided. The review model is used to review the defect detection results of a product, and the defect detection results are obtained by a preset defect detection algorithm. The method includes: displaying a labeling window of the target review model in response to a user-inputted labeling instruction for the target review model; displaying sample defect detection results of a sample product in the labeling window, the sample defect detection results including at least one defect record, each defect record corresponding to a defect location detected on the sample product by the preset defect detection algorithm; and determining the category labels corresponding to each of the at least some defect records in the sample defect detection results in response to a user's labeling operation for at least some defect records, and saving the at least some defect records and their corresponding category labels as training data in a sample library, the training data being used to train the review model.

[0007] For example, the annotation window includes a first display area for displaying label information items of category labels. Each label information item is associated with a unique category label. The label information item includes the label name of the associated category label and the operation instruction corresponding to the category label. In response to a user's annotation operation on at least some defect records in the sample defect detection results, determining the category label corresponding to each of the at least some defect records includes: when the operation instruction input by the user for any defect record is consistent with the operation instruction corresponding to any category label, determining that the category label corresponding to the defect record is the category label corresponding to the operation instruction. The annotation operation includes the user's input of the operation instruction. The label information items in the first display area are editable information items. The method further includes one or more of the following operations: in response to a user's import operation, importing label information items from a preset import location to the first display area for display; in response to a user's add operation, adding a new label information item in the first display area; in response to a user's delete operation on any label information item, deleting the label information item from the first display area; in response to a user's edit operation on any label information item, editing at least some of the information contained in the label information item.

[0008] For example, the first display area displays one or more of the following controls: an import control, an add control, a delete control, and an edit control, wherein the import operation includes a triggering operation on the import control, the add operation includes a triggering operation on the add control, the delete operation includes a triggering operation on the delete control, and the edit operation includes a triggering operation on the edit control; and / or, the label information item also includes a method of highlighting the defect location on the product image of the sample product (specifically, the defect image described below) when the defect location has a category label associated with the label information item.

[0009] For example, in response to a user's editing operation on any label information item, editing at least some of the information contained in the label information item includes: in response to the user's editing operation on any label information item, displaying a label editing window; in response to the setting information entered by the user in the label editing window, determining at least some of the information contained in the label information item, and setting the remaining information of the label information item to one of the default information or a random setting in a manner distinguishable from other category labels.

[0010] For example, before displaying the sample defect detection results of the sample product in the annotation window, the method further includes: responding to a defect selection instruction input by the user, searching for matching defect records from a preset database according to the search conditions indicated by the defect selection instruction to obtain initial defect detection results. The preset database stores defect detection results corresponding to at least one batch of products processed by at least one machine. The search conditions include one or more of the following: machine identification information, product batch identification information, product identification information, and product image scanning time. The initial defect detection results are then determined as sample defect detection results. Alternatively, in response to the user's selection operation for at least some defect records in the initial defect detection results, the user-selected defect records are determined as sample defect detection results.

[0011] For example, before searching for matching defect records from a preset database according to the search conditions indicated by the defect selection instruction in response to a user input, and obtaining an initial defect detection result, the method further includes: displaying a defect selection control in an annotation window; displaying a defect selection window in response to a user triggering the defect selection control, and providing an input area for search conditions in the defect selection window; wherein the defect selection instruction includes input information for one or more search conditions in the input area.

[0012] For example, the defect selection window includes a second display area and a third display area. In response to a user's selection operation for at least a portion of the defect records in the initial defect detection results, the user-selected defect records are determined as sample defect detection results. This includes: displaying product information items corresponding to each product in the initial defect detection results in the second display area, where each product information item includes product identification information; displaying the defect records of the product corresponding to that product information item in the third display area in response to a user's selection operation for any product information item; adding the user-selected defect records to the temporary defect detection results in response to a user-input completion command; and determining the current temporary defect detection results as sample defect detection results in response to a user-input completion command.

[0013] For example, the defect selection window displays a completion control, and the completion instructions include the operation instructions entered by the user when performing a trigger operation on the completion control.

[0014] For example, the annotation window also includes a fourth display area, and the method further includes: in response to a user's selection operation on any defect record in the sample defect detection results, displaying the annotation information of the defect record in the fourth display area, the annotation information including the label name of the corresponding category label when it is not annotated and when it is annotated.

[0015] For example, the method further includes: in response to a training instruction input by a user, displaying a model selection window, the model selection window including a first selection control and a second selection control; in response to a user's trigger operation on the first selection control, activating a third selection control of a pre-trained model; in response to a user's selection operation of a pre-trained model through the third selection control, using the user-selected pre-trained model as the target re-judgment model, and training the target re-judgment model based on training data in the sample library; in response to a user's trigger operation on the second selection control, using a preset initial model as the target re-judgment model, and training the target re-judgment model based on training data in the sample library.

[0016] For example, before displaying the annotation window of the target re-judgment model in response to a user-input annotation instruction for the target re-judgment model, the method further includes: displaying a model management page, which displays model project information items, each model project information item being associated with a unique model project, the model project representing a project for a single annotation and training of a single re-judgment model, the model project information item including one or more of the following information: model project name, annotation status, creation time, update time, a first control, a second control, and a third control; the first control is used to display the annotation window in response to a user's trigger operation, the annotation instruction including the operation instruction entered by the user when performing a trigger operation on the first control, the second control is used to set the model project information item to an editable state in response to a user's trigger operation, so that the user can edit the model project name of the model project information item, and the third control is used to delete the corresponding model project information item and the model project associated with the model project information item in response to a user's trigger operation.

[0017] According to another aspect of the present invention, a model management device for a re-judgment model is also provided. The re-judgment model is used to re-judge the defect detection results of a product. The defect detection results are obtained by a preset defect detection algorithm. The device includes: a first display module, used to display a labeling window of the target re-judgment model in response to a labeling instruction input by a user for the target re-judgment model; a second display module, used to display the sample defect detection results of a sample product in the labeling window, the sample defect detection results including at least one defect record, each defect record corresponding to a defect location detected on the sample product by the preset defect detection algorithm; and a determination module, used to determine the category label corresponding to each of the at least some defect records in response to a labeling operation by a user for at least some defect records in the sample defect detection results, and save the at least some defect records and the corresponding category labels as training data in a sample library, the training data being used to train the re-judgment model.

[0018] According to another aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the model management method of the above-described re-judgment model.

[0019] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, wherein the program instructions are used to execute the model management method of the above-described re-judgment model when running.

[0020] According to embodiments of the present invention, the model management method, apparatus, electronic device, and storage medium for the reassessment model can display an annotation window of the target reassessment model in response to a user's annotation command. The user can view the sample defect detection results of the sample product in the annotation window and annotate at least some of the defect records. The user-annotated category labels are saved as new category labels for the reassessment model in the sample library for training the reassessment model. This model management scheme allows users to manage the reassessment model themselves and customize the category labels. Thus, when the existing category labels of the reassessment model cannot meet the user's classification needs, the user can expand the category labels themselves without having to return to the reassessment model designers for adjustments. In this way, when the same reassessment model is applied in different scenarios, users in different scenarios can adaptively adjust the category labels of the reassessment model according to their own needs, which can greatly improve the management autonomy of the reassessment model, expand the application scenarios of the reassessment model, reduce the development workload and cost of the reassessment model, and improve the user experience. Attached Figure Description

[0021] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0022] Figure 1 A schematic flowchart illustrating a model management method for a review model according to an embodiment of the present invention is shown.

[0023] Figure 2 A schematic diagram of a labeling window according to an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram of a label editing window according to an embodiment of the present invention is shown;

[0025] Figure 4 A schematic diagram of a defect selection window according to an embodiment of the present invention is shown;

[0026] Figure 5 and Figure 6 A schematic diagram of a model selection window in different display states according to an embodiment of the present invention is shown;

[0027] Figure 7 A schematic diagram illustrating a model management page according to an embodiment of the present invention is shown;

[0028] Figure 8 A schematic block diagram of a model management device for a review model according to an embodiment of the present invention is shown; and

[0029] Figure 9 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0031] In existing technologies, once a review model is designed, it can be deployed in the product manufacturing site's inspection system or in the cloud. The local inspection system can then download the model directly or from the cloud and use it to review the defect detection results of its products. The inspection system can acquire product images, perform defect detection using a preset defect detection algorithm, obtain the defect detection results, and then use the review model to review those results. However, in existing technologies, the product review model cannot be managed or controlled by the users (e.g., product manufacturers). When the existing category labels of the review model cannot meet the user's classification needs, the model designer must be contacted for adjustments. Since product production data is typically important and extensive, it cannot be transferred out of the production site for model training. Therefore, adjusting the review model is a complex, time-consuming, and costly task for both the model's users and designers.

[0032] To at least partially address the aforementioned problems, embodiments of the present invention provide a model management method for review and judgment models. This method allows users to manage review and judgment models themselves and customize their category labels. Thus, when existing category labels in the review and judgment model cannot meet the user's classification needs, the user can expand the category labels independently without needing to return the review and judgment model to the designers for adjustments. In this way, when the same review and judgment model is applied in different scenarios, users in different scenarios can adaptively adjust the category labels of the review and judgment model according to their own needs, which can greatly improve the management autonomy of the review and judgment model, expand the application scenarios of the review and judgment model, reduce the development workload and cost of the review and judgment model, and improve the user experience.

[0033] Figure 1 This document illustrates a schematic flowchart of a model management method 100 for a review model according to an embodiment of the present invention. The model management method 100 for the review model described herein can be applied to any electronic device with data processing capabilities and / or instruction execution capabilities, i.e., it is executed by an electronic device. This electronic device may include, but is not limited to, personal computers, servers, mobile terminals, etc. Exemplarily and not limitingly, the electronic device may include, for example, a testing system deployed at a product manufacturing site, i.e., the model management method 100 for the review model is executed by the testing system deployed at the product manufacturing site. The review model described herein is used to review the defect detection results of a product, the defect detection results being obtained through a preset defect detection algorithm. The preset defect detection algorithm may include at least one defect detection algorithm to detect at least one defect in the product. The preset defect detection algorithm described herein can be any type of algorithm, including but not limited to deep learning models and / or traditional image processing algorithms, etc. Figure 1 As shown, the model management method 100 includes steps S110, S120 and S130.

[0034] Step S110: In response to the user's input of a labeling instruction for the target reassessment model, the labeling window of the target reassessment model is displayed.

[0035] The electronic device used in the model management method 100 for executing the re-judgment model may include an input device or be communicatively connected to an input device. The communication connection described herein may include any wired and / or wireless connection. The input device may include, but is not limited to, one or more of a mouse, keyboard, touchpad, touchscreen, trackball, microphone, etc. Users can perform various operations through the input device, i.e., input various instructions and / or information. That is, the various user instructions described herein can be input in any way, including but not limited to input via triggering annotation controls on the user interface (UI), input via shortcut keys, input via voice, etc. It is understood that the aforementioned user interface is an interactive interface capable of displaying information; the model management page and various windows (such as annotation windows) described herein can all be displayed on the user interface. Similarly, the various user operations described herein can also be implemented in any operation mode; when a user performs any operation, they input the corresponding instruction (or operation instruction). That is, the operations performed by the user may include, but are not limited to, triggering annotation controls on the user interface (UI), triggering shortcut keys, providing voice commands, etc.

[0036] When a user inputs a labeling instruction for any target reassessment model, the labeling window of the target reassessment model can be displayed on the user interface. Figure 2 A schematic diagram of an annotation window according to an embodiment of the present invention is shown. Exemplarily, the upper right corner of the annotation window may display one or more of a maximize control, a minimize control, and a close control (these controls are not shown in the figure), which are used to control the maximization, minimization, and closing of the annotation window, respectively. Similarly, other windows and pages described herein, such as label editing windows and model management pages, may also have one or more of a maximize control, a minimize control, and a close control. The setting methods and functions of various controls can be understood with reference to the model management page, and will not be elaborated further herein. An electronic device for performing the model management method 100 for re-judging models may include an output device or be communicatively connected to an output device. The output device may include any type of display device, which may be touch-sensitive or non-touch-sensitive. Any content that needs to be displayed can be displayed through the display device, such as the annotation window of the target re-judgment model.

[0037] Step S120: Display the sample defect detection results of the sample product in the annotation window. The sample defect detection results include at least one defect record, and each defect record corresponds to a defect location detected on the sample product by a preset defect detection algorithm.

[0038] As mentioned above, the sample products can be any type of product requiring defect detection, including but not limited to wafers, lithium battery electrodes, etc. The number of sample products can be one or more, meaning that the defect records of any one or more sample products can be displayed in the annotation window. For example... Figure 2 As shown, the sample defect detection results of the sample product are displayed in the left area of ​​the annotation window, specifically in area 210 marked with a dashed box. It should be understood that part of the content displayed in area 210 is obscured by the product image above; therefore, in... Figure 2 It is not visible in the middle, but the displayed content is visible when the product image is not displayed or is displayed in a smaller size. Figure 2The sample product shown is a wafer, and each defect record corresponds to a defect location on a die or chip on the wafer. Each sample product can include one or more detection areas (or sub-regions, represented by Zones). Any two detection areas of the same product can partially overlap or not overlap at all. For each detection area, one or more defect detection algorithms can be used for detection. Each defect detection algorithm can detect one type of defect and obtain a corresponding defect location. Therefore, the defect detection result of each detection area can include one or more defect locations. Each defect location can specifically include one or more non-contiguous image regions. These image regions are determined by the preset defect detection algorithm to contain the defects that the algorithm is trying to detect, such as particles, scratches, pits, etc. The re-judgment model has one or more category labels, which can include, but are not limited to, normal (ok), scratch, burr, etc. The category labels used by the re-judgment model to classify defects can be partially the same and partially different from the category labels used by the preset defect detection algorithm to classify defects, or they can be completely the same or completely different. The category labels of the review model can be set as needed. For example, a defect classified as a scratch by the preset defect detection algorithm may be negligible for some products, in which case the review model can classify it as normal. However, for some products, the impact may be significant, and the review model can classify it as a minor scratch. When reviewing the defect detection results, the review model can classify each defect record according to its existing category labels. In essence, during classification, the review model can determine the score for each defect location in each defect record, representing the probability that the defect record belongs to a preset category label. If the score for any defect location belonging to any category label exceeds the review model's score threshold, the review model can determine that the defect location belongs to the category indicated by that category label. Furthermore, by way of example and not limitation, the re-judgment model can also compare the image area occupied by the defect location with an area threshold. When the image area occupied by the defect location exceeds the area threshold, the model determines that the defect location belongs to the category indicated by the preset defect category label used by the re-judgment model (in this case, the re-judgment model can be used to detect a single defect category, such as a scratch). Furthermore, by way of example and not limitation, the re-judgment model can compare the score corresponding to the defect location with a score threshold and compare the image area occupied by the defect location with an area threshold. When the score of any defect location belonging to any category label exceeds the score threshold of the re-judgment model, and the image area occupied by the defect location exceeds the area threshold, the model determines that the defect location belongs to the category indicated by that category label.

[0039] like Figure 2As shown, the sample defect detection results include multiple defect records, each defect record corresponding to a single type of defect in a single sub-region of a single die.

[0040] Step S130: In response to the user's annotation operation on at least some defect records in the sample defect detection results, determine the category labels corresponding to each of the at least some defect records, and save the at least some defect records and their corresponding category labels as training data in the sample library. The training data is used to train the re-judgment model.

[0041] Users can annotate at least some defect records in the sample defect detection results. That is, users can annotate all or only some of the defect records in the sample defect detection results. Defect records can contain location information of the defect and defect category information detected by a preset defect detection algorithm (which can be represented by category labels provided by the preset defect detection algorithm). User annotation of defect records is equivalent to annotating the category label used by the re-judgment model when classifying the defect record. After user annotation, the user-annotated category labels can be associated with the corresponding defect records and saved in the sample library as training data. The re-judgment model can then be trained using the training data in the sample library, enabling it to be extended to recognize user-defined category labels. This extension method allows users to conveniently manage and control the training and learning of their private local re-judgment model and enables rapid iteration and deployment of new models.

[0042] By employing the above technical solution, the annotation window of the target review model can be displayed in response to user annotation commands. Users can view the sample defect detection results of the sample products in the annotation window and annotate at least some of the defect records. The user-annotated category labels are saved as new category labels for the review model in the sample library for training. This model management scheme allows users to manage the review model themselves and customize its category labels. Thus, when the existing category labels of the review model cannot meet the user's classification needs, the user can expand the category labels themselves without having to return to the review model designers for adjustments. In this way, when the same review model is applied in different scenarios, users in different scenarios can adaptively adjust the category labels of the review model according to their own needs, which can greatly improve the management autonomy of the review model, expand the application scenarios of the review model, reduce the development workload and cost of the review model, and improve the user experience.

[0043] According to an embodiment of the present invention, the annotation window includes a first display area for displaying label information items of category labels. Each label information item is associated with a unique category label, and the label information item includes the label name of the associated category label and the operation instruction corresponding to the category label. In response to a user's annotation operation on at least some defect records in the sample defect detection results, determining the category label corresponding to each of the at least some defect records (step S130) includes: when the operation instruction input by the user for any defect record is consistent with the operation instruction corresponding to any category label, determining that the category label corresponding to the defect record is the category label corresponding to the operation instruction. The annotation operation includes the user inputting the operation instruction.

[0044] See Figure 2 In the upper right corner of the labeling window, a dashed box marks the first display area 220, which is labeled "Label Category". This area displays the label information items for each category label. For example, if the user has not imported or created any category labels, the first display area may only display the label information items corresponding to "Unlabeled". After the user imports and / or creates (i.e., adds) new category labels, the first display area may display the label information items corresponding to these imported and / or created category labels. It should be noted that the positions of the various display areas, pages, windows, and controls described herein can be arbitrarily set as needed and are not limited to the location shown in the accompanying drawings.

[0045] The label information field can include the label name of the associated category label and the corresponding operation instruction. As mentioned above, the operation instruction can include any type of instruction, including but not limited to control trigger instructions, shortcut key input instructions, and voice input instructions. For example, different shortcut keys can be set for different category labels, and the names of the shortcut keys can be displayed in the label information field, such as... Figure 2 The shortcut key "Back" is shown. When the user enters an operation command corresponding to any category label, it indicates that the current defect record will be marked with that category label.

[0046] For example, the label information items in the first display area are editable information items, and the method further includes one or more of the following operations: in response to a user's import operation, importing label information items from a preset import location to display in the first display area; in response to a user's add operation, adding a new label information item in the first display area; in response to a user's delete operation on any label information item, deleting the label information item from the first display area; in response to a user's edit operation on any label information item, editing at least some of the information contained in the label information item.

[0047] Each label information item in the first display area can be an editable information item. For example, a user can import label information items from a preset import location to the first display area via an import operation. The preset import location can be any location storing label information items of preset category labels, such as a preset memory, external device, network, or cloud. For example, a user can also add a new label information item to the first display area via an add operation. Optionally, after performing the add operation, a new label information item can be directly added to the first display area. The information in this label information item can be blank or default information, and the user can set the information in the newly added label information item. Alternatively, after performing the add operation, a label editing window (described below) can pop up. After the user sets the information in the label information item in the label editing window, the newly added label information item is displayed in the first display area. For example, a user can also delete any label information item via a delete operation. For example, a user can also set at least some of the information contained in a label information item via an edit operation, such as setting one or more of the following: label name, label color, operation instructions, etc. for category labels. Each of the above import, add, delete, and edit operations can be a command input operation of any form, including but not limited to one or more of the following: shortcut key input, control triggering, and voice input. For example, in Figure 2 In the first display area, a "Add" button control is shown. After the user clicks the "Add" button control, a new label information item will be added and displayed in the first display area. The label information items displayed in the first display area can be regarded as a "dictionary", which allows users to quickly find the corresponding operation instructions for each category of label so as to make correct labeling based on the operation instructions.

[0048] By employing the above technical solution, displaying label information items in the first display area allows users to easily understand the corresponding operation instructions for each category of label, guiding them to label quickly and accurately. Furthermore, the label information items are editable; users can edit them in the first display area as needed, adjusting the types and number of category labels to better customize different category labels.

[0049] According to an embodiment of the present invention, the first display area displays one or more of the following controls: an import control, an add control, a delete control, and an edit control. The import operation includes a trigger operation on the import control, the add operation includes a trigger operation on the add control, the delete operation includes a trigger operation on the delete control, and the edit operation includes a trigger operation on the edit control.

[0050] As mentioned above, each of the import, add, delete, and edit operations can be a command input operation of any form, including but not limited to one or more of shortcut key input, control triggering, and voice input. By displaying interactive controls, users can easily perform the import, add, delete, and edit operations by triggering these controls. This eliminates the need for users to memorize the corresponding operation methods, helping to improve the accuracy of their input.

[0051] According to an embodiment of the present invention, the label information item also includes a method of highlighting the defect location on the product image of the sample product when the defect location has a category label associated with the label information item.

[0052] Highlighting methods can be any method that displays the defect location in a recognizable way on the product image, including but not limited to using a preset color to highlight the entire defect location with a mask that distinguishes it from other image areas, or using a preset color to highlight the outline of the defect location with a mask that distinguishes it from other image areas. For example, when the defect location is classified as a first-category label, it can be displayed entirely with a red mask on the product image; when the defect location is classified as a second-category label, it can be displayed entirely with a green mask on the product image, and so on. It should be noted that the terms "first," "second," and "third," etc., used herein are for distinguishing purposes only and have no other special meaning.

[0053] By highlighting certain defects with different category labels on the product image, the defect locations can be shown in the label information section, which helps users better understand how to label categories.

[0054] According to an embodiment of the present invention, the first display area displays one or more of the following controls: an import control, an add control, a delete control, and an edit control. The import operation includes a trigger operation on the import control, the add operation includes a trigger operation on the add control, the delete operation includes a trigger operation on the delete control, and the edit operation includes a trigger operation on the edit control. Furthermore, the label information item also includes a method for highlighting the defect location on the product image of the sample product when the defect location has a category label associated with the label information item.

[0055] The implementation method and technical effects of this embodiment can be understood by referring to the embodiments described above, and will not be repeated here.

[0056] According to an embodiment of the present invention, in response to a user's editing operation on any label information item, editing at least a portion of the information contained in the label information item includes: in response to the user's editing operation on any label information item, displaying a label editing window; in response to setting information entered by the user in the label editing window, determining at least a portion of the information contained in the label information item, wherein the remaining information of the label information item is set to one of the default information or randomly set in a manner distinguishable from other category labels.

[0057] For example, an edit button control can be displayed in each label information item in the first display area. The user's action of triggering (i.e., clicking) the edit button control constitutes an editing operation. In response to the editing operation, a corresponding label editing window can pop up. In the label editing window, the user can set at least some of the information in the label information item. The remaining unset information can be set to one of the default information or randomly selected in a way that distinguishes it from other category labels. For example, if the user does not set a label color, the system can automatically set the label color of that category label to one of the default colors or randomly selected, as long as it can be distinguished from other category labels.

[0058] Figure 3 A schematic diagram of a label editing window according to an embodiment of the present invention is shown. Figure 3 The label editing window shown is labeled "Edit Label Category". In this window, you can set the label color, label name, and shortcut key. Label names can be selected from a preset label library (i.e., "Select from Library") or defined by the user (i.e., "Customize"). Figure 3 In the illustrated embodiment, when the user selects the "Select from Library" button control, a selection control, such as a drop-down menu control, can be displayed in the label editing window to select any preset label name from the preset label library as the label name for the current category. Figure 3 In the illustrated embodiment, when the user selects the "Custom" button control, a text box can be displayed in the label editing window. Figure 3 The text box shown reads "Create New Category," and users can directly enter the label name for the category label in the text box.

[0059] By adopting the above technical solution, users can edit and define the label information items of each category of labels through the label editing window, which can further improve the interactivity of model management and enhance the user experience.

[0060] According to an embodiment of the present invention, before displaying the sample defect detection results of the sample product in the annotation window, the method further includes: responding to a defect selection instruction input by a user, searching for matching defect records from a preset database according to the search conditions indicated by the defect selection instruction to obtain initial defect detection results. The preset database stores defect detection results corresponding to at least one batch of products processed by at least one machine. The search conditions include one or more of the following: machine identification information, product batch identification information, product identification information, and product image scanning time; determining the initial defect detection results as sample defect detection results, or, responding to a user's selection operation for at least some defect records in the initial defect detection results, determining the defect records selected by the user as sample defect detection results.

[0061] Product images have their own scanning times. Within a certain product image scanning time, image scanning and defect detection can be performed on one or more batches of products processed by one or more machines. A single batch of products can contain one or more products. After defect detection is performed on the products using a preset defect detection algorithm, the obtained defect detection results can be stored in a preset database. Therefore, the preset database can store the defect detection results corresponding to at least one batch of products processed by at least one machine. Users can search for defect records to be marked from the preset database by inputting defect selection instructions. The defect selection instructions include information about the search criteria, which can include one or more of the following: machine identification information, product batch identification information, product identification information, and product image scanning time. Machine identification information is used to identify different machines, product batch identification information is used to identify different product batches, and product identification information is used to identify different products. It is understood that the above three types of identification information can be any form of identification information, including but not limited to numbers, names, etc.

[0062] The initial defect detection results can include all defect records for one or more products. After retrieving the initial defect detection results, all of the initial defect detection results can be identified as sample defect detection results, or, in response to a user's selection of at least some defect records in the initial defect detection results, the user-selected defect records can be identified as sample defect detection results. It can be understood that the product to which the defect records in the final identified sample defect detection results belong is the sample product.

[0063] The above technical solution allows users to search for defect detection results according to search criteria, making it convenient for users to quickly select the defect records to be labeled from the preset database as needed, which can effectively improve the efficiency of defect labeling.

[0064] According to an embodiment of the present invention, before searching for matching defect records from a preset database according to the search conditions indicated by the defect selection instruction in response to a user input, in order to obtain an initial defect detection result, the method further includes: displaying a defect selection control in a labeling window; displaying a defect selection window in response to a user triggering operation on the defect selection control, and providing an input area for search conditions in the defect selection window; wherein the defect selection instruction includes input information in the input area for one or more search conditions.

[0065] See back Figure 2 In area 210 of the annotation window, there is an "Add Picture" button control, which is the defect selection control. When the user triggers (i.e. clicks) the "Add Picture" button control, the defect selection window will pop up. Figure 4 A schematic diagram of a defect selection window according to an embodiment of the present invention is shown. Figure 4 As shown, the defect selection window is labeled "Add Picture" window. In this window, there is an input area 410 for providing search criteria, which is indicated by a dashed box. Figure 4 In this embodiment, the product is a wafer. Figure 4 The search criteria shown include product batch identification information (Lotid), product identification information (Waferid), and scan time. Users can enter the corresponding information as search criteria in any one or more of the text boxes. Furthermore, Figure 4 The defect selection window also displays a "Search" button control. When the user triggers (i.e. clicks) this button control, it can search for matching defect records from the preset database based on the user's current search criteria to obtain initial defect detection results.

[0066] By adopting the above technical solution, users can interact with the system through a visual defect selection window, allowing them to quickly and accurately set search criteria to search for defect records.

[0067] According to an embodiment of the present invention, the defect selection window includes a second display area and a third display area. In response to a user's selection operation on at least a portion of the defect records in the initial defect detection results, the user-selected defect records are determined as sample defect detection results. This includes: displaying product information items corresponding to each product in the initial defect detection results in the second display area, the product information items including product identification information; displaying the defect records of the product corresponding to the product information item in the third display area in response to a user's selection operation on any product information item; adding the user-selected defect records to the temporary defect detection results in response to a user-input completion command; and determining the current temporary defect detection results as sample defect detection results in response to a user-input completion command.

[0068] See Figure 4 In the defect selection window, the second display area 420 and the third display area 430 are marked with dashed boxes. The second display area displays product information items corresponding to each product in the initial defect detection results. Figure 4 The search results contain two product information items. Each product information item includes at least product identification information so that the user can identify the searched product. When the user selects either product information item, the defect records of the product corresponding to the currently selected product information item are displayed in the third display area of ​​the defect selection window. It should be noted that the "AI Classification" column in the third display area shows the classification results from previous AI model reviews, and is unrelated to the annotation and training of the current target review model. In one embodiment, all defect records of the current product can be selected in response to a user's input of a select all command. The operation of inputting a select all command is the selection operation. For example, a select all control can be provided in the defect selection window, such as... Figure 4 In the third display area, the "Select All" button control to the right of "Select Defect Image" adds all defect records for the current product to the temporary defect detection results when triggered by the user. Additionally, a separate selection control can be displayed for each defect record. Figure 4 The options are shown as checkboxes. A separate selection control allows the user to select one or more defect records from the current product's defect records. The user-selected defect records can be added to the temporary defect detection results. The defect selection window allows the user to select defect records across products (i.e., across product images or across defect pictures). That is, after selecting any product in the second display area and at least some of the defect records for that product in the third display area, the user can switch to selecting another product in the second display area and at least some of the defect records for that other product in the third display area. This operation can be repeated, and the user can select any number of defect records for any number of products. Furthermore, "across products" can refer to products from the same batch or products from different batches. For example, the defect selection window can also display a save control (such as...). Figure 4 The "Save Current Selection" control shown can add the currently selected defect record to the temporary defect detection results for saving in response to the user's trigger operation on the save control.

[0069] Users can enter completion commands at any time, and these commands can be entered via keyboard shortcuts, controls, voice, etc. For example... Figure 4 As shown, in the defect selection window, a "Add Complete" button control can be displayed. When the user triggers (i.e. clicks) this button control, the selection is completed, that is, the current temporary defect detection result is confirmed to be the above sample defect detection result.

[0070] By employing the above technical solution, different display areas within the defect selection window can respond to the user's selection of product information items, displaying the corresponding product's defect records in real time. This facilitates switching between different products and selecting defect records. The defect records selected by the user are added to the temporary defect detection results and saved until a completion command is received from the user, confirming the selection and obtaining the sample defect detection results. This solution allows users to easily adjust the selected defect records at any time, providing a convenient and flexible method for defect record selection.

[0071] According to an embodiment of the present invention, the defect selection window displays a completion control, and the completion instruction includes the operation instruction entered by the user when performing a trigger operation on the completion control.

[0072] As mentioned above, a completion control can be displayed in the defect selection window so that users can input completion commands. This control input scheme is highly interactive, reduces the burden on users to remember shortcut keys and other operation methods, and helps to further improve the user experience.

[0073] According to an embodiment of the present invention, the annotation window further includes a fourth display area, and the method further includes: in response to a user's selection operation on any defect record in the sample defect detection results, displaying the annotation information of the defect record in the fourth display area, the annotation information including the label name of the corresponding category label when it is not annotated and when it is annotated.

[0074] Return to reference Figure 2 The lower right corner of the annotation window displays an "Annotation Label" area, which is the fourth display area 230, marked with a dashed box. This fourth display area is used to display the annotation information for defect records. When a user selects any defect record, its annotation information is displayed in real-time in this fourth display area. If the currently selected defect record is not yet annotated, the displayed annotation information may include "Unannotated" information. If the currently selected defect record is already annotated, the displayed annotation information may include the label name of the category label for that defect record. Optionally, when a defect record is already annotated, the corresponding annotation information may include at least part of the label information item of the category label for that defect record. For example, in addition to the category label, the annotation information may also include the annotation color corresponding to the category label and / or the operation instructions corresponding to the category label.

[0075] By adopting the above technical solution, the annotation information of the currently selected defect record can be displayed in real time as the user selects the defect record, so that the user can understand the annotation status of each defect record in detail.

[0076] For example, the annotation window may also include an image display area (which may be referred to as a fifth display area) for displaying defect images, such as... Figure 2As shown, a product image can be displayed in the middle area of ​​the annotation window. Exemplarily, the method may further include: in response to a user's selection of any defect record in the sample defect detection results, displaying a defect image including the defect location in the defect record in a fifth display area. The defect image may be the entire product image including the defect location in the user-selected defect record, i.e., the entire product image of the sample product to which the defect location belongs, or it may be an image area including the defect location in the user-selected defect record, which is a portion of the entire product image of the sample product to which the defect location belongs. Displaying an image area instead of the entire product image allows the user to view the defect location information more clearly. Exemplarily, when displaying the defect image, the defect location of the currently selected defect record can be highlighted on the defect image.

[0077] According to an embodiment of the present invention, the method further includes: responding to a training instruction input by a user, displaying a model selection window, the model selection window including a first selection control and a second selection control; responding to a user's trigger operation on the first selection control, activating a third selection control of a pre-trained model; responding to a user's selection operation on the pre-trained model through the third selection control, using the user-selected pre-trained model as the target re-judgment model, and training the target re-judgment model based on training data in the sample library; responding to a user's trigger operation on the second selection control, using a preset initial model as the target re-judgment model, and training the target re-judgment model based on training data in the sample library.

[0078] Similar to other operation commands, training commands can be input via keyboard shortcuts, control triggers, voice, etc. For example, the annotation window can also display training controls (such as...). Figure 2 The "Training" button control shown in the upper right corner can be used to generate training instructions, which may include the corresponding input commands when the training control is triggered. Figure 5 and Figure 6 This diagram illustrates a model selection window in different display states according to an embodiment of the present invention. Figure 5 and 6 In the text, the model selection window is labeled "Select Pre-trained Model". For example... Figure 5 and 6 As shown, the model selection window can display the model name and model notes of the re-judgment model. The model notes can be used to describe the detection purpose of the re-judgment model (e.g., detecting "scratches"). For example, the model selection window can display a first selection control (such as...). Figure 5 and 6 The "Select Pre-trained Model" control in the middle, Figure 5This shows the display state of the model selection window when the user triggers the first selection control. In response to the user's triggering of the first selection control, a third selection control of the pre-trained model can be activated, such as... Figure 5 The "Project Assigned" dropdown menu and "Search" button controls shown allow users to select a pre-trained model from a preset model library via a third selection control. The preset model library can store one or more different versions of pre-trained models. For any two different versions of pre-trained models, they can be trained based on the same initial re-judgment model, or one can be iteratively trained based on the other. Alternatively, these two different versions can be trained based on different initial re-judgment models. For example, for an initial re-judgment model V1, three versions V2, V3, and V4 have been iteratively trained sequentially. The preset model library can include these four different versions of re-judgment models, and the user can select version V3 as the target re-judgment model for training. When the user selects any pre-trained model via the third selection control, the selected pre-trained model can be used as the target re-judgment model, and training can be performed on the target re-judgment model based on training data in the sample library. Optionally, training the target re-judgment model based on training data in the sample library can be executed in response to a user-inputted training start command. Figure 5 and 6 As shown, the model selection window can display "OK and Start" and "Abandon Training" buttons. When the user triggers (i.e. clicks) the "OK and Start" button, the target re-judgment model can be trained based on the training data in the sample library. When the user triggers (i.e. clicks) the "Abandon Training" button, the operation of training the target re-judgment model based on the training data in the sample library will no longer be performed.

[0079] For example, the model selection window may display a second selection control (such as...) Figure 5 and 6 The "Start Training Now" control in the middle, Figure 6 This shows the display state of the model selection window when the user triggers the second selection control. In response to the user's triggering of the second selection control, a preset initial model can be directly used as the target re-judgment model, and the target re-judgment model can be trained based on training data in the sample library. Similarly, in this case, training the target re-judgment model based on training data in the sample library can be executed in response to the user's input of a training start command. For example... Figure 5 and 6As shown, the model selection window can display "OK and Start" and "Abandon Training" buttons. When the user triggers (i.e. clicks) the "OK and Start" button, the target re-judgment model can be trained based on the training data in the sample library. When the user triggers (i.e. clicks) the "Abandon Training" button, the operation of training the target re-judgment model based on the training data in the sample library will no longer be performed.

[0080] By employing the above technical solution, a model selection window is provided, allowing users to conveniently choose a pre-trained model or a preset initial model as the target re-judgment model for training as needed. This solution provides users with greater freedom in model selection, facilitating more flexible training and management of the re-judgment model.

[0081] According to an embodiment of the present invention, before displaying the annotation window of the target re-judgment model in response to a user-input annotation instruction for the target re-judgment model, the method further includes: displaying a model management page, the model management page displaying model project information items, each model project information item being associated with a unique model project, the model project representing a project for a single annotation and training of a single re-judgment model, the model project information item including one or more of the following information: model project name, annotation status, creation time, update time, a first control, a second control, and a third control; the first control is used to display the annotation window in response to a user's trigger operation, the annotation instruction including the operation instruction input by the user when performing a trigger operation on the first control, the second control is used to set the model project information item to an editable state in response to a user's trigger operation, so that the user can edit the model project name of the model project information item, and the third control is used to delete the corresponding model project information item and the model project associated with the model project information item in response to a user's trigger operation.

[0082] The model management page can be displayed automatically when preset conditions are met, or it can be displayed in response to a user opening the page. For example, preset conditions may include, but are not limited to, powering on, completion of product image acquisition, etc. See also Figure 7 This diagram illustrates a model management page according to an embodiment of the present invention. Figure 7As shown, the model management page can display model project information items, each associated with a unique model project. For example, different model project information items can be displayed in rows on the model management page, meaning each model project information item is displayed as one row. A model project represents a project for a single annotation and training run on a single re-judgment model. One or more model projects can be created for the same re-judgment model. In different model projects, the same or different category labels can be used to annotate and train the re-judgment model. Of course, any two different model projects can also be projects for annotating and training different re-judgment models. Model project information items can include one or more of the following information: model project name, annotation status, creation time, update time, first control, second control, and third control. Figure 7 As shown, the model project information items can include the model project name, status (i.e., annotation status), creation time, and update time. The annotation status indicates the annotation status of the model in the current model project, and can include states such as "annotating in progress," "annotated," and "not annotated." Figure 7 As shown, the model engineering information item may also include a first control (i.e., the "Annotation / Training" control), which, when triggered by the user, displays the aforementioned annotation window. For example... Figure 7 As shown, the model project information item may also include a second control (i.e., an "Edit Name" control). When the user triggers this control, it can be set to an editable state in response to the user's triggering operation, allowing the user to edit the model project name of the model project information item. Figure 7 As shown, the model project information item may also include a third control (i.e., a "delete" control). When the user triggers this control, they can delete the corresponding model project information item and the model project associated with it. For example, the model management page may also display a model creation control, i.e. Figure 7 The "New" control shown responds to the user's triggering operation on the model creation control, allowing the addition of a new model project and corresponding model project information items.

[0083] By adopting the above technical solution and providing a model management page, users can easily manage each model project, which allows them to label and / or train the re-judgment model through different model projects.

[0084] According to another aspect of the present invention, a model management device for a re-judgment model is provided. The re-judgment model is used to re-judge the defect detection results of a product. The defect detection results are obtained by detecting through a preset defect detection algorithm. Figure 8 A schematic block diagram of a model management device 800 for reviewing a model according to an embodiment of the present invention is shown. Figure 8As shown, the device 800 may include a first display module 810, a second display module 820, and a determination module 830.

[0085] The first display module 810 is used to display the annotation window of the target reassessment model in response to the annotation command input by the user for the target reassessment model.

[0086] The second display module 820 is used to display the sample defect detection results of the sample product in the annotation window. The sample defect detection results include at least one defect record, and each defect record corresponds to a defect location detected on the sample product by a preset defect detection algorithm.

[0087] The determination module 830 is used to respond to the user's annotation operation on at least some defect records in the sample defect detection results, determine the category label corresponding to each of the at least some defect records, and save the at least some defect records and their corresponding category labels as training data in the sample library. The training data is used to train the re-judgment model.

[0088] According to another aspect of the present invention, an electronic device is also provided. Figure 9 A schematic block diagram of an electronic device 900 according to an embodiment of the present invention is shown, such as Figure 9 As shown, the electronic device 900 may include a processor 910 and a memory 920. The memory 920 stores a computer program, and the processor 910 executes the computer program to implement the aforementioned model management method for the re-evaluation model.

[0089] According to another aspect of the present invention, a storage medium is also provided. This medium stores a computer program / instructions, which, when executed by a processor, implement the model management method for the aforementioned review model. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0090] Those skilled in the art can understand the specific implementation schemes and beneficial effects of the above-mentioned model management device, electronic device and storage medium for the review model by reading the relevant description of the model management method for the review model. For the sake of brevity, they will not be described in detail here.

[0091] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0092] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0094] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0095] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0096] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0097] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0098] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the model management device for the re-judgment model according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0099] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0100] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A model management method for a re-judgment model, characterized in that, The re-judgment model is used to re-judge the defect detection results of the product. The defect detection results are obtained through a preset defect detection algorithm. The method includes: In response to a user-input annotation command for the target reassessment model, the annotation window of the target reassessment model is displayed; The labeling window displays the sample product's defect detection results, which include at least one defect record. Each defect record corresponds to a defect location detected on the sample product by the preset defect detection algorithm. In response to the user's annotation operation on at least some of the defect records in the sample defect detection results, the category label corresponding to each of the at least some defect records is determined, and the at least some defect records and their corresponding category labels are saved as training data in the sample library. The training data is used to train the re-judgment model.

2. The method according to claim 1, characterized in that, The labeling window includes a first display area, which is used to display label information items of category labels. Each label information item is associated with a unique category label. The label information item includes the label name of the associated category label and the operation instruction corresponding to the category label. The step of responding to a user's annotation operation on at least some defect records in the sample defect detection results, and determining the category label corresponding to each of the at least some defect records, includes: When the operation command input by the user for any defect record is consistent with the operation command corresponding to any category label, the category label corresponding to the defect record is determined to be the category label corresponding to the operation command. The labeling operation includes the operation of the user inputting the operation command. Wherein, the label information items in the first display area are editable information items, and the method further includes one or more of the following operations: In response to the user's import operation, the tag information items are imported from the preset import location and displayed in the first display area; In response to the user's addition operation, a new label information item is added to the first display area; In response to a user's deletion operation on any tag information item, the tag information item is deleted from the first display area; In response to a user's edit operation on any tag information item, at least some of the information contained in that tag information item is edited.

3. The method according to claim 2, characterized in that, The first display area displays one or more of the following controls: an import control, an add control, a delete control, and an edit control. The import operation includes triggering the import control; the add operation includes triggering the add control; the delete operation includes triggering the delete control; and the edit operation includes triggering the edit control; and / or, The label information item also includes a method for highlighting the defect location on the product image of the sample product when the category label associated with the label information item is present.

4. The method according to claim 2 or 3, characterized in that, The action of responding to a user's edit operation on any tag information item, and editing at least a portion of the information contained in that tag information item, includes: In response to a user's editing action on any tag information item, a tag editing window is displayed; In response to the setting information entered by the user in the label editing window, at least some of the information contained in the label information item is determined, and the remaining information of the label information item is set to one of the default information or randomly set in a way that is distinguishable from other category labels.

5. The method according to any one of claims 1-3, characterized in that, Before displaying the sample defect detection results of the sample product in the annotation window, the method further includes: In response to a defect selection instruction input by the user, a matching defect record is searched from a preset database according to the search conditions indicated by the defect selection instruction to obtain an initial defect detection result. The preset database stores the defect detection results corresponding to at least one batch of products processed by at least one machine. The search conditions include one or more of the following: machine identification information, product batch identification information, product identification information, and product image scanning time. The initial defect detection result is determined as the sample defect detection result, or, in response to the user's selection operation for at least a portion of the defect records in the initial defect detection result, the defect record selected by the user is determined as the sample defect detection result.

6. The method according to claim 5, characterized in that, Before responding to a user-inputted defect selection instruction and searching for matching defect records from a preset database according to the search criteria indicated by the defect selection instruction to obtain initial defect detection results, the method further includes: The defect selection control is displayed in the annotation window; In response to a user's triggering action on the defect selection control, a defect selection window is displayed, and an input area for the search criteria is provided in the defect selection window; The defect selection instruction includes input information in the input area for one or more search criteria.

7. The method according to claim 6, characterized in that, The defect selection window includes a second display area and a third display area. The step of responding to a user's selection operation for at least a portion of the defect records in the initial defect detection results, and determining the user-selected defect record as the sample defect detection result, includes: In the second display area, the product information items corresponding to each product in the initial defect detection results are displayed, and the product information items include product identification information; In response to a user's selection of any product information item, the defect record of the product corresponding to that product information item is displayed in the third display area; In response to the user's selection of at least some defect records for the product corresponding to the product information item, the user-selected defect records are added to the temporary defect detection results; In response to the user's input completion command, the current temporary defect detection result is determined as the sample defect detection result.

8. The method according to claim 7, characterized in that, The defect selection window displays a completion control, and the completion instruction includes the operation instruction entered by the user when performing a trigger operation on the completion control.

9. The method according to any one of claims 1-3, characterized in that, The annotation window also includes a fourth display area, and the method further includes: In response to the user's selection of any defect record in the sample defect detection results, the annotation information of the defect record is displayed in the fourth display area. The annotation information includes the label name of the corresponding category label when it is unannotated and annotated.

10. The method according to any one of claims 1-3, characterized in that, The method further includes: In response to a training instruction input by the user, a model selection window is displayed, the model selection window including a first selection control and a second selection control; In response to the user's triggering operation on the first selection control, the third selection control of the pre-trained model is activated. In response to the user's selection operation on the pre-trained model through the third selection control, the pre-trained model selected by the user is used as the target re-judgment model, and the target re-judgment model is trained based on the training data in the sample library. In response to the user's triggering operation on the second selection control, a preset initial model is used as the target re-judgment model, and the target re-judgment model is trained based on the training data in the sample library.

11. The method according to any one of claims 1-3, characterized in that, Before displaying the annotation window of the target reassessment model in response to a user-input annotation instruction for the target reassessment model, the method further includes: The model management page displays model project information items. Each model project information item is associated with a unique model project, which represents a project for a single annotation and training of a single review model. The model project information item includes one or more of the following information: model project name, annotation status, creation time, update time, first control, second control, and third control. The first control is used to display the annotation window in response to a user's trigger operation. The annotation instruction includes the operation instruction entered by the user when triggering the first control. The second control is used to set the model project information item to an editable state in response to a user's trigger operation, so that the user can edit the model project name of the model project information item. The third control is used to delete the corresponding model project information item and the model project associated with the model project information item in response to a user's trigger operation.

12. A model management device for a re-judgment model, characterized in that, The re-judgment model is used to re-judge the defect detection results of the product. The defect detection results are obtained through a preset defect detection algorithm. The device includes: The first display module is used to display the annotation window of the target re-judgment model in response to the annotation command input by the user for the target re-judgment model; The second display module is used to display the sample defect detection results of the sample product in the annotation window. The sample defect detection results include at least one defect record, and each defect record corresponds to a defect location detected on the sample product by the preset defect detection algorithm. The determination module is used to respond to the user's annotation operation on at least some defect records in the sample defect detection results, determine the category label corresponding to each of the at least some defect records, and save the at least some defect records and their corresponding category labels as training data in the sample library. The training data is used to train the re-judgment model.

13. An electronic device comprising a processor and a memory, wherein, The memory stores computer program instructions, which, when executed by the processor, are used to perform the model management method of the review model as described in any one of claims 1-11.

14. A storage medium on which program instructions are stored, wherein, The program instructions are used to execute the model management method of the re-judgment model as described in any one of claims 1-11 when the program is running.

Images