Method for generating flowchart of defect detection algorithm, defect detection method and system

Abstract

Embodiments of the present application provide a generation method of a flowchart of a defect detection algorithm, a defect detection method and system, an electronic device and a storage medium. The generation method of the flowchart of the defect detection algorithm comprises: a naming step of naming at least one folder based on a preset naming rule, wherein each folder is used to store training sample images of a corresponding model, each model is used to detect target information in a to-be-recognized image, and the name of each folder comprises a model category identifier and / or a target identifier of the corresponding model; and a generation step of generating and displaying a flowchart of a defect detection algorithm based on the model category identifier and / or the target identifier in the name of each folder in response to a confirmation operation of a user, wherein the flowchart shows the running path of the steps of the defect detection algorithm. The above technical solution is simple and easy to implement, has low requirements on users, is suitable for a wide range of users, and provides a better user experience.

Description

Technical Field

[0001] This invention relates to the field of automatic inspection, and more specifically to a method for generating a flowchart of a defect detection algorithm, as well as a defect detection method, system, electronic device, and storage medium. Background Technology

[0002] In recent years, computer vision has been widely used in the field of automatic inspection. For example, computer vision technology is used to detect defects in critical components of electronic devices so that they can be repaired in a timely manner. However, in practical applications, because defects of certain targets may have many different forms, it is impossible to use a general detection algorithm to identify all defects.

[0003] Take wafer defect detection as an example. Because different types of wafers contain different types or quantities of components, the imaged wafers are complex and varied, making it impossible to use a single, universal defect detection algorithm to identify defects on different wafers. Therefore, different defect detection algorithms need to be developed based on the specific wafer.

[0004] Existing technologies typically involve users dragging and dropping modules and connecting them to build defect detection algorithm flows. This requires users to drag and drop different modules sequentially into the flow to assemble a final algorithm flow. However, this approach requires users to understand certain operating rules and possess a certain level of competence and expertise to correctly perform drag-and-drop and connection operations to generate the desired flow, thus having limitations. On the other hand, it also requires users to perform relatively complex operations, resulting in a poor user experience. Summary of the Invention

[0005] The present invention addresses the aforementioned problems. According to one aspect of the invention, a method for generating a flowchart of a defect detection algorithm is provided. The method includes: a naming step, naming at least one folder based on a preset naming rule, wherein each folder stores training sample images of a corresponding model, each model is used to detect target information in the image to be identified, and the name of each folder includes a model category identifier and / or a target identifier of the corresponding model; and a generation step, in response to a user's confirmation operation, generating and displaying a flowchart of the defect detection algorithm based on the model category identifier and / or target identifier in the name of each folder, wherein the flowchart illustrates the execution path of the steps of the defect detection algorithm.

[0006] For example, the folder includes a first type of folder, wherein each first type of folder is used to store training sample images of a corresponding first model, each first model is used to identify defect regions of at least one defect in the image to be identified, target information includes information of the defect region, and target identifier includes a type identifier of the defect identified by the corresponding first model; the generation step includes: in response to the user's confirmation operation, generating and displaying a flowchart of the defect detection algorithm based on the model category identifier and / or target identifier in the name of the first type of folder, the steps of the defect detection algorithm including defect detection steps performed using the first model.

[0007] For example, the folder further includes a second type of folder, wherein each second type of folder is used to store training sample images of the corresponding second model, each second model is used to identify target regions in the image to be identified that are configured with at least one specific defect, the target information includes information about the target region, and the target identifier includes the identifier of the target of the specific defect identified by the corresponding second model; the generation step includes: in response to the confirmation operation, generating and displaying a flowchart based on the model category identifier and / or target identifier in the respective names of the first type of folder and the second type of folder, wherein the steps of the defect detection algorithm further include a target region detection step and a post-processing step performed using the second model, in which the defect detection result of the image to be identified is determined based on the defect region detected by the defect detection step, the target region detected by the target region detection step, and the configuration relationship between the defect region and the target region.

[0008] For example, the name of the second type of folder or the first type of folder also includes a configuration relationship identifier; the generation step includes: in response to the confirmation operation, reading the name of the first type of folder and the name of the second type of folder; based on the configuration relationship identifier in the name of the first type of folder or the second type of folder, obtaining the configuration relationship between the target area of ​​each specific defect and the defect area of ​​that defect; generating and displaying a flowchart of the defect detection algorithm based on the name of the first type of folder, the name of the second type of folder and the configuration relationship.

[0009] For example, prior to the generation step, the method further includes: obtaining the configuration relationship between the target area of ​​each specific defect and the defect area of ​​that defect, which is pre-configured by the user; the generation step includes: in response to a confirmation operation, generating and displaying a flowchart of the defect detection algorithm based on the name of the first type of folder, the name of the second type of folder, and the configuration relationship.

[0010] For example, there may be multiple first models and / or second models.

[0011] For example, the first model can be a target detection model and a semantic segmentation model; and / or the second model can be a target detection model and a semantic segmentation model, wherein the target detection model is used to identify defect regions or target regions with first morphological features in the image to be identified, and the semantic segmentation model is used to identify defect regions or target regions with second morphological features in the image to be identified.

[0012] For example, the folder includes a third type of folder, each third type of folder is used to store training sample images of the corresponding third model, each third model is used to detect whether the image to be identified is an abnormal condition, wherein the target information includes abnormal condition information; the generation step includes: in response to the confirmation operation, generating and displaying a flowchart based at least on the model category identifier and / or target identifier in the name of the third type of folder, wherein the steps of the defect detection algorithm include an anomaly detection step performed using the third model.

[0013] For example, the generation step includes: in response to the confirmation operation, generating and displaying a flowchart based at least on the model category identifier and / or target identifier in the names of the first type of folder and the third type of folder, wherein the defect detection algorithm further includes a post-processing step, in which the defect detection result of the image to be identified is determined based on the result of whether the defect region detected by the defect detection step and the image determined by the anomaly detection step is an abnormal condition; wherein, if the anomaly detection step determines that the image to be identified is an abnormal condition and the defect detection step determines that there is no defect region in the image to be identified, the defect detection result of the image to be identified is determined to be that there is a new type of defect region in the image to be identified.

[0014] For example, the third model includes an anomaly detection model and / or a binary classification model, wherein the training sample images of the anomaly detection model include only normal images, and the training sample images of the binary classification model include labeled normal images and labeled anomaly images.

[0015] For example, there are multiple folders, each including a parent folder and subfolders within each parent folder. The model categories of the models corresponding to the subfolders under each parent folder are the same, while the model categories of the models corresponding to the subfolders under different parent folders are different. The name of each parent folder includes a model category identifier, and the name of each subfolder includes a target identifier.

[0016] For example, prior to the naming step, the method further includes: creating a folder in response to a user's creation operation; and / or deleting the folder in response to a user's deletion operation.

[0017] According to a second aspect of the present invention, a defect detection method is also provided, comprising: storing training sample images of corresponding models in folders named according to preset naming rules, wherein each model is used to detect target information in the image to be identified, and the name of each folder includes the model category identifier and / or target identifier of the corresponding model; automatically training the model based on the training sample images and the model category identifier and / or target identifier in the name of each folder in response to a user's storage confirmation operation, so as to obtain a trained model; and executing the defect detection algorithm using the trained model based on the running path shown in the flowchart generated by the flowchart generation method of the above-described defect detection algorithm.

[0018] According to a third aspect of the present invention, a flowchart generation system for a defect detection algorithm is also provided, comprising: a naming module for naming at least one folder based on a preset naming rule, wherein each folder is used to store training sample images of a corresponding model, each model is used to detect target information in an image to be identified, and the name of each folder includes a model category identifier and / or a target identifier of the corresponding model; and a generation module for generating and displaying a flowchart of the defect detection algorithm based on the model category identifier and / or target identifier in the name of each folder in response to a user's confirmation operation, wherein the flowchart illustrates the running path of the steps of the defect detection algorithm.

[0019] According to a fourth aspect of the present invention, a defect detection system is also provided, comprising: a storage module for storing training sample images of corresponding models in folders named according to a preset naming rule, wherein each model is used to detect target information in an image to be identified, and the name of each folder includes a model category identifier and / or a target identifier of the corresponding model; a training module for automatically training the model based on the training sample images and the model category identifier and / or target identifier in the name of each folder in response to a user's storage confirmation operation, so as to obtain a trained model; and a detection module for executing a defect detection algorithm using the trained model, based on the running path shown in the flowchart generated by the flowchart generation method of the above-described defect detection algorithm.

[0020] According to a fifth aspect of the present invention, an electronic device is also provided, comprising a display, a processor, and a memory, wherein the display is used to display a user interface, the memory stores computer program instructions, and the computer program instructions are executed by the processor to perform a flowchart generation method for executing the above-described defect detection algorithm and / or the above-described defect detection method.

[0021] According to a sixth aspect of the present invention, a storage medium is also provided, on which program instructions are stored, wherein the program instructions, when executed, are used to perform the flowchart generation method and / or the defect detection method described above.

[0022] In the above technical solution, only the folder containing the training sample images of the defect detection model needs to be named according to a preset naming rule. After user confirmation, the flowchart of the defect detection algorithm can be displayed on the user interface. This solution allows users to intuitively obtain the running path of the currently configured defect detection algorithm, enabling them to quickly verify and improve the algorithm path in real time. This, in turn, helps generate a more accurate defect detection algorithm that better meets the user's personalized needs. Furthermore, this method is simple to implement, has low user requirements, and therefore has strong applicability and a better user experience.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0025] Figure 1 A schematic flowchart illustrating a method for generating a flowchart of a defect detection algorithm according to an embodiment of the present invention is shown.

[0026] Figure 2 A schematic diagram of a defect detection algorithm according to an embodiment of the present invention is shown;

[0027] Figure 3 A schematic diagram of a flowchart of a defect detection algorithm according to another embodiment of the present invention is shown;

[0028] Figure 4 A schematic diagram of a defect detection algorithm according to yet another embodiment of the present invention is shown;

[0029] Figure 5 A schematic flowchart of a defect detection method according to an embodiment of the present invention is shown;

[0030] Figure 6 A schematic block diagram of a flowchart generation system for a defect detection algorithm according to an embodiment of the present invention is shown.

[0031] Figure 7 A schematic block diagram of a defect detection system according to an embodiment of the present invention is shown; and

[0032] Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0034] Figure 1 A schematic flowchart of a method 100 for generating a flowchart of a defect detection algorithm according to an embodiment of the present invention is shown. Figure 1 As shown, the flowchart generation method 100 for the defect detection algorithm may include the following naming step S120 and generation step S140.

[0035] In the naming step S120, at least one folder is named according to a preset naming rule. Each folder is used to store training sample images of the corresponding model, each model is used to detect target information in the image to be identified, and the name of each folder includes the model category identifier and / or target identifier of the corresponding model.

[0036] For example, step S120 may be performed in response to a user's folder naming operation. Exemplarily, and not limitingly, the user's folder naming operation may include: selecting any folder and then clicking the folder with the mouse, using the F2 shortcut key, or entering the folder name via the keyboard. In step S120, when the system recognizes the user's above-mentioned operation, it names the folder accordingly as a response. For example, the user-entered folder name may be displayed in a corresponding position on the user interface, such as after the folder identifier.

[0037] Alternatively, step S120 can be executed directly by the system, without requiring the user to perform the folder naming operation. For example, if the system has pre-configured the names of each folder, step S120 can directly display the corresponding name of each folder in the corresponding location of the user interface.

[0038] For example, the preset naming rules can be the system's preset rules for the name content of each folder. For instance, these might include which identifiers each folder name should include, which optional letters or numbers can represent each identifier, the order of the identifiers, and the character limit for each identifier. It's easy to understand that preset naming rules indicate folder naming methods that the system can easily recognize or accept. In the aforementioned example of a user performing a folder naming operation, the user can name at least one folder based on information about the relevant folder naming rules pre-stored in the system, such as entering the name content of each folder under the guidance of the system's preset naming rule file. Naming folders based on preset naming rules allows users to quickly and accurately enter folder names that are easier for the system to recognize. Of course, the system can also automatically execute a folder naming algorithm based on preset naming rules to achieve step S120, meaning that the names of each folder pre-configured by the system can completely conform to the naming rules without requiring any user intervention.

[0039] For example, the named folders can be stored in a system-preset storage path. In the aforementioned example of a user performing a folder naming operation, the user can find the location of these folders by searching the storage path, and then perform the folder naming operation as described above.

[0040] For example, the folder naming operation in step S120 can be named one folder at a time, or a certain number of folders can be named in batches at the same time, or all folders can be named with one click, etc. There are many suitable and feasible methods, and the present invention does not limit them.

[0041] According to an embodiment of the present invention, each folder is used to store training sample images of the corresponding model, and each model is used to detect target information in the image to be identified.

[0042] For example, the image to be identified can be an image of any object to be defect-detected. In other words, the image to be identified can include the object to be defect-detected. The object to be defect-detected can be any suitable object, including but not limited to metals, glass, paper, electronic components, and other objects with strict requirements and clear indicators for appearance; this invention does not limit it.

[0043] For example, each model can be any suitable model capable of performing defect detection related to an object. For instance, it can be a deep learning-based neural network model, including but not limited to object detection models, semantic segmentation models, and many other types of models.

[0044] For example, the different models corresponding to different folders can be models of the same category or models of different categories. For two models of the same category, their network structures can be exactly the same or not exactly the same. For example, the above models can be models capable of performing defect detection algorithms. Within the system, an appropriate number, category, and structure of models can be selected and set according to the actual needs of executing the defect detection algorithm. By finding the storage path of the folders corresponding to different models, in step S120, different folders used to store training sample images of different models can be named differently based on preset folder naming rules, so as to clearly distinguish the storage purpose of each folder.

[0045] For example, the training sample images for each model can be sample images with labeled target information or normal sample images without labeled target information. The training sample images differ between models. For example, the target information can be any suitable information in the image to be identified that is related to defect detection, including but not limited to defect type information, defect state information, and defect-related location information. Defect-related location information includes, for example, the possible locations of defects and locations that are normally present but easily identified as defects. It is easy to understand that inputting the image to be identified into a model trained using the training sample images will output the target information in the image to be identified.

[0046] The name of each folder may include the model category identifier and / or target identifier of the corresponding model. For example, the preset naming rules may include information such as the representation method of the model category identifier and / or target identifier contained in the folder name, and the order of the identifiers.

[0047] By way of example and not limitation, model category identifiers may be, for example, "Detect Model" for object detection models, "Segment Model" for semantic segmentation models, etc., and the present invention does not limit them.

[0048] For example, the target identifier can be a result identifier used to represent the output result of each model, which can be an identifier of the target information contained in the output result of each model. For example, if the model output result includes defect category information or defect region of a defect in the image to be identified, the target identifier can include an identifier of the defect type, such as "Defect a", "Defect 1", "Crack", etc. As another example, if the model output result includes a target region of a defect in the image to be identified, the target identifier can include an identifier of the target region, such as "Pad area", "Cat Region", etc.

[0049] For example, different naming methods can be used based on preset naming rules and different storage purposes of folders. For instance, for a folder that corresponds to only one model, its name may include a target identifier, or it may also include a model category identifier. For a folder that corresponds to multiple models of different categories, its name may include a model category identifier; for a folder that corresponds to multiple models of the same category, its name may include a target identifier.

[0050] According to embodiments of the present invention, naming folders based on model category and / or model output results helps users intuitively obtain key information about the model. This not only facilitates storing training sample images of different models in the correct folders, but also makes it easier to intuitively display model category and / or model output results in the subsequent generated flowchart steps, allowing users to judge whether they are consistent with the ideally designed algorithm flowchart.

[0051] In step S140, in response to the user's confirmation, a flowchart of the defect detection algorithm is generated and displayed based on the model category identifier and / or target identifier in the name of each folder. The flowchart illustrates the execution path of the steps of the defect detection algorithm.

[0052] For example, a user's confirmation action could be clicking a confirmation control displayed on the user interface. For instance, the user interface's menu bar could include a "Confirm Display Flowchart" button. After naming the folder, the user could click this button. Once the system recognizes this user action, it can load a preset display algorithm and display the flowchart of the defect detection algorithm in real time on the user interface.

[0053] As stated above, each folder can correspond to a model capable of detecting one or more target information in the image to be identified, and the name of each folder can include the model category identifier and / or target identifier of the corresponding model. Therefore, in step S140, the model category information of each model can be obtained through the model category identifier contained in the folder name, and the target information identified by each model can be obtained through the target identifier in the folder name. That is, the key information of each model can be quickly obtained according to the folder name in the preset path, and then a defect detection algorithm flowchart can be formed according to the key information, and the flowchart can be displayed on the user interface for user review. The key information can be directly the model category identifier and / or target identifier, or it can be key information generated based on the model category identifier and / or target identifier. For example, a one-to-one corresponding icon can be generated according to each model category identifier, and the icon and target identifier are displayed as key information on the flowchart.

[0054] According to embodiments of the present invention, the flowchart of the defect detection algorithm can illustrate the execution path of the algorithm's steps. For example, the steps of the defect detection algorithm may include detection steps for each model corresponding to a folder, and the key information of the model contained in the corresponding folder name can be visually presented in the illustrated detection steps for each model, facilitating user verification of the algorithm's correctness. For example, the steps of the defect detection algorithm may also include other conventional steps preset by the system or selectable by the user, such as image preprocessing steps for the image to be identified, output steps for the detection results of each model, and post-processing steps.

[0055] Figure 2 A schematic diagram of a defect detection algorithm according to an embodiment of the present invention is shown. Figure 2 As shown in the flowchart of the defect detection algorithm, the algorithm's execution path includes multiple steps. These steps include image preprocessing step 210, detection steps for the two models 220, and outputting the detection results for each model 230. It is easy to understand that image preprocessing step 210 and detection result output step 230 can be pre-set system steps that do not require user modification, while model detection step 220 can be generated based on user selection or operation.

[0056] For example, in this flowchart, the information displayed in each model's detection step may include key model information contained in the name of the corresponding folder. For example... Figure 2 As shown, the model detection step 220 can include the parallel execution of the detection steps for the first model "Detect Model1 Defect1" and the second model "Detect Model2 Defect2". Parallel execution is a pre-set execution order rule within the system. Of course, in practical applications, the system can also preset other execution order rules such as serial, serial + parallel, etc. It is easy to see that... Figure 2The illustrated defect detection algorithm includes detection steps for two models of the same type, but used to detect different target information. It's easy to understand that in this example, step S120 has correctly named at least two folders, meaning at least two folders are named according to a preset naming rule within a preset storage path. It's also easy to understand that the more folders named according to the preset naming rule, the more corresponding models there are, and the more detection steps for different models are shown in the flowchart. Therefore, users can check whether the multiple steps of the illustrated defect detection algorithm are the expected execution path by generating a flowchart with key information. If so, model training can be performed according to the path shown in the flowchart, followed by defect detection of the image to be recognized based on the trained model; if not, the user can return to step S120 to check and modify the names of each folder, and check the number of folders to delete unnecessary folders or create new ones, etc.

[0057] It should be noted that, Figure 2 The flowchart of the defect detection algorithm shown is merely an example, and any existing or future flowcharts are also within the scope of protection of this invention.

[0058] It is understandable that the corresponding model can be trained using the training sample images in the folder. During the execution of the defect detection algorithm shown in the flowchart, the trained model can be used to perform various defect detections. Thus, defect detection of the image to be recognized is achieved.

[0059] In the above technical solution, after naming the folders containing training sample images for defect detection models according to preset naming rules, and upon user confirmation, a flowchart of the defect detection algorithm is displayed on the user interface based on the model category identifier and / or target identifier in the name of each folder. This solution allows users to intuitively obtain the running path of the currently configured defect detection algorithm, enabling them to quickly verify and improve the algorithm path in real time, thereby helping to generate more accurate defect detection algorithms that better meet the user's personalized needs. Furthermore, this method is simple to implement, has low user requirements, and therefore has strong applicability and a better user experience.

[0060] For example, prior to the naming step S120, method 100 further includes: step S110, creating a folder in response to a user's creation operation; and / or step S111, deleting a folder in response to a user's deletion operation.

[0061] For example, the user's creation operation may include various methods such as clicking the creation control or using mouse or keyboard shortcuts. For example, in step S110, when the system receives the user's operation information, it can create a corresponding number of folders under the path operated on by the user, as the folders operated on in step S120. For example, in step S110, the folder creation path can be a preset path of the system. For example, the system includes a folder storage path indicator file, which indicates the storage path of folders corresponding to different models. The user can consult this indicator file to find the preset storage path of the desired folder, and then create the corresponding folder under that path.

[0062] For example, a user's deletion operation may include clicking to select one or more folders, clicking the delete control in the user interface, or using the "Delete" shortcut key on the keyboard. For instance, if the user has created a large number of folders or the system already has many folders, the user can selectively delete some redundant folders. For example, if the preset storage path already contains 8 folders, and the number of defect detection-related models is 5, the user may actually need 5 folders. Therefore, 3 folders can be deleted in step S111, and the remaining 5 folders can be named in step S120.

[0063] The flowchart generation method 100 for the defect detection algorithm according to an embodiment of the present invention may further include a folder creation step and / or a folder deletion step performed by the user, which helps to generate a flowchart for the defect detection algorithm that better meets the user's personalized needs.

[0064] For example, there are multiple folders, each including a parent folder and subfolders within each parent folder. The model categories of the subfolders under each parent folder are the same. The model categories of the subfolders under different parent folders are different. The name of each parent folder includes a model category identifier, and the name of each subfolder includes a target identifier.

[0065] According to embodiments of the present invention, the number of folders is multiple and presented in a multi-level manner, for example, a two-level folder structure consisting of a parent folder and its child folders. Exemplarily, each parent folder may include one child folder, or it may include multiple child folders. Optionally, each parent folder may include multiple child folders.

[0066] For example, when the number of models is small, each folder can be used to store training sample images for one model, and the name of each folder can include the model category identifier and the target identifier. When the number of models is large, folders corresponding to models of the same type can be stored as subfolders within a parent folder. In this case, each subfolder can be used to directly store training sample images for one model, while each parent folder can be used to indirectly store training sample images for one or more models of the same type. For example, the name of each parent folder can include the model category identifier corresponding to the subfolder, while the name of each subfolder can only include the target identifier, without including the model category identifier. For example, a parent folder might be named "Detect Model," and its three subfolders might be named "Defect1," "Defect2," and "Defect3."

[0067] According to the above scheme, when there are a large number of models, training sample images for different types of models can be stored in a multi-level folder structure. This not only visually presents the distribution of different model types but also simplifies folder names, thereby saving user time and improving user experience.

[0068] For example, the folder includes a first type of folder, wherein each first type of folder is used to store training sample images of a corresponding first model, each first model is used to identify defect regions of at least one defect in the image to be identified, target information includes information of the defect region, and target identifier includes a type identifier of the defect identified by the corresponding first model; the generation step S140 includes: in response to the user's confirmation operation, generating and displaying a flowchart of the defect detection algorithm based on the model category identifier and / or target identifier in the name of the first type of folder, the steps of the defect detection algorithm including defect detection steps performed using the first model.

[0069] For example, the model utilized by the defect detection algorithm may include a first model, and the folder corresponding to each first model is a first type of folder. The first model may be a defect recognition model for identifying defect regions in at least one defect in an image to be recognized. For example, a defect region may be a defective area of ​​an object in the image to be recognized, and it may be a partial region of the image. It is readily understood that normal areas and defective areas of an object in the image to be recognized may have different morphologies, and defective regions may be detected based on their different morphologies, such as grayscale, texture, etc. For example, in the case where the image to be recognized is an image of some kind of metal, the defective region may be an area showing scratches on the metal object.

[0070] It is easy to understand that the object to be defect-detected may include various types of defects, such as surface redundancy defects, slip line defects, stacking fault defects, scratch defects, and pattern defects on the wafer surface. Due to the variability in defect morphology, a single universal model cannot be used to detect all types of defects. For example, defects can first be classified according to their morphology. For instance, the defects that may appear in the image to be identified can be divided into five categories. Then, multiple first models can be used to detect the defect regions of each of the five types of defects. Optionally, there can be five first models, each used to identify the defect region of one type of defect. Alternatively, the number of first models can be less than five, for example, four, where three models identify the defect region of one type of defect, and the other model can be used to identify the defect regions of two similar defects.

[0071] It is easy to understand that the number of folders in the first category is positively correlated with the number of first models. The number of folders in the first category can be equal to or greater than the number of first models. For example, the models used by the defect detection algorithm can be all first models or only some of the first models. The folders in the first category can be all the folders named in step S120 or only some of them.

[0072] For example, the first model can be any suitable neural network model, such as various forms of object detection models, semantic segmentation models, etc. The present invention does not limit it, as long as it can be used to identify defective regions in the image to be identified.

[0073] For example, the model categories of two first models corresponding to any two first-category folders can be exactly the same or not. For example, the network structures of first models in the same category can be exactly the same or not. This invention does not limit these aspects.

[0074] Similarly, users can select the appropriate number, category, and network structure of the first model based on the actual needs of executing the defect detection algorithm, such as the number of defect types, the requirements for detection accuracy and detection speed.

[0075] For example, the name of the first type of folder may include the model category identifier of the corresponding first model and the target identifier of the target information detected by the first model. For example, the target identifier in the name of the first type of folder may be the type identifier of the defect identified by the first model. It is easy to understand that if the first model corresponding to the first type of folder is used to detect a defect in the image to be identified, then the target identifier in the name of the first type of folder may be the defect identifier of that defect, such as "Defect1"; if the first model corresponding to the first type of folder is used to detect multiple defects in the image to be identified, then the target identifier in the name of the first type of folder may be the defect identifier of the multiple defects, such as "Defect2Defect3Defect4".

[0076] For example, the system may include an instruction file for folder storage paths. Users can use this instruction file to find the storage path of the first type of folder corresponding to the first model, and then perform operations such as folder creation, folder deletion, and folder naming under the storage path.

[0077] According to an embodiment of the present invention, after receiving the user's confirmation operation in generation step S140, a flowchart of the defect detection algorithm is generated and displayed based on the model category identifier and / or target identifier in the name of the first type of folder. It is readily understood that the execution path of the defect detection algorithm steps illustrated in the flowchart may include the defect detection steps performed by the first model.

[0078] For example, relevant information of each first model can be obtained according to the name of the first type of folder in the preset storage path, and then a defect detection algorithm flowchart including the defect detection steps of the first model can be generated, and the flowchart can be displayed on the user interface for user verification.

[0079] For example, refer again Figure 2 In the diagram, detection step 220 can be a defect detection step performed by two first models. "Detect Model1 Defect1" can be the defect detection step of the first first model, and "Detect Model2 Defect2" can be the defect detection step of the second first model. It's easy to understand that the two first models can be of the same category, sharing the same model category identifier "Detect Model". Each of the two first models can be used to detect a defect region of a specific type of defect. Specifically, the first first model can be used to detect defect 1 "Defect1", and the second first model can be used to detect defect 2 "Defect2".

[0080] According to the above scheme, after naming the first type of folders according to the preset naming rules, a flowchart of the defect detection algorithm, including the defect detection steps performed by each first model, can be automatically generated on the user interface based on simple user operations. This method is not only simple to operate and requires little knowledge from the user, but also intuitively presents the running path of the defect detection algorithm for easy user verification. Furthermore, since each first model can be used to detect the defect region of at least one defect in the image to be identified, it helps to generate defect detection algorithms with higher accuracy.

[0081] For example, the folder also includes a second type of folder, wherein each second type of folder is used to store training sample images of the corresponding second model, each second model is used to identify target regions configured for at least one specific defect in the image to be identified, the target information includes information about the target region, and the target identifier includes the identifier of the target region of the specific defect identified by the corresponding second model; the generation step S140 includes: in response to the confirmation operation, generating and displaying a flowchart based on the model category identifier and / or target identifier in the names of the first type of folder and the second type of folder, wherein the steps of the defect detection algorithm further include a target region detection step and a post-processing step performed using the second model, in which the defect detection result of the image to be identified is determined based on the defect region detected by the defect detection step, the target region detected by the target region detection step, and the configuration relationship between the defect region and the target region.

[0082] For example, the model used by the defect detection algorithm may include a first model and a second model, and the folders involved may include a first type of folder corresponding to the first model and a second type of folder corresponding to the second model.

[0083] According to embodiments of the present invention, each second model corresponding to the second type of folder is used to identify a target region in the image to be identified, configured for at least one specific defect. Exemplarily, similar to a defect region, the target region can be a partial region in the image to be identified. Exemplarily, the target region of the image to be identified can be a region related to the defect to be detected. According to embodiments of the present invention, the target region can be a target region configured for at least one specific defect. In other words, for a specific defect, not only can a corresponding defect region exist, but a corresponding target region can also be configured for it. For example, the number of first models corresponding to the first type of folder can be four, which can be used to detect defects a, b, c, and d in the image to be identified, respectively, wherein defects b and c can be specific defects. Exemplarily, the number of second models corresponding to the second type of folder can be two, which can be used to identify the target regions of defect b and defect c, respectively.

[0084] For example, the target area may include at least one of the following: the misidentified area of ​​the defect, or the specific area where the defect is located.

[0085] For example, a misidentified region of a specific defect can be a normal region that has a high similarity to the defective region of the specific defect. It is easy to understand that because the misidentified region of a specific defect and the defective region are highly similar, the first model may identify the misidentified region as a defective region, leading to over-detection of the defect. To effectively avoid this over-detection, the misidentified region can be identified by a second model. This allows the misidentified region to be excluded in subsequent steps when the first model makes a misidentification, thereby improving the accuracy of defect detection.

[0086] For example, the specific region where a specific defect is located can be a relatively fixed region where only that specific defect can exist. For instance, certain specific defects on a wafer exist only on specific devices, and these specific defects cannot exist in other parts of the wafer. For example, certain specific defects exist only on the pin connection area of ​​a silicon wafer or on the electrode area used for chip interconnection, etc., and the second model is used to identify and output the aforementioned regions in the image to be identified. For example, in the example where the aforementioned specific defects include defects b and c, defect b exists only in a specific region. The first model identifies the defect region of defect b in the image to be identified, and the second model identifies the specific region in the image where defect b may exist. In subsequent steps, defect regions located only in that specific region can be filtered from the defect regions output by the first model. This can reduce the number of incorrectly identified defect regions, ensure the accuracy of the final defect detection results, and improve the precision of defect detection.

[0087] It is easy to understand that when both the second model and the first model are used to detect specific defects in the image to be identified, dual detection of a certain specific defect in the image to be identified can be achieved by the two models.

[0088] For example, the second model corresponding to the second type of folder can be used to identify the target area configured for the specific defect as described above. The target area for the specific defect identified by each second model may include only the misidentified area of ​​the specific defect, or only the specific area where the specific defect is located, or both. For example, in the aforementioned example, the second model can be used to identify both the misidentified area of ​​defect b and the specific area where defect b is located.

[0089] For example, similar to the first model, each second model can be used to identify a target region of a specific defect in the image to be identified, or it can be used to identify target regions of multiple specific defects. Those skilled in the art can easily understand this scheme and its possible extensions, which will not be elaborated here.

[0090] For example, the number of the second model can be any suitable positive integer, and the present invention does not limit it. For example, and not limitingly, the number of the second model is not greater than the number of the first model, and the number of the second type of folders is not greater than the number of the first type of folders.

[0091] For example, similar to the first model, the second model can also be any existing or future model that can identify the target region in the image to be identified, and the present invention does not limit it.

[0092] For example, similar to the first model, the model categories of each second model may or may not be exactly the same. For example, the model categories of the second models may be the same as or different from the model categories of the first models. Optionally, the model categories of the first models and the second models may be exactly the same. Alternatively, the model categories in the multiple first models and the multiple second models may be partially the same, and the first models may include all the model categories of the second models. For example, the second models may include object detection models, and the first models may include object detection models and at least one other type of model.

[0093] For example, the network structures of multiple second models belonging to the same category may be completely identical or not completely identical. Optionally, the network structures of multiple second models belonging to the same category may be completely identical, and the network structures of first and second models belonging to the same model category may also be identical, which can greatly reduce the cost of model maintenance.

[0094] According to embodiments of the present invention, users can select a first model and a second model with appropriate number, appropriate category, and appropriate structure based on the actual needs of executing the defect detection algorithm.

[0095] According to embodiments of the present invention, the name of the second type of folder corresponding to the second model may include the model category identifier of the corresponding second model and / or the identifier of the target area of ​​the specific defect identified by the second model. Exemplarily, the model category identifier of the second model may adopt a representation method similar to that of the aforementioned first model, which will not be repeated here. Exemplarily, the identifier of the target area of ​​the specific defect identified by the second model may adopt any suitable representation method, such as "Area 1", "Pad Area", "Pump Area", "Cat 1", or any other recognizable representation method.

[0096] For example, the name of the second type of folder may also include a defect type identifier for the specific defect targeted by the target area identified by the corresponding second model. For example, the defect type identifier for the specific defect may be the defect type identifier of the aforementioned example, such as "Defect1", "Defect 1", etc. It is easy to understand that when the defect identified by the first model corresponding to a certain first type of folder and the defect targeted by the target area identified by the second model corresponding to a certain second type of folder are the same type of defect, the names of the first type of folder and the second type of folder may both contain the same defect type identifier.

[0097] For example, the system may include an instruction file for folder storage paths. Users can use this instruction file to find the storage paths of the first type of folder corresponding to the first model and the second type of folder corresponding to the second model, and then perform operations such as folder creation, folder deletion, and folder naming under the corresponding storage paths. For example, the storage paths of the first type of folder and the second type of folder may be the same or different; this invention does not limit this.

[0098] According to an embodiment of the present invention, in generation step S140, after receiving the user's confirmation operation, a flowchart of the defect detection algorithm can be generated and displayed based on the model category identifier and / or target identifier in the names of the first type of folder and the second type of folder. It is readily understood that the execution path of the defect detection algorithm steps illustrated in this flowchart may include the defect detection steps performed by the first model and the target area detection steps performed by the second model.

[0099] Figure 3 A schematic diagram of a flowchart of another defect detection algorithm according to an embodiment of the present invention is shown. Figure 3As shown in the flowchart of the defect detection algorithm, the execution path of multiple steps of the algorithm is also illustrated. The detection steps of the model include a defect detection step 310 executed by a first model and a target region detection step 320 executed by a second model, both executed in parallel. The defect detection step 310 executed by the first model includes a "Detect Model1 Defect1" step and a "Detect Model2 Defect2" step. The target region detection step 320 executed by the second model includes a "Detect Model2 Defect2 Cat1" step. For example, the name of each model's detection step can be consistent with the name of the corresponding folder. Based on the aforementioned folder naming example, it can be understood that the model type of these three models is the same type: "Detect Model". The two first models are used to detect the defect regions "Defect1" and "Defect2" respectively. For example, the second model can be used to detect the target region Cat1 for a specific defect "Defect2".

[0100] It is easy to understand that when the model includes a first model and a second model, there are multiple detection results for the image to be identified. Simultaneously, there are also two models outputting detection results for a specific defect. For example, to facilitate obtaining the final detection result for that defect in the image to be identified, post-processing can be performed on the detection results of multiple models. According to embodiments of the present invention, the steps of the defect detection algorithm shown in the flowchart of the defect detection algorithm may further include a post-processing step.

[0101] According to an embodiment of the present invention, in the post-processing step, the defect detection result of the image to be identified can be determined based on the defect region detected by the defect detection step, the target region detected by the target region detection step, and the configuration relationship between the defect region and the target region.

[0102] For example, when the output of the first model includes a defect region of a specific defect for which a target region is configured, the defect region and the target region can be post-processed based on the configuration relationship between the defect region, the target region output by the second model, and the defect region of the specific defect and the target region to determine the defect detection result. For example, the configuration relationship can include correspondence information between the target region of each specific defect and the defect region of that defect, for example, in... Figure 3In this model, a configuration relationship exists between the defect region of "Defect2" output by the first model and the target region of "Defect2" output by the second model. For example, the configuration relationship may also include information on the configuration method, which differs for different types of target regions. For example, if the target region is a misidentified region of a specific defect, the configuration method for the defect region of that specific defect can be a filtering configuration. The defect region and target region configured in this way can be post-processed using a filtering method. For instance, the misidentified region can be deleted from the defect region to obtain the filtered defect region as the final detection result. For example, if the target region is a specific region where a specific defect is located, the configuration method for the defect region of that specific defect can be a retention configuration. The defect region and target region configured in this way can be post-processed using a retention method. For example, the overlapping area between the defect region of a specific defect output by the first model and the determined target region output by the second model can be obtained, and the defect region at the overlapping area can be retained as the final detection result. Alternatively, it can be further determined whether the overlapping area between the defect region of the specific defect and the specific region meets a preset range requirement. For example, it can be determined whether the area ratio of the overlapping area to the specific area reaches a preset ratio threshold, such as 60%. Only when the ratio threshold is reached will the defect area in the overlapping area be regarded as the defect area of ​​the specific defect.

[0103] For example, refer again Figure 3 Following the detection step of the model, a post-processing step 330 and a result output step may also be included. For example, in the post-processing step 330, based on the configuration relationship between the defect region of "Defect2" output by the second first model and the target region Cat1 of "Defect2" output by the second model, post-processing operations are performed according to the different processing methods described above for different configuration methods to obtain the final detection result. Finally, the final defect detection result can be output through the result output step.

[0104] According to the above scheme, after naming the first and second type of folders according to the preset naming rules, a flowchart of the defect detection algorithm, including the defect detection steps performed by the first model, the target area detection steps performed by the second model, and the post-processing steps, can be generated based on simple user operations. This method is simple and intuitive, facilitating user verification and correction of the algorithm path. Furthermore, the addition of the second model enables dual detection of certain defects by both models, and the final defect detection result can be obtained by combining the detection results of both models in the post-processing step. Therefore, it also helps to generate a high-precision defect detection algorithm, resulting in a better user experience.

[0105] For example, the name of the second type of folder or the first type of folder also includes a configuration relationship identifier between the defect area and the target area; the generation step S140 includes: step S141, in response to the aforementioned confirmation operation, reading the name of the first type of folder and the name of the second type of folder; step S142, based on the configuration relationship identifier in the name of the first type of folder or the second type of folder, obtaining the configuration relationship between the target area of ​​each specific defect and the defect area of ​​that defect; step S143, generating and displaying a flowchart of the defect detection algorithm based on the name of the first type of folder, the name of the second type of folder and the configuration relationship.

[0106] As stated above, when the model includes a first model and a second model, a configuration relationship exists between the defect region and the target region of a specific defect. In other words, a configuration relationship exists between the outputs of the first model used to detect a specific defect in the image to be identified and the outputs of the second model used to detect the target region for that specific defect. To facilitate users in obtaining the configuration relationship between the outputs of different models, folders can be named based on this configuration relationship. For example, a first-type folder and a second-type folder, which correspond to the outputs of the models and have a configuration relationship, may include a configuration relationship identifier in their names. In particular, if the first-type folder or the second-type folder includes multiple layers of folders, such as parent and subfolders, the name of the subfolder directly corresponding to the model may include a configuration relationship identifier.

[0107] As stated above, the configuration relationship can include information about the configuration method between model detection results. Therefore, corresponding configuration relationship identifiers can be set based on the configuration method between model results. Exemplarily, but not limitingly, in the aforementioned example, the configuration method between the output results of the first model and the second model can include methods of retaining and filtering configurations. The configuration relationship identifier in the corresponding folder name can include "In" or "Not In". For example, the name of the first type of folder or the second type of folder can include "A In B" or "A Not In B", where A is the target identifier corresponding to the first model, representing the defect region of a specific defect output by that model, and B is the target identifier corresponding to the second model, representing the target region for a specific defect output by that model. "A In B" can indicate a retain configuration method between the results of the two models, meaning the second model is used to detect the specific region B that may exist for the specific defect A detected by the first model. "A Not In B" can indicate a filtering configuration method between the results of the two models, meaning the second model is used to detect the misidentified region B of the specific defect A detected by the first model.

[0108] According to an embodiment of the present invention, when the name of the folder includes a configuration relationship identifier, in the generation step S140, the configuration relationship between the detection results of the first model and the second model can be obtained by reading the name of the folder, and then a flowchart of the defect detection algorithm can be generated and displayed based on the key information of the model and the configuration relationship information contained in the folder name.

[0109] For example, in step S141, after recognizing the user's operation on the confirmation control, the names of each first-type folder and second-type folder under the preset path can be read. Since the folder names include the model category identifier of the corresponding model and / or the target identifier of the model output result, key information of each model can be obtained. Simultaneously, the names of the first-type or second-type folders with configuration relationships between the detection results of the corresponding models also include configuration relationship identifiers. Therefore, in step S142, based on the read configuration relationship identifiers, the configuration relationship between the output results of the first model and the second model can be obtained, i.e., the configuration relationship between the target area of ​​each specific defect and the defect area of ​​that defect. Then, in step S143, a flowchart of the defect detection algorithm can be generated and displayed based on the obtained key information of the models and the configuration relationship between the model results. It is easy to understand that this flowchart can show the running path of multiple steps of the defect detection algorithm, including the defect detection step performed by the first model, the target area detection step performed by the second model, and the post-processing step. Furthermore, in the post-processing step, the corresponding post-processing operation can be automatically executed based on the obtained configuration relationship between the output results of the first model and the second model.

[0110] Figure 4 A flowchart of a defect detection algorithm according to another embodiment of the present invention is shown. Figure 4As shown in the flowchart of the defect detection algorithm, the detection steps and post-processing steps are executed in parallel by two first models and two second models. This includes the detection step of the first first model for the defect region "Defect1" in the image to be identified, the detection step of the second first model for the defect region "Defect2", the detection step of the first second model for the target region "Cat1" of "Defect1" in the image to be identified, and the detection step of the second second model for the target region "Cat2" of "Defect2" in the image to be identified. It is easy to understand that "Defect1" and "Defect2" are specific defects. Furthermore, there is a configuration relationship between the detection results of the first first model and the first second model; based on the configuration relationship identifier "in" in the name, the configuration method between them is a reserved configuration. There is also a configuration relationship between the detection results of the second first model and the second second model; based on the configuration relationship identifier "not in" in the name, the configuration method between them is a filtering configuration. After obtaining the above configuration method, the detection results of the four models can be automatically post-processed in the post-processing step based on the corresponding configuration method to obtain the processed defect detection results and output the final defect detection results in the result output step.

[0111] It should be noted that, in Figure 4 In the flowchart of the defect detection algorithm, displaying configuration relationship information in the detection step of the second model is just one example. Alternatively, configuration relationship information can be displayed in the detection step of the first model. For example, the detection step of the first first model can be represented as "Detect Model1 Defect1 InCat1", and the detection step of the first second model can be represented as "Detect Model3 Cat1". When there is a configuration relationship between the detection results of multiple first models and the detection result of a second model, the configuration relationship information can be displayed in the detection step of the first model, which can more accurately present the configuration relationship information. Alternatively, configuration relationship information can also be displayed in the post-processing step. For example, the detection step of the first first model can be represented as "DetectModel1Defect1", the detection step of the second first model can be represented as "Detect Model2 Defect2", the detection step of the first second model can be represented as "Detect Model3 Cat1", the detection step of the second second model can be represented as "Detect Model4 Cat2", and the post-processing step can be represented as "Defect1 In Cat1 & Defect2 Not InCat2".

[0112] The above solution allows for the acquisition of the configuration relationship between the detection results of the first and second models by reading the names of the first and second type of folders. Based on this configuration relationship, a flowchart of the defect detection algorithm, including defect detection steps, target area detection steps, and post-processing steps, can be generated. Furthermore, the configuration relationship information can be displayed at any of these three steps, providing a clear and accurate view of the user's currently configured algorithm's execution path. Moreover, this solution eliminates the need for user-configured configuration files, simplifying the implementation process and saving computational resources.

[0113] For example, before generating step S140, the method 100 further includes step S130: obtaining the configuration relationship between the target area of ​​each specific defect pre-configured by the user and the defect area of ​​that defect; generating step S140 includes: in response to the confirmation operation, generating and displaying a flowchart of the defect detection algorithm based on the name of the first type of folder, the name of the second type of folder and the configuration relationship.

[0114] According to embodiments of the present invention, the configuration relationship identifier may not be reflected in the folder name; instead, the configuration relationship can be pre-defined by the user. For example, the user can edit the model configuration relationship in the system's configuration file. It is easy to understand that a configuration file is a user-configurable file containing information required for the successful operation of the program. If the user is familiar with configuration file editing methods, they can edit the configuration relationship information between the detection results of different models in the defect detection algorithm within the configuration file. In one example, the user can directly edit configuration relationship information such as "A In B" and "ANOT In B" in the configuration file. (See reference...) Figure 3 For example, users can edit the text information in the configuration file to represent "Defect2in Cat1". Correspondingly, the name of the second type folder corresponding to the second model in the figure does not need to include the configuration relationship "in", that is, its name can be "Detect model 2Cat1".

[0115] In another example, besides editing the configuration relationship information mentioned above in the configuration file, the correspondence between each model and the detected target information can also be edited in the configuration file. For example, the configuration file can specify which defect each first model is used to detect or which target region each second model is used to detect, that is, pre-configuring the correspondence between the models and the output results. (See again...) Figure 4Users can specify the correspondence between "Detect Model1" and "Defect1", "Detect Model2" and "Defect2", "Detect Model3" and "Cat1", and "Detect Model4" and "Cat2" in the configuration file, and edit the configuration relationship information "Defect1 In Cat1" and "Defect2 Not In Cat2". In this case, the name of at least one folder can include only the model category identifier and the distinguishing symbol for the same type of model. For example, the distinguishing symbol for the same type of model can be Arabic numerals. Figure 4 In the example, the four folders can be named “Detect Model1”, “Detect Model2”, “Detect Model3”, and “Detect Model4”, respectively. “Detect Model” is the model category identifier, and the number following it is the distinguishing symbol for the same type of model.

[0116] Alternatively, a user interface including a configuration relationship editing window can be displayed on the host computer, allowing the user to edit the configuration relationship information between the detection results of the first model and the detection results of the second model. Of course, the above configuration relationships can also be pre-configured in other suitable ways, and this invention does not limit such pre-configuration.

[0117] For example, after the user completes the pre-configuration of the configuration relationship, such as saving the edited configuration file in the corresponding path in the system, in step S130, the configuration relationship between the target area and the defect area of ​​each specific defect can be obtained by reading the configuration file. Then, in step S140, a flowchart of the defect detection algorithm can be generated and displayed based on the folder name and the configuration relationship. This scheme is readily understood by those skilled in the art and will not be described in detail here.

[0118] According to the above scheme, the configuration relationship between the detection results of the first model and the second model can be obtained by pre-configuring configuration relationship information by the user. A flowchart of the defect detection algorithm is then generated based on the folder name and configuration relationship. This scheme greatly simplifies the folder naming process, especially when users need to perform folder naming operations; it makes naming simpler and more user-friendly.

[0119] For example, there may be multiple first-type and / or second-type models. As stated above, each folder may directly correspond to one model or indirectly correspond to multiple models, therefore there may also be multiple first-type and / or second-type folders.

[0120] As stated above, the number of first and second models can be set according to actual detection needs. Actual detection needs may include the requirements for detecting different types of defects, the computational power requirements of the models, and the required detection accuracy. For example, when multiple types of defects need to be detected, the number of first models can be increased accordingly. For instance, five first models can be used to detect the defect areas of five different defects. Conversely, when the number of defect types targeting specific areas is small, fewer second models can be set, for example, only one second model can be used to identify the target area of ​​a specific defect. Of course, the number of second models can also be set to more, for example, five second models, each detecting the target area of ​​one type of defect. Similarly, the number of first models can be set to one, for example, using one first model to detect the defect areas of two different defects. The number of second models can be set to multiple, for example, each second model can detect the target area of ​​one type of defect. In short, there are multiple possible settings for the number of first and second models, as long as they meet the detection requirements.

[0121] Employing multiple primary models and / or multiple secondary models allows for more targeted identification of defect regions and target regions for different types of defects. This contributes to the development of defect detection algorithms with higher accuracy and greater applicability.

[0122] For example, the first model can be a target detection model and a semantic segmentation model; and / or the second model can be a target detection model and a semantic segmentation model, wherein the target detection model is used to identify defect regions or target regions with first morphological features in the image to be identified, and the semantic segmentation model is used to identify defect regions or target regions with second morphological features in the image to be identified.

[0123] For example, at least one of the first model and the second model includes an object detection model and a semantic segmentation model. Optionally, the first model includes an object detection model and a semantic segmentation model, and the second model includes one of these types; alternatively, both the first model and the second model include an object detection model and a semantic segmentation model.

[0124] For example, the object detection model can be any existing or future neural network model capable of object detection. Examples include Faster R-CNN, YOLO series models, Single Shot Multibox Detector (SSD), and Fully Convolutional One-Stage Object Detection (FCOS). Similarly, the semantic segmentation model can be various forms of neural network models, such as U-Net, FCN, SegNet, PspNet, and DeepLab series models.

[0125] According to embodiments of the present invention, a target detection model can be used to identify defect regions or target regions with first morphological features in an image to be identified. Exemplarily, the first morphological feature can be not smaller than a first size, for example, larger than a certain size threshold. Since target detection models can more easily and relatively accurately identify larger targets, using a target detection model to identify large-sized defect regions in an image to be identified results in higher detection efficiency and accuracy.

[0126] According to embodiments of the present invention, a semantic segmentation model can be used to identify defect regions or target regions with second morphological features in an image to be identified. Exemplarily, the second morphological feature can be an elongated feature or other morphological features that are easily recognized by the semantic segmentation model. For example, a semantic segmentation model can be used to identify scratches or cracks in a metal image. For elongated defects, the semantic segmentation model has a smaller detection error; therefore, using a semantic segmentation model to detect elongated defects can also improve detection accuracy.

[0127] According to the above scheme, different types of models can be used to identify defect regions or target regions with different morphological features in the image to be identified. The model has a higher degree of adaptability to the detected defects, which helps to generate defect detection algorithms with higher detection accuracy and efficiency.

[0128] For example, the folder includes a third type of folder, each third type of folder is used to store training sample images of the corresponding third model, each third model is used to detect whether the image to be identified is an abnormal condition, wherein the target information includes abnormal condition information; the generation step S140 includes: in response to the confirmation operation, generating and displaying a flowchart based at least on the model category identifier and / or target identifier in the name of the third type of folder, wherein the steps of the defect detection algorithm include an anomaly detection step performed using the third model.

[0129] For example, the model used by the defect detection algorithm may include a third model, and the folder involved may include a third type of folder for storing training sample images of the third model.

[0130] For example, the number of third models can be one or two, and the number of third-class folders can be one or two.

[0131] According to embodiments of the present invention, a third model can be used to detect whether an image to be identified is in an abnormal state. Exemplarily, the third model can output abnormal state information of the image. Exemplarily, the abnormal state information can be OK information indicating that the image is a normal image or NG information indicating that the image is an abnormal image.

[0132] For example, the third model can be any suitable model, as long as it can determine whether the image to be identified is a normal image, and the present invention does not limit it.

[0133] For example, the name of the third category folder corresponding to the third model may include the category identifier of the third model and / or the target identifier of the target information output by the third model. For example, and not limitingly, since the number of third models is relatively small, for simplicity, the name of the third category folder may only include the model category identifier or the target identifier. Optionally, the name of the third category folder may only include the model category identifier, such as "Outlier Detect Model". Alternatively, the name of the third category folder may only include the target identifier, such as "Yes / No". Of course, the name of the third category folder may also include a user-defined name, which is not limited by this invention.

[0134] According to an embodiment of the present invention, in the naming step S120, the third type of folder can be named according to a preset naming rule. Then, in the generation step S140, after receiving the user's confirmation operation, a flowchart of the defect detection algorithm can be generated and displayed based at least on the model category identifier and / or target identifier in the name of the third type of folder. It is easy to understand that the multiple steps shown in the flowchart of the defect detection algorithm can at least include an anomaly detection step performed by the third model to detect whether the image to be identified is an abnormal condition.

[0135] This solution is simple to operate, easy to implement, and requires a small number of models. It can generate effective defect detection algorithms while saving storage space and computing resources.

[0136] For example, the third model includes an anomaly detection model and / or a binary classification model, wherein the training sample images of the anomaly detection model include only normal images, and the training sample images of the binary classification model include labeled normal images and labeled anomaly images.

[0137] For example, both the anomaly detection model and the binary classification model can effectively detect whether an image to be identified is an anomaly. For example, the third model can include either or both of these models. It can be configured according to actual needs.

[0138] According to embodiments of the present invention, the training sample images for the anomaly detection model are normal sample images. Exemplarily, when the number of training sample images with defects is small, the third model preferably includes only the anomaly detection model. The anomaly detection model can be used to detect whether an image is normal, that is, it can determine whether there are any unknown or known types of defects in the image. In other words, as long as the image to be identified is abnormal, the anomaly detection model can detect it.

[0139] According to embodiments of the present invention, the training sample images for the binary classification model include: normal sample images labeled with normal information and defect sample images labeled with abnormal information. For example, when there are many training sample images of various types of defects, such as when the number is comparable to the number of normal sample images, the third model may only include the binary classification model. Optionally, when there are many training sample images of various types of defects, anomaly detection can be achieved by using both a binary classification model and an anomaly detection model.

[0140] Therefore, the advantages of anomaly detection models and binary classification models can be utilized to perform accurate detection. When detecting whether an image to be identified is abnormal based on the above two models, double-reassessment can be performed based on the two anomaly detection results, thereby further improving the detection accuracy.

[0141] For example, the generation step S140 includes: in response to the confirmation operation, generating and displaying a flowchart based at least on the model category identifier and / or target identifier in the names of the first type of folder and the third type of folder, wherein the defect detection algorithm further includes a post-processing step, in which the defect detection result of the image to be identified is determined based on the result of whether the defect region detected by the defect detection step and the image determined by the anomaly detection step is an abnormal condition; wherein, if the anomaly detection step determines that the image to be identified is an abnormal condition and the defect detection step determines that there is no defect region in the image to be identified, the defect detection result of the image to be identified is determined to be that there is a new type of defect region on the image to be identified.

[0142] For example, the models used by the defect detection algorithm may include a first model and a third model, and the folders may include a first type of folder and a third type of folder. In the naming step S120, the first type of folder and the third type of folder can be named according to a preset naming rule. In the generation step S140, after receiving the user's confirmation operation information, a flowchart of the defect detection algorithm can be automatically generated and displayed based on the names of the first type of folder and the third type of folder. It is easy to understand that this flowchart can show the operation path of multiple steps, including the defect detection steps performed by the first model and the anomaly detection steps performed by the third model.

[0143] Similarly, in order to obtain the final detection results, the flowchart may also include a post-processing step, in which the detection results of the first model and the third model are processed.

[0144] For example, the post-processing step may include: if the anomaly detection step determines that the image to be identified is in an abnormal state, and the defect detection step determines that there is no defective region in the image to be identified, then determining that the defect detection result of the image to be identified is that there is a new type of defective region in the image to be identified. For example, when the output of the third model is NG information, and the defect detection results of each first model indicate that there is no defective region in the image, it can be determined that the defect detection result of the image to be identified is that there is a new type of defective region in the image to be identified.

[0145] It's easy to understand that when a new type of defect appears in an image to be identified, the limited number of sample images makes it difficult to train a first model to identify this new type of defect. A third model, however, can effectively detect this problem by detecting suddenly appearing new types of defects. Therefore, defect detection is more comprehensive and the accuracy is higher.

[0146] If the anomaly detection step determines that the image to be identified is in an abnormal state, and the defect detection step determines that there is a defective area in the image to be identified, then the defect detection result of the image to be identified can be determined as the final detection result of the image to be identified.

[0147] If the anomaly detection step determines that the image to be identified is normal, and the defect detection step determines that there is a defective area in the image to be identified, then the defect detection result of the image to be identified can be determined as a detection error.

[0148] If the anomaly detection step determines that the image to be identified is normal, and the defect detection step determines that there are no defective areas in the image to be identified, then the defect detection result of the image to be identified can be determined as the image to be identified being normal.

[0149] In another example, the models used by the defect detection algorithm may include a first model, a second model, and a third model, and the folders may include first-class folders, second-class folders, and third-class folders. For example, in the naming step S120, the first-class folders, second-class folders, and third-class folders can be named according to preset naming rules. In the generation step S140, after receiving the user's confirmation operation information, a flowchart of the defect detection algorithm can be automatically generated and displayed based on the names of the first-class folders, second-class folders, and third-class folders. It is easy to understand that this flowchart can show the execution path of multiple steps, including the defect detection steps performed by the first model, the target area detection steps performed by the second model, and the anomaly detection steps performed by the third model. Similarly, this flowchart can also include a post-processing step, where the detection results of the three models can be processed to obtain the final defect detection result.

[0150] According to an embodiment of the present invention, in the post-processing step, the detection results of the first model and the second model can be post-processed first to obtain the first detection result, and then the final defect detection result can be determined based on the detection result output by the third model.

[0151] For example, when the third model outputs NG information, and the first detection result indicates that the image to be identified includes a defective region, the defect detection result of the image to be identified is determined to be a defective region on the image to be identified. For instance, when the third model outputs NG information, and the first detection result indicates that the image to be identified includes a defective region A with a certain defect, then the final defect detection result of the image to be identified can be determined to include information about the presence of defective region A with that defect in the image. In this scheme, the double verification of the presence of defective regions in the image to be identified results in a higher accuracy of the detection result.

[0152] For example, when the third model outputs NG information and the first detection result indicates that there is no defect region in the image, it can be determined that the defect detection result of the image to be identified is that there is a new type of defect region on the image to be identified.

[0153] For example, if the third model outputs an OK message, and the first detection result indicates that the image to be identified includes a defective region, then the defect detection result of the image to be identified can be considered an error. In this case, the cause of the error can be further verified manually. The model that erred can be identified by examining the image, and then that model can be further corrected. This approach helps to promptly identify problems with the model, thereby further ensuring the accuracy of the detection results.

[0154] For example, when the third model outputs an OK message and the first detection result indicates that the image to be identified does not contain a defect area, the final defect detection result can be considered as the object in the image to be identified being intact and free of defects.

[0155] According to the above scheme, a flowchart of the defect detection algorithm can be automatically generated based on the folder name through a simple folder naming operation. This flowchart includes at least the defect detection steps performed by the first model, the target area detection steps performed by the second model, the anomaly detection steps performed by the third model, and post-processing steps for the detection results from multiple models. On the one hand, this scheme is very simple and can intuitively present the execution path of the currently configured algorithm flow to the user, facilitating user verification and correction. On the other hand, since this algorithm, in addition to performing defect detection for common types of defects through the first and second models, also uses an anomaly detection model and / or a binary classification model to re-evaluate the detection results, it helps to generate defect detection algorithms with higher detection accuracy.

[0156] According to a second aspect of the present invention, a defect detection method is also provided. Figure 5 A schematic flowchart of a defect detection method 500 according to an embodiment of the present invention is shown. Figure 5 As shown, the defect detection method 500 may include the following steps S510, S520 and S530.

[0157] Step S510: Store the training sample images of the corresponding models in folders named according to preset naming rules. Each model is used to detect target information in the image to be identified, and the name of each folder includes the model category identifier and / or target identifier of the corresponding model.

[0158] For example, the folders can be folders in a preset path, and the names of each folder can be folders named according to preset naming rules. For example, the preset naming rules can be system-preset rules for the name content of each folder. For example, what identifier content should each folder name include at least, what optional letters or numbers can be used to represent each identifier content, the order of each identifier content, the character limit for each identifier content, etc.

[0159] For example, step S510 may be performed in response to an image storage operation performed by the user. For example, and not limitingly, the image storage operation performed by the user may include a copy operation where the user copies the training sample images of the model to each folder, or an upload operation where the user selects the folder and clicks the "Upload Training Sample Images" control on the interface. Of course, any other suitable image storage operation may also be used.

[0160] For example, each model can be any suitable model capable of performing defect detection-related tasks. For instance, it can be a deep learning-based neural network model, including but not limited to object detection models, semantic segmentation models, and many other types of models.

[0161] For example, the number of training sample images for each model can be any suitable number, such as 100 images, 1000 images, etc., and the present invention does not limit it.

[0162] For example, the training sample images for each model can be sample images with labeled target information or normal sample images without labeled target information. The training sample images differ between models. For example, the models can include the first model, second model, and third model in the aforementioned defect detection algorithm flowchart generation method 100. For instance, the training sample images for the first model can be images of defect regions labeled with at least one defect; the training sample images for the second model can be images of target regions labeled with a specific defect; and the training sample images for the third model can be normal images, or normal images labeled with normal information and abnormal images labeled with abnormal information.

[0163] For example, the target information can be any suitable information in the image to be identified that is related to defect detection, including but not limited to defect type information, defect state information, and defect-related location information. It is easy to understand that inputting the image to be identified into a model trained with training sample images can output the target information in the image to be identified.

[0164] Examples of model type identifiers and / or target identifiers included in the folder name are the same as those in the aforementioned method 100 for generating the defect detection algorithm flowchart, and will not be repeated here.

[0165] For example, in step S510, the training sample images of each classified model can be stored in the corresponding folder. For instance, the training sample images of different classified models may have been pre-stored in a certain storage path, and the user can copy and paste the training sample images of each model in that path to the corresponding folder. Alternatively, the training sample images of different models may use different naming conventions, and the system can automatically find the image files with the corresponding names and automatically store the training sample images of different models in the corresponding folders.

[0166] In step S520, in response to the user's storage confirmation operation, the model is automatically trained based on the training sample images and the model category identifier and / or target identifier in the name of each folder to obtain the trained model.

[0167] For example, step S510 above has stored the training sample images of different models into folders named according to preset naming rules under preset paths. For example, after the user confirms that all training sample images have been stored in the corresponding folders, the storage can be confirmed by clicking the storage confirmation control on the user interface, for example.

[0168] For example, prior to step S520, the defect detection method 500 may further include displaying the number of training sample images stored in each folder on a user interface. For example, a user can confirm whether storage is complete or if there are any storage errors by viewing the number of training sample images. For example, after the user confirms that storage is complete, they can click the storage confirmation control.

[0169] For example, after receiving the user's storage confirmation, the system can automatically read the name of each folder under the preset path. Since each folder name can include model category identifiers and / or target identifiers, key model information can be obtained through each folder name. For example, based on the key model information corresponding to each folder and other system-defined rules, the system can determine whether to use the training sample images in each folder to train a model such as an object detection model or a semantic segmentation model. For instance, if the folder name corresponding to the model includes "Detect Model," the training sample images in that folder are input into the object detection model, and the object detection model is automatically trained; if the folder name corresponding to the model includes "Segment Model," the training sample images in that folder are input into the semantic segmentation model, and the semantic segmentation model is automatically trained.

[0170] Step S530: Using the trained model, the defect detection algorithm is executed based on the running path shown in the flowchart generated by the flowchart generation method 100 of the above-mentioned defect detection algorithm.

[0171] For example, after training each model is completed through step S520, the user interface can display a "Training Complete" message and save the trained model file to a preset path. The defect detection algorithm is executed based on the trained model and the execution path shown in the flowchart generated by the aforementioned defect detection algorithm flowchart generation method 100, and the defect detection algorithm can be used to complete the defect detection of the image to be recognized.

[0172] According to the above technical solution, by storing training sample images of different models into corresponding folders named according to preset naming rules, model training can be automatically completed based on the folder names and preset rules, resulting in a trained model. Finally, the defect detection algorithm can be executed based on the trained model and the execution path of the defect detection flowchart, thereby achieving defect detection in the image to be recognized. This solution is simple to operate and has low user requirements, therefore it has strong applicability and a better user experience.

[0173] According to a third aspect of the present invention, a system for generating flowcharts of a defect detection algorithm is also provided. Figure 6 A schematic block diagram of a flowchart generation system for a defect detection algorithm according to an embodiment of the present invention is shown. As shown, the system 600 includes a naming module 610 and a generation module 620.

[0174] The naming module 610 is used to name at least one folder based on a preset naming rule, wherein each folder is used to store training sample images of the corresponding model, each model is used to detect target information in the image to be identified, and the name of each folder includes the model category identifier and / or target identifier of the corresponding model.

[0175] The generation module 620 is used to generate and display a flowchart of the defect detection algorithm in response to the user's confirmation operation, based on the model category identifier and / or target identifier in the name of each folder, wherein the flowchart shows the execution path of the steps of the defect detection algorithm.

[0176] According to a fourth aspect of the present invention, a defect detection system is also provided. Figure 7 A schematic block diagram of a defect detection system according to an embodiment of the present invention is shown. As shown, the defect detection system 700 includes a storage module 710, a training module 720, and a detection module 730.

[0177] The storage module 710 is used to store training sample images of the corresponding models in folders named according to a preset naming rule. Each model is used to detect target information in the image to be identified, and the name of each folder includes the model category identifier and / or target identifier of the corresponding model.

[0178] The training module 720 is used to automatically train the model based on the model category identifier and / or target identifier in the name of each folder in response to the user's storage confirmation operation, so as to obtain the trained model.

[0179] The detection module 730 is used to execute the defect detection algorithm using a trained model and based on the running path shown in the flowchart generated by the flowchart generation method 100 of the above-mentioned defect detection algorithm.

[0180] According to a fifth aspect of the present invention, an electronic device is also provided. Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. As shown, the electronic device 800 includes a display 810, a processor 820, and a memory 830, wherein the display 810 is used to display a user interface, and the memory 830 stores computer program instructions, which, when executed by the processor 820, are used to execute the flowchart generation method 100 and / or the defect detection method 500 described above.

[0181] According to a sixth aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the flowchart generation method 100 and / or the defect detection method 500 described above. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0182] Those skilled in the art can understand the specific implementation schemes of the flowchart generation method system, electronic devices, and storage media of the above-mentioned defect detection algorithm by reading the relevant description of the flowchart generation method. For the sake of brevity, they will not be described in detail here.

[0183] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0184] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0186] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0187] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0188] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0189] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0190] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the flowchart generation method system and defect detection algorithm of the defect detection algorithm according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0191] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0192] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of generating a flowchart of a defect detection algorithm, characterized by, The method comprises: a naming step of naming at least one folder based on a preset naming rule, wherein each of the folders is used to store training sample images of a corresponding model, different folders correspond to different models, each model is used to detect target information in a to-be-recognized image, and the name of each of the folders comprises a model category identifier and / or a target identifier of the corresponding model; the folders comprise first-type folders and second-type folders, each of the first-type folders is used to store training sample images of a first model corresponding thereto, each of the second-type folders is used to store training sample images of a second model corresponding thereto, each of the first models is used to identify a defect region of at least one defect in the to-be-recognized image, and each of the second models is used to identify a target region configured for at least one specific defect in the to-be-recognized image; the target identifier comprises a category identifier of a defect identified by the corresponding first model or an identifier of a target region of a specific defect identified by the corresponding second model; a generating step of generating and displaying a flowchart of a defect detection algorithm based on the model category identifier and / or the target identifier in the name of each of the folders in response to a confirmation operation of a user, wherein the flowchart shows a running path of steps of the defect detection algorithm.

2. The method of claim 1, wherein the generating step comprises: in response to the confirmation operation of the user, generating and displaying the flowchart of the defect detection algorithm based on the model category identifier and / or the target identifier in the name of the first-type folder, wherein the steps of the defect detection algorithm comprise a defect detection step performed by using the first model.

3. The method of claim 2, wherein the generating step comprises: in response to the confirmation operation, generating and displaying the flowchart based on the model category identifier and / or the target identifier in the name of each of the first-type folder and the second-type folder, wherein the steps of the defect detection algorithm further comprise a target region detection step performed by using the second model and a post-processing step, in which a defect detection result of the to-be-recognized image is determined based on a defect region detected by the defect detection step, a target region detected by the target region detection step, and a configuration relationship between the defect region and the target region. the name of the second-type folder or the first-type folder further comprises a configuration relationship identifier of the configuration relationship; the generating step comprises:

4. The method of claim 3, wherein, in response to the confirmation operation, reading the name of the first-type folder and the name of the second-type folder; based on the configuration relationship identifier in the name of the first-type folder or the second-type folder, acquiring the configuration relationship between the target region of each specific defect and the defect region of the defect; generating and displaying the flowchart of the defect detection algorithm based on the name of the first-type folder, the name of the second-type folder, and the configuration relationship. Before the generating step, the method further comprises: ​ 5. The method of claim 3, wherein, ​ obtaining a configuration relationship between a target region of each specific defect pre-configured by a user and a defect region of the specific defect; the generating step comprises: in response to the confirmation operation, generating and displaying a flowchart of the defect detection algorithm based on the name of the first type of folder, the name of the second type of folder, and the configuration relationship.

6. The method of generating a flowchart of a defect detection algorithm according to claim 3, wherein, The number of the first model and / or the second model is multiple.

7. The method of claim 3, wherein the method further comprises: The types of the first model include target detection models and semantic segmentation models; and / or The types of the second model include target detection models and semantic segmentation models, The target detection model is used to identify a defect region or a target region with a first morphological feature in the image to be identified, and the semantic segmentation model is used to identify a defect region or a target region with a second morphological feature in the image to be identified.

8. The method of claim 1 to 7, wherein, The folder includes a third type of folder, each of which is used to store training sample images of a corresponding third model, each of which is used to detect whether the image to be identified is in an abnormal condition, and the target information includes abnormal condition information. The generating step comprises: in response to the confirmation operation, generating and displaying the flowchart based on at least the model category identifier and / or the target identifier in the name of the third type of folder, wherein the steps of the defect detection algorithm include an anomaly detection step performed using the third model.

9. The method of claim 8, when dependent on claim 2, wherein, The generating step comprises: in response to the confirmation operation, generating and displaying the flowchart based on at least the model category identifier and / or the target identifier in the name of each of the first type of folder and the third type of folder, wherein the steps of the defect detection algorithm further include a post-processing step in which a defect detection result of the image to be identified is determined based on a defect region detected by the defect detection step and a result of whether the image to be identified is in an abnormal condition determined by the anomaly detection step. In the case where the anomaly detection step determines that the image to be identified is in an abnormal condition and the defect detection step determines that the image to be identified does not have a defect region, the defect detection result of the image to be identified is determined to be a new type of defect region on the image to be identified.

10. The method of claim 8, wherein, The third model includes an anomaly detection model and / or a binary classification model, wherein the training sample images of the anomaly detection model only include normal images, and the training sample images of the binary classification model include labeled normal images and labeled abnormal images.

11. The method of claim 1 to 7, wherein, The number of the folder is multiple, the folder includes a parent folder and a subfolder in each of the parent folder, wherein, The model categories of the models corresponding to the subfolders under each parent folder are the same; The model categories of the models corresponding to the subfolders under different parent folders are different, The name of each parent folder includes a model category identifier, and the name of a subfolder includes a target identifier.

12. The method of generating a flowchart of a defect detection algorithm according to any one of claims 1 to 7, characterized in that, Before the naming step, the method further comprises: in response to a user's creation operation, creating the folder; and / or in response to a user's deletion operation, deleting the folder.

13. A defect detection method characterized by, The method comprises: The training sample images of the models corresponding to the folders named based on the preset naming rules are stored in the folders, wherein different folders correspond to different models, each model is used for detecting target information in a to-be-identified image, and the name of each folder includes a model category identifier and / or a target identifier of the corresponding model; the folders include first type folders and second type folders, each first type folder is used for storing training sample images of a first model corresponding thereto, each second type folder is used for storing training sample images of a second model corresponding thereto, each first model is used for identifying a defect region of at least one defect in a to-be-identified image, and each second model is used for identifying a target region configured for at least one specific defect in the to-be-identified image; the target identifier includes a category identifier of a defect identified by the corresponding first model or an identifier of a target region of a specific defect identified by the corresponding second model; In response to a storage confirmation operation of a user, the models are trained based on the training sample images and the model category identifier and / or the target identifier in the name of each folder to obtain trained models; The trained models are used to execute a defect detection algorithm based on a running path shown in a flowchart generated by the method for generating a flowchart of a defect detection algorithm according to any one of claims 1 to 12.

14. A system for generating a flowchart of a defect detection algorithm, the system comprising: Comprise: A naming module is configured to name at least one folder based on a preset naming rule, wherein each folder is used for storing training sample images of a model corresponding thereto, different folders correspond to different models, each model is used for detecting target information in a to-be-identified image, and the name of each folder includes a model category identifier and / or a target identifier of the corresponding model; the folders include first type folders and second type folders, each first type folder is used for storing training sample images of a first model corresponding thereto, each second type folder is used for storing training sample images of a second model corresponding thereto, each first model is used for identifying a defect region of at least one defect in a to-be-identified image, and each second model is used for identifying a target region configured for at least one specific defect in the to-be-identified image; the target identifier includes a category identifier of a defect identified by the corresponding first model or an identifier of a target region of a specific defect identified by the corresponding second model; A generation module is configured to generate and display a flowchart of a defect detection algorithm based on a model category identifier and / or a target identifier in the name of each folder in response to a confirmation operation of a user, wherein the flowchart shows a running path of steps of the defect detection algorithm.

15. A defect detection system, characterized by, Comprise: a storage module, configured to store training sample images of a model in a folder named based on a preset naming rule, wherein different folders correspond to different models, each model is configured to detect target information in a to-be-identified image, and a name of each folder includes a model category identifier and / or a target identifier of the corresponding model; the folder includes a first type of folder and a second type of folder, each first type of folder is configured to store training sample images of a first model corresponding thereto, each second type of folder is configured to store training sample images of a second model corresponding thereto, each first model is configured to identify a defect region of at least one defect in a to-be-identified image, and each second model is configured to identify a target region configured for at least one specific defect in the to-be-identified image, and the target identifier includes a category identifier of a defect identified by the corresponding first model or an identifier of a target region of a specific defect identified by the corresponding second model; a training module, configured to automatically train the model based on the training sample images and the model category identifier and / or the target identifier in the name of each folder in response to a storage confirmation operation of a user, to obtain a trained model; a detection module, configured to execute a defect detection algorithm by using the trained model and based on a running path shown in a flowchart generated by the method for generating a flowchart of a defect detection algorithm according to any one of claims 1 to 12.

16. An electronic device comprising a display, a processor, and a memory, wherein, the display is configured to display a user interface, and the memory stores computer program instructions, which, when executed by the processor, are configured to execute the method for generating a flowchart of a defect detection algorithm according to any one of claims 1 to 12 and / or the defect detection method according to claim 13.

17. A storage medium, in which program instructions are stored, the program instructions, when executed, are configured to execute the method for generating a flowchart of a defect detection algorithm according to any one of claims 1 to 12 and / or the defect detection method according to claim 13.

Images