Modeling management method and device of detection algorithm, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a modeling management method and device for a detection algorithm, electronic equipment and a storage medium. The method comprises the steps that a modeling management page corresponding to a target algorithm formula of a product is displayed, the modeling management page comprises first editing controls of multiple detection units under the target algorithm formula, and the multiple detection units are in one-to-one correspondence with multiple product units on the product; in response to a trigger operation of a user on a first editing control of any target detection unit in the plurality of detection units, displaying an editing window of the target detection unit; and in response to an editing operation of a user in the editing window, setting a detection algorithm included in the target detection unit, the detection algorithm being used for detecting the target product unit corresponding to the target detection unit. According to the scheme, a user is allowed to carry out unified editing and management on detection algorithms of multiple product units on the same algorithm formula in a man-machine interaction mode, the configuration of the detection algorithms is more visual, and the modeling time can be greatly shortened.

Description

Technical Field

[0001] This invention relates to the field of product testing technology, and more specifically to a modeling and management method and apparatus for testing algorithms, electronic equipment, and storage medium. Background Technology

[0002] During or after product manufacturing, various inspections are required, such as detecting the location of predetermined features and checking for pre-defined defects. For example, the following describes the problems existing in the prior art using wafer defect detection as an example. It should be understood that wafers are merely an example; other similar products requiring inspection (such as LCD screen panels and lithium battery electrodes) also face the same issues. In semiconductor manufacturing processes, microscopic defects on the wafer surface directly affect chip yield and reliability. Wafer defect detection is a critical quality control step in semiconductor manufacturing, aiming to identify and locate minute defects on or inside the wafer surface to ensure chip yield and reliability. The wafer inspection process typically includes high-resolution imaging, image processing, and intelligent analysis. For example, high-contrast images of the wafer surface can be acquired, and defect features can be captured using line-scan or area-scan cameras under different lighting conditions. Subsequently, deep learning models or traditional image processing defect detection algorithms can be combined to classify and identify defects in the images.

[0003] In wafer inspection scenarios, it is necessary to design wafers of various specifications and dies of different sizes and structures. Other products also have similar issues, namely, complex product structures, and the same product may have different inspection targets (referred to as product units in this paper) that need to be inspected. In traditional solutions, when modeling inspection algorithms, the created algorithm formula only supports the inspection of a single product unit. Therefore, when the requirements are complex, multiple algorithm formulas need to be called in combination to achieve the desired results. This approach lacks effective management of algorithm formulas and cannot meet the needs of complex inspection scenarios. Summary of the Invention

[0004] The present invention is proposed in view of the above-mentioned problems. The present invention provides a modeling and management method for detection algorithms, a modeling and management device for detection algorithms, an electronic device, and a storage medium.

[0005] According to one aspect of the present invention, a modeling management method for a detection algorithm is provided. The method includes: displaying a modeling management page corresponding to a target algorithm formula for a product, the modeling management page including first editing controls for each of multiple detection units under the target algorithm formula, the multiple detection units corresponding one-to-one with multiple product units on the product; responding to a user's triggering operation on the first editing control of any target detection unit among the multiple detection units, displaying an editing window for the target detection unit; responding to the user's editing operation in the editing window, setting the detection algorithm included in the target detection unit, the detection algorithm being used to detect the target product unit corresponding to the target detection unit.

[0006] According to another aspect of the present invention, a modeling management device for a detection algorithm is also provided. The device includes: a first display module for displaying a modeling management page corresponding to a target algorithm formula of a product, the modeling management page including first editing controls for each of a plurality of detection units under the target algorithm formula, the plurality of detection units corresponding one-to-one with a plurality of product units on the product; a second display module for displaying an editing window for a target detection unit in response to a user's triggering operation on the first editing control of any target detection unit among the plurality of detection units; and a setting module for setting the detection algorithm included in the target detection unit in response to the user's editing operation in the editing window, the detection algorithm being used to detect the target product unit corresponding to the target detection unit.

[0007] According to another aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the modeling and management method of the detection algorithm described above.

[0008] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, wherein the program instructions are used to execute the modeling and management method of the detection algorithm described above when running.

[0009] According to embodiments of the present invention, a modeling management method and apparatus for detection algorithms, an electronic device, and a storage medium provide a user with a modeling management page corresponding to a target algorithm formula. This page includes first editing controls for each of multiple detection units under the target algorithm formula. Through the display of this page and the user's editing of any target detection unit among the multiple detection units using the first editing controls, the user can set the detection algorithm for any detection unit in the target algorithm formula. Each detection unit corresponds one-to-one with a product unit. This allows the user to uniformly edit and manage the detection algorithms for multiple product units under the same algorithm formula through human-computer interaction. When facing complex detection scenarios, a single algorithm formula can be invoked to detect complex product structures. Furthermore, this human-computer interaction scheme makes the configuration of detection algorithms for each product unit more intuitive, significantly shortening modeling time and effectively improving modeling management efficiency. Attached Figure Description

[0010] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0011] Figure 1 A schematic flowchart illustrating a modeling and management method for a detection algorithm according to an embodiment of the present invention is shown.

[0012] Figure 2 A schematic diagram illustrating a modeling management page according to an embodiment of the present invention;

[0013] Figure 3 A schematic diagram of a first editing window according to an embodiment of the present invention is shown;

[0014] Figure 4 A schematic diagram showing a second editing window for template matching functionality according to an embodiment of the present invention;

[0015] Figure 5 A schematic diagram illustrating a second editing window for binarization functionality according to an embodiment of the present invention;

[0016] Figure 6 A schematic diagram showing a second editing window for feature extraction functionality according to an embodiment of the present invention;

[0017] Figure 7 A schematic diagram showing a second editing window for a threshold extraction function according to an embodiment of the present invention;

[0018] Figure 8A schematic diagram showing a second editing window for the protection line extraction function according to an embodiment of the present invention;

[0019] Figure 9 A schematic diagram showing a preview window according to an embodiment of the present invention;

[0020] Figure 10 A schematic diagram showing a preview window according to another embodiment of the present invention;

[0021] Figure 11 A schematic diagram showing a preview window and an editing window according to an embodiment of the present invention is provided;

[0022] Figure 12 A schematic diagram of a first prompt window according to an embodiment of the present invention is shown;

[0023] Figure 13 A schematic diagram of a second prompt window according to an embodiment of the present invention is shown;

[0024] Figure 14 A schematic diagram of a third prompt window according to an embodiment of the present invention is shown;

[0025] Figure 15 The startup page of an algorithm modeling tool according to an embodiment of the present invention is shown;

[0026] Figure 16 A schematic block diagram showing a modeling and management apparatus for a detection algorithm according to an embodiment of the present invention; and

[0027] Figure 17 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0029] As mentioned above, in existing technologies, product structures are complex, and the same product may have different detection targets (referred to as product units in this paper) that need to be detected. The number of each type of product unit can be one or more. For example, on a wafer containing RGB light-emitting diode (LED) chips, there are a large number of red (R) LED chips, green (G) LED chips, and blue (B) LED chips. Red LED chips can be considered as the first product unit, green LED chips as the second product unit, and blue LED chips as the third product unit. It should be noted that the terms "first," "second," and "third," etc., used in this paper are for distinguishing purposes only and have no other special meaning. It is understood that the number of each of the first, second, and third product units on the wafer can be one or more, and commonly, very large numbers, such as tens of thousands. Each product unit has its own corresponding detection algorithm, which can be managed through algorithm recipes. In traditional solutions, when modeling the detection algorithm, a set of algorithm recipes is created that only supports the detection of a single product unit. Therefore, when the requirements are complex, multiple sets of algorithm recipes need to be called together to achieve the desired results. For example, regarding the aforementioned RGB LED chip, if the red, green, and blue LED chips are considered as three different product units, the traditional approach requires three sets of algorithm formulas. However, the detection system can only run one algorithm formula per run, meaning the system would need to scan and test three times to complete the inspection of the RGB LED chip. This approach significantly hinders the efficiency of the detection system. Furthermore, traditional detection algorithms do not provide a user-friendly interactive solution during modeling, allowing users (algorithm developers or product manufacturers, etc.) to visually manage and control the algorithm formulas. Therefore, traditional modeling solutions lack effective management of algorithm formulas and cannot meet the needs of complex inspection scenarios.

[0030] To at least partially address the aforementioned problems, embodiments of the present invention provide a modeling and management method for detection algorithms. This approach allows users to uniformly edit and manage detection algorithms for multiple product units using the same algorithm formula through human-computer interaction. When facing complex detection scenarios, a single algorithm formula can be invoked to perform simultaneous detection of multiple targets, enabling the scanning and detection of a large number of targets at once, such as scanning and detecting all products to be tested simultaneously. Furthermore, this human-computer interaction approach makes the configuration of detection algorithms for each product unit more intuitive, significantly shortening modeling time and effectively improving modeling and management efficiency.

[0031] Figure 1This document illustrates a schematic flowchart of a modeling and management method 100 for a detection algorithm according to an embodiment of the present invention. The modeling and management method 100 for a detection algorithm described herein can be applied to any electronic device with data processing capabilities and / or instruction execution capabilities, i.e., it is executed by an electronic device. This electronic device may include, but is not limited to, personal computers, servers, mobile terminals, etc. Exemplarily and not limitingly, the electronic device may include, for example, a detection system deployed at a product manufacturing site, i.e., the modeling and management method 100 for the detection algorithm is executed by the detection system deployed at the product manufacturing site. The detection algorithm described herein can be any type of detection algorithm, including but not limited to target detection algorithms for detecting the location of preset features of a product and / or defect detection algorithms for detecting whether a product has preset defects. The modeling and management method 100 for the detection algorithm can be an algorithm implemented through software functions, and the software implementing the method can be considered an algorithm modeling tool. The algorithm modeling tool can be nested within the main software of the overall product detection algorithm (e.g., a wafer inspection algorithm, i.e., an AOI algorithm), or it can be launched independently as an application program (APP). The above nesting is achieved by embedding the algorithm modeling tool within the main software of the detection algorithm through project references. In traditional detection algorithm design, the algorithm parameter tuning module is not independent and is entirely a function within the main detection algorithm software. However, the algorithm modeling tool according to embodiments of the present invention can be implemented independently as a separate class library, thus enabling it to be launched as an independent application. This algorithm modeling tool can manage a list of algorithm recipe projects and selectively maintain the modeling of multiple algorithm recipes. The algorithm modeling tool described in this paper supports the management of multiple detection units under a single algorithm recipe, which is more comprehensive compared to the traditional mode that only supports a single detection unit under a single algorithm recipe. Furthermore, the algorithm modeling tool described in this paper can simultaneously support the editing of multiple detection units. The settings of multiple detection units are clearly defined, independent yet interconnected (i.e., belonging to the same algorithm recipe), enabling the realization of complex requirements that traditionally require multiple algorithm recipes, such as the complex requirements of various wafer sizes, various die designs, and various detection scenarios in the field of wafer inspection. Moreover, the algorithm modeling tool described in this paper offers a more intuitive human-computer interaction and more streamlined data and business logic compared to traditional detection algorithm modeling processes, significantly reducing modeling time.

[0032] like Figure 1 As shown, the modeling management method 100 includes steps S110, S120 and S130.

[0033] Step S110: Display the modeling management page corresponding to the target algorithm formula of the product. The modeling management page contains the first editing control of each of the multiple detection units under the target algorithm formula. The multiple detection units correspond one-to-one with the multiple product units on the product.

[0034] The electronic device used in the modeling management method 100 for executing the detection algorithm may include an output device or a communication connection to the output device. The communication connection described herein may include any wired and / or wireless connection. The output device may include any type of display device, which may be touch-sensitive or non-touch-sensitive. The display device can display any content that needs to be displayed, such as the modeling management page corresponding to the target algorithm recipe of the product. The modeling management page may be displayed automatically when preset conditions are met, or it may be displayed in response to a user's page opening operation. For example, preset conditions may include, but are not limited to, power-on, completion of product image acquisition of the product under test, etc. The product described herein can be any type of product that needs to be tested, including but not limited to wafers, lithium battery electrodes, etc. For example, the target algorithm recipe may be one of at least one algorithm recipe, which may include a newly created algorithm recipe by the user and / or an algorithm recipe previously set by the user. The target algorithm recipe includes detection algorithms corresponding to various product units, and each product unit may be tested using one or more detection algorithms.

[0035] Figure 2 A schematic diagram of a modeling management page according to an embodiment of the present invention is shown. Exemplarily, as... Figure 2 As shown, the modeling management page can contain editing controls (i.e., the first editing controls) for each of the detection units U1, G1, and G2. Detection units U1, G1, and G2 correspond one-to-one with three different product units, such as the red LED chip, green LED chip, and blue LED chip mentioned above. Figure 2 The first editing control in the example is a button control, but this is just an example and can be implemented in other forms. For example, a drop-down menu control can be used to display the identification information of three detection units. Each identification information displayed in the drop-down menu can be regarded as the first editing control corresponding to the detection unit, and the user's selection of the identification information of the detection unit is the triggering operation of the first editing control of that detection unit.

[0036] For example, Figure 2The upper right corner of the modeling management page shown may display one or more of the following controls: maximize, minimize, and close. These controls are used to control the maximization, minimization, and closing of the modeling management page, respectively. Similarly, other windows and pages described herein, such as editing windows, may also have one or more of these controls. The settings and functions of each control can be understood by referring to the modeling management page; they will not be elaborated upon here. Exemplarily, but not restrictively, the modeling management page may include a display area for displaying global parameters. To distinguish it from other display areas described herein, this display area may be referred to as the ninth display area. Figure 2 As shown, the ninth display area 210 is marked with a dashed box. During the entire algorithm recipe detection process, some parameters are applicable to all detection units and all detection regions under all detection units, eliminating the need for individual settings for each detection unit or region. Therefore, global parameters can be set, which are shared by all detection algorithms in the target algorithm recipe; that is, the parameters of each detection algorithm in the target algorithm recipe can include global parameters. For example... Figure 2 As shown, global parameters can include an enable indicator parameter to indicate whether the GPU is enabled. Figure 2 The parameter shown is named "Enable GPU" and is an alignment indicator parameter used to indicate whether subpixel alignment is enabled. Figure 2 The parameter shown is named "Subpixel Alignment," and the style indicator parameter is used to indicate the defect display style. Figure 2 The parameter names shown are "Defect Display Style," etc. It should be noted that the number and types of various parameters shown in the accompanying figures are merely examples; for example, Figure 2 The number and type of the global parameters shown can vary and are not limited to this. Figure 2 The example shown. Exemplarily, the modeling management method may further include: determining the parameter values ​​of the aforementioned at least some global parameters in response to a user's setting operation on parameter values ​​of at least some global parameters in the ninth display area. Global parameters not set by the user can be default parameter values. Specific parameters for each detection algorithm can be displayed and set in the parameter panel of the fifth display area 320 described below. Exemplarily, and not limitingly, the modeling management page may also display a parameter preview control 220, which the user can use to input a preview command to open the preview window described below. It can be understood that clicking... Figure 2 The first editing control of the detection unit shown, or clicking the parameter preview control, will pop up a new window for display.

[0037] In step S120, in response to the user's trigger operation on the first editing control of any of the multiple detection units, the editing window of the target detection unit is displayed.

[0038] The electronic device used in the modeling management method 100 for executing the detection algorithm may include an input device or be communicatively connected to an input device. Exemplarily, the input device may include, but is not limited to, one or more of a mouse, keyboard, touchpad, touchscreen, trackball, microphone, etc. Users can perform various operations through the input device, i.e., input various instructions and / or information. That is, the various user instructions and / or information described herein can be input in any input method, including but not limited to input via triggering annotation controls on the user interface (UI), input via shortcut keys, input via voice, etc. It is understood that the aforementioned user interface is an interactive interface capable of displaying information; the modeling management page and various windows (such as editing windows) described herein can all be displayed on the user interface. Similarly, the various user operations described herein can also be implemented in any operation method; when a user performs any operation, they input the corresponding instruction (or operation instruction) and / or information. That is, the operations performed by the user may include, but are not limited to, triggering annotation controls on the user interface (UI), triggering shortcut keys, providing voice commands, etc.

[0039] When a user triggers (e.g., clicks) the first editing control of any one of multiple detection units (referred to as the target detection unit for convenience), the editing window of the target detection unit can be displayed accordingly. For example, the editing window may include the first editing window. Figure 3 A schematic diagram of a first editing window according to an embodiment of the present invention is shown. Figure 3 As shown, the first editing window displays various parameters of the detection algorithm, specifically, for example... Figure 3 The parameters in the "Threshold Settings" panel are displayed in the right-hand area. Users can edit various parameters of the detection algorithm in the first editing window to set the detection algorithm for the target detection unit.

[0040] Step S130: In response to the user's editing operation in the editing window, the detection algorithm included in the target detection unit is set. The detection algorithm is used to detect the target product unit corresponding to the target detection unit.

[0041] A target detection unit is any detection unit under a target algorithm recipe, used to detect the target product unit corresponding to that target detection unit. Each of some or all detection units under a target algorithm recipe can serve as a target detection unit, allowing the user to configure the detection algorithms contained within that unit. Each detection unit under a target algorithm recipe can include one or more detection algorithms, such as one or more different defect detection algorithms. Editing operations can be performed via keyboard shortcuts, control triggers, etc. Through editing operations, the detection algorithms contained in the target detection unit can be set, specifically the type and / or parameters of the detection algorithm. A target product unit is the product unit corresponding to the target detection unit, such as a red LED chip. It can be understood that, through the modeling management page, users can configure the detection algorithms contained in some or all of multiple detection units to set the detection algorithms required for the product unit corresponding to those units during detection. The algorithm recipes described in this document, including but not limited to target algorithm recipes, can support two-dimensional (2D) and / or three-dimensional (3D) detection of products. 2D and 3D detection each have their own corresponding detection algorithms. The modeling management method 100 allows for the configuration of detection algorithms for 2D and / or 3D detection.

[0042] The above technical solution provides users with a modeling management page corresponding to the target algorithm formula. This page contains first editing controls for each of the multiple detection units under the target algorithm formula. Through the display of this page and the user's editing of any target detection unit within the multiple detection units using these first editing controls, users can configure the detection algorithm for any detection unit in the target algorithm formula. Each detection unit corresponds one-to-one with a product unit. This allows users to uniformly edit and manage the detection algorithms for multiple product units under the same algorithm formula through human-computer interaction. When facing complex detection scenarios, a single algorithm formula can be invoked to detect complex product structures. Furthermore, this human-computer interaction solution makes the configuration of detection algorithms for each product unit more intuitive, significantly shortening modeling time and effectively improving modeling management efficiency.

[0043] According to an embodiment of the present invention, the editing window includes a first editing window, which displays a first product image containing a target product unit of a standard product. In response to a user's editing operation in the editing window, setting the detection algorithm included in the target detection unit includes: in response to a user's region setting operation on the first product image, setting a detection region corresponding to the target detection unit, each detection region including at least one region of interest on the target product unit; for each detection region, in response to the user's algorithm setting operation, determining the detection algorithm used by the detection region; wherein, the detection algorithm included in the target detection unit includes the detection algorithm used by each detection region corresponding to the target detection unit, the editing operation includes a region setting operation and an algorithm setting operation, and the detection algorithm used by each detection region is used to detect each region of interest included in the detection region.

[0044] See also Figure 3 The first product image, containing the target product unit of the standard product, is displayed in the middle area of ​​the first editing window. The standard product can be a product that meets normal product requirements, such as being defect-free and having an image clarity that meets requirements. Since the detection algorithm in the algorithm formula needs to be set to detect the product, the first product image of the standard product can be used as a reference to accurately determine the detection area of ​​similar products and configure a suitable detection algorithm for the detection area. If the product has defects, its product image may have problems such as contamination, occlusion, or burrs, making it difficult to accurately determine the detection area based on the product image, and also difficult to accurately set the type and / or parameters of the detection algorithm. The first product image can be a preset image or can be set by the user. The first product image contains the target product unit of the standard product. The first product image can be the entire product image of the standard product, or a portion of the entire product image of the standard product, as long as it contains the target product unit of the standard product. Displaying an image area instead of the entire product image allows the user to view the information of the detection area more clearly. For example, as shown... Figure 3As shown, the first product image may include one or more of the following: a standard golden image, a bright golden image, and a dark golden image of a standard product. In response to the user's image switching operation, the user can switch between the golden image, the bright golden image, and the dark golden image, using the switched image as the first product image. On this first product image, the user can determine the region of interest (including the first region of interest and / or the second region of interest described herein) and highlight the determined region of interest, allowing the user to choose an image suitable for observation to view the region of interest and adjust the settings information of the detection area. The image switching operation may include control triggering operations, shortcut key input operations, etc. For example, an image switching control may be displayed in the first editing window. The image switching control can be any type of control, including but not limited to button controls, drop-down menu controls, etc. Figure 3 The image switching controls include Golden Image, Bright Golden Image, and Dark Golden Image. Figure 1 There are three corresponding button controls. Responding to the user's trigger operation on the image switching control, it can switch between Golden image, bright Golden image and dark Golden image. This switching can be performed at any time in response to the user's operation, and the image switched to each time is used as the first product image. Figure 3 The product shown is an LED chip. The diagram shows three LED chips: a red LED chip, a green LED chip, and a blue LED chip. Each LED chip can be considered a product unit. Figure 3One of the product units is marked with a white solid line box. Of course, this is just an example; these three LED chips can also be considered the same product unit. That is, the division of product units can be set as needed; each discrete component on the product can be treated as a separate product unit, or multiple discrete components can be treated as a single product unit. Each product unit can include one or more detection areas (or sub-regions, represented by Zone). Any two detection areas of the same product can partially overlap or not overlap at all. For each detection area, one or more detection algorithms can be used for detection. When the detection algorithm is a defect detection algorithm, each defect detection algorithm can detect one type of defect and obtain a corresponding defect location. Therefore, the defect detection result of each detection area can include one or more defect locations. Each defect location can specifically include one or more non-contiguous image areas, which are determined by the defect detection algorithm to contain the defect it is looking for, such as particles, scratches, dents, etc. Any two detection algorithms corresponding to the same detection area can be executed independently, or one detection algorithm can be executed after the first one has been executed, with the second detection algorithm continuing to execute based on the former's detection result. Similarly, detection algorithms corresponding to any two different detection regions can be executed independently, or, after the detection algorithm for one detection region has been executed, the detection algorithm for the other detection region can continue to be executed based on the detection results of the former. Each detection region may include at least one region of interest on the target product unit. The detection area (ROI) can be understood as the target area to be detected. It can be divided according to product structure and detection requirements. For example, the entire surface of a red LED chip can be considered as one detection area (called a Surface Zone), the left pad area of ​​the red LED chip as another detection area, and the right pad area of ​​the red LED chip as yet another detection area. In this embodiment, the detection areas corresponding to the target detection unit can be set in response to the user's area setting operation. For example, the user can create any number of area identifiers for the detection areas, such as Zone1, Zone2, Zone3, etc., and set the location of the corresponding detection area for each area identifier. Various identifiers described herein, such as area identifiers, component identifiers, image identifiers, unit identifiers, etc., can be represented by any type of identifier information, including but not limited to one or more of number, name, and icon. The electronic device used for the modeling management method to execute the detection algorithm can store the user-set area identifiers and locations together as the detection area. Exemplarily, the entire detection area can be detected, or only the region of interest (ROI) within the detection area can be detected. Detecting only the ROI can effectively reduce workload. A region of interest (ROI) can be understood as the area of ​​focus within a detection region. For each ROI, the user can configure the detection algorithm used, which can then be applied to each ROI contained within that ROI.

[0045] By adopting the above technical solution, users can set the detection area corresponding to the target detection unit and the detection algorithm used for each detection area based on the first product image. This allows the target detection unit to be divided into different detection areas, and the detection algorithm can be applied to the region of interest of each detection area. This enables more detailed and accurate modeling of the detection algorithm for the local area of ​​each detection unit. This solution helps to realize the detection of the areas of focus for users and can effectively reduce the amount of computation.

[0046] According to an embodiment of the present invention, in response to a user's region setting operation on a first product image, setting a detection region corresponding to a target detection unit includes: for each detection region corresponding to the target detection unit, in response to a user's region of interest setting operation on the first product image, determining the position of each region of interest included in the detection region; wherein, the region setting operation includes a region of interest setting operation.

[0047] For each detection region, the user can set the positions of the corresponding regions of interest (ROIs). ROI setting can be implemented in any way, such as manually drawing the ROIs by the user or determining the ROIs using a region extraction function based on user-inputted extraction information.

[0048] By employing the above technical solution, the location of each region of interest (ROI) within the detection area can be determined in response to the user's setting of the ROI for the first product image. This allows users to customize the ROI for each detection area, enabling them to define the location of the desired detection region and better meet their personalized needs.

[0049] According to an embodiment of the present invention, the first editing window includes a fourth display area, which includes detection area components corresponding to each detection area of ​​the target detection unit. Each detection area component is used to display the area identifier of the corresponding detection area and the type of detection algorithm used by the detection area. In response to the user's area setting operation for the first product image, the detection area corresponding to the target detection unit is set. The method further includes: in response to the user's identifier setting operation in the detection area component, determining the area identifier of the detection area corresponding to the detection area component. The area setting operation further includes an identifier setting operation.

[0050] Continue to refer to Figure 3 In this diagram, the fourth display area 310 is marked with a dashed box. For example... Figure 3 As shown, the fourth display area 310 may include one or more detection area components, each corresponding to a detection area. Within each detection area component, an area identifier corresponding to the detection area may be displayed. For example, the component identifier of the detection area component may also be displayed, such as... Figure 3 The numbers “1” and “2” are shown in the image. Figure 3 The system displays two detection area components. The upper component, numbered 1 (which can be called detection area component 1), corresponds to the surface detection area, and its zone name is set to "Surface Zone". The lower component, numbered 2 (which can be called detection area component 2), is obtained by copying detection area component 1, so its zone name is similar to that of detection area component 1, "Surface Zone-copy". The zone name can be set by the electronic device by default or by the user through any input method. The detection area component can also display the type of detection algorithm used for the corresponding detection area (i.e., algorithm type). This type can be set by the electronic device by default or by the user through any input method. Figure 3As shown, detection area component 1 contains an algorithm of type "Surface-(Conventional Surface Defect Detection)". Detection area component 2 is obtained by copying detection area component 1, therefore its algorithm type is the same as that of detection area component 1. Note that... Figure 3 The region identifiers and algorithm types shown for the detection areas are merely examples and not limitations of the invention. The specific region names and algorithm types for each detection area can be adjusted and set as needed. For example... Figure 3 As shown, the region identifier and algorithm type have corresponding text box controls and menu bar controls, respectively. Users can set the region identifier and algorithm type through the text box controls and menu bar controls.

[0051] The detection region component is used to identify and manage detection regions. For example, after a new detection region component is generated by creating, copying, or otherwise, the region identifier and / or algorithm type in the detection region component can be set.

[0052] By employing the above technical solution, detection area components for each detection region can be displayed in the first editing window. Users can determine the region identifier of each detection region through the identifier setting operation within the detection area component, facilitating the differentiation and management of each detection region. Furthermore, the detection area component also displays the type of detection algorithm used by the detection region. The display of region identifiers and algorithm types makes it easier for users to understand and identify each detection region and manage it.

[0053] According to an embodiment of the present invention, in response to a user's algorithm setting operation, determining the detection algorithm used by the detection area includes: in response to a user's type setting operation, determining the type of the detection algorithm used by the detection area; performing a parameter setting operation, the parameter setting operation including: a first operation, or a first operation and a second operation, or a second operation and a third operation, or a first operation, a second operation, and a third operation; wherein, the first operation includes: based on the determined type, determining the parameter values ​​of at least some parameters of the detection algorithm used by the detection area from a preset parameter library, the preset parameter library storing default parameter values ​​of at least some parameters of each of at least one type of detection algorithm; the second operation includes: based on the determined type, displaying a parameter panel in a fifth display area of ​​a first editing window, the parameter panel being used to display the parameter values ​​of the detection algorithm used by the detection area; the third operation includes: in response to a user's parameter value setting operation on at least some parameters in the parameter panel, determining a new parameter value of the parameters set by the parameter setting operation; wherein, the algorithm setting operation includes a type setting operation and a parameter setting operation.

[0054] The user can set the algorithm type in any way, including but not limited to selecting it from the menu bar control corresponding to the algorithm type in the detection area component. For example, the user can set the detection algorithm type through keyboard shortcuts, text box controls, voice input, or other methods without selecting it from the detection area component. Of course, the menu bar control corresponding to the algorithm type in the detection area component can also be replaced with other types of controls, such as text box controls, and the user can set the algorithm type through the replaced control.

[0055] After determining the type of detection algorithm, its corresponding parameters can be set. The execution method of parameter setting is arbitrary and can be understood by referring to the description of user operation implementation above; it will not be repeated here. In one embodiment, the parameter setting operation may include a first operation. The first operation includes: based on the determined type, determining the parameter values ​​of at least some parameters of the detection algorithm used in the detection area from a preset parameter library. The preset parameter library stores default parameter values ​​for at least some parameters of at least one type of detection algorithm. The preset parameter library can be used to store default parameter values ​​for at least some parameters of at least one type of detection algorithm. For example, the preset parameter library may store default parameter values ​​for at least some parameters of defect detection algorithm A, defect detection algorithm B, and defect detection algorithm C. When the user sets the algorithm type to defect detection algorithm B, the default parameter values ​​corresponding to defect detection algorithm B can be determined from the preset parameter library and used as the parameter values ​​for at least some parameters of the detection algorithm currently set by the user. For any detection algorithm, after determining the parameter values ​​of at least some parameters corresponding to the detection algorithm from the preset parameter library, the user can directly adopt or further adjust these parameter values ​​as needed. Furthermore, for any detection algorithm, if the preset parameter library only stores default parameter values ​​for some parameters, the electronic device used to execute the modeling management method 100 for the detection algorithm can autonomously set and / or respond to the user's parameter value setting operation to set the parameter values ​​for the remaining parameters. This autonomous setting can include, for example, random setting. Using the above technical solution, through the first operation, the parameter values ​​for at least some parameters can be automatically determined from the preset parameter library based on the type of detection algorithm, thus helping to reduce the workload of the user in setting parameters.

[0056] In one embodiment, the parameter setting operation may include a first operation and a second operation. The first operation includes: determining, based on the determined type, the parameter values ​​of at least some parameters of the detection algorithm used in the detection area from a preset parameter library, wherein the preset parameter library stores default parameter values ​​of at least some parameters of each of at least one type of detection algorithm. The second operation includes: based on the determined type, displaying a parameter panel in a fifth display area of ​​a first editing window, wherein the parameter panel is used to display the parameter values ​​of the detection algorithm used in the detection area.

[0057] The first operation can be understood by referring to the description above, and will not be repeated here. The second operation displays the parameter values ​​of each parameter of the detection algorithm in the first editing window. When the first operation is performed, the first editing window displays the parameter values ​​of at least some of the parameters of the detection algorithm determined from the preset parameter library. For parameters not stored in the preset parameter library, blank parameter values ​​or parameter values ​​set by the electronic device itself can be displayed in the parameter panel. See also... Figure 3 The fifth display area 320 is indicated by a dashed box. This fifth display area 320 includes a parameter panel, which is labeled "Threshold Setting" in the first editing window. Figure 3 As shown, the parameter panel displays the parameter values ​​of each parameter of the detection algorithm used in the current detection area. Figure 3 The parameter panel displays the parameter values ​​in a list format, with each row representing a parameter. The left side shows the parameter name, and the right side shows the parameter value. Parameter values ​​can include, but are not limited to, the numerical value of the corresponding parameter, the mode value of the corresponding parameter, and a status value indicating whether the corresponding parameter is enabled or disabled. For example, for "bright defect threshold," the parameter value can be a value within a preset range; for "threshold mode," the parameter value can be the mode value; and for "hole filling," the parameter value can be either "hole filling enabled" or "hole filling disabled." This parameter display scheme allows users to easily view and understand the parameter status of the detection algorithm used in each detection area, assisting them in quickly and accurately managing modeling. For example, in response to the user's selection of a detection area, the fifth display area can display the parameter values ​​of the detection algorithm used in the currently selected detection area. The user's selection of a detection area can include, but is not limited to, selecting the aforementioned detection area components, or selecting the detection area via shortcut key input. The fifth display area can display the parameter values ​​of the detection algorithm used in the currently selected detection area in real time. As the user selects a detection area, the parameters in the parameter panel can also change accordingly.

[0058] In one embodiment, the parameter setting operation may include a second operation and a third operation. The second operation includes: displaying a parameter panel in a fifth display area of ​​the first editing window based on the determined type. The parameter panel is used to display the parameter values ​​of the detection algorithm used in the detection area. The third operation includes: determining new parameter values ​​for the parameters set by the parameter setting operation in response to the user's parameter setting operation in the parameter panel; wherein the algorithm setting operation includes a type setting operation and a parameter setting operation.

[0059] The second step can be understood by referring to the description above, and will not be repeated here. See also Figure 3 Users can set the values ​​of all or some parameters in the parameter panel of the fifth display area 320, such as by entering values ​​through text boxes or selecting status values ​​through checkboxes. The parameter values ​​displayed in the parameter panel can be any value, such as blank values, default values ​​determined from a preset parameter library, parameter values ​​set by the electronic device itself, and parameter values ​​previously set by the user. Regardless of previous parameter settings, users can adjust some or all parameter values ​​of the detection algorithm through the parameter panel to obtain new parameter values ​​when needed. This technical solution allows users to set the parameter values ​​of at least some parameters of the detection algorithm through the parameter panel, giving users a high degree of freedom in setting parameters and facilitating personalized algorithm parameter settings to better meet the needs of different users.

[0060] In one embodiment, the parameter setting operation may include a first operation, a second operation, and a third operation. The first operation includes: determining, based on the determined type, the parameter values ​​of at least some parameters of the detection algorithm used by the detection region from a preset parameter library, wherein the preset parameter library stores default parameter values ​​for at least some parameters of each of at least one type of detection algorithm. The second operation includes: based on the determined type, displaying a parameter panel in a fifth display area of ​​a first editing window, the parameter panel being used to display the parameter values ​​of the detection algorithm used by the detection region. The third operation includes: in response to the user's parameter value setting operation on the parameter panel, determining new parameter values ​​for the parameters set by the parameter setting operation; wherein the algorithm setting operation includes a type setting operation and a parameter setting operation.

[0061] The implementation methods and corresponding technical effects of the first, second, and third operations can be referred to the description above, and will not be repeated here. In this embodiment, after determining the parameter values ​​of at least some parameters of the detection algorithm from the preset parameter library through the first operation, the determined parameter values ​​can be displayed through the parameter panel. The user can further set at least some parameter values ​​in the parameter panel as needed. The parameter to which the user-set parameter value belongs can be partially the same as or partially different from the parameter to which the parameter value determined through the preset parameter library belongs, or they can be completely the same or completely different. In this way, after the parameter values ​​of the detection algorithm are determined and displayed based on the preset parameter library, the user can further set the parameter values, which reduces the user's workload while also taking into account the user's personalized setting needs.

[0062] According to an embodiment of the present invention, the first editing window includes a fourth display area, which contains detection area components corresponding to each detection area of ​​the target detection unit. Each detection area component is used to display the area identifier of the corresponding detection area and the type of detection algorithm used by the detection area. The type setting operation is a setting operation performed by the user in the detection area component.

[0063] The display method and contents of the detection region component have been described above and will not be repeated here. In this embodiment, users can set the type of detection algorithm in the detection region component. This allows the region identifier and algorithm type of the detection region to be managed through a single detection region component. Compared with other independent algorithm type setting methods, this scheme, which sets the algorithm type through the detection region component, helps users accurately determine the algorithm type corresponding to each detection region.

[0064] According to an embodiment of the present invention, the fourth display area further includes one or more of the following controls: a creation control, an import control, an export control, a copy control corresponding to each detection area, and a delete control corresponding to each detection area; in response to the user's editing operation in the editing window, setting the detection algorithm included in the target detection unit, and further including: in response to the user's triggering operation on the creation control, creating a new detection area component; in response to the user's triggering operation on the import control and the selection operation on the setting information file of any detection unit stored in the preset file library, converting the setting information file of the detection unit into setting information, and importing the converted setting information into the first editing window for display, wherein the preset file library is used to store the setting information file of the detection unit, the setting information file contains the setting information of the detection unit, and the setting information includes the area identifier of each detection area of ​​the detection unit and each detection of the detection unit. The type of detection algorithm used in the region and the parameter values ​​of the detection algorithm; in response to the user's trigger operation on the export control, the setting information of the target detection unit is converted into a setting information file in a preset format and the setting information file is exported to a preset file library for storage; in response to the user's trigger operation on the delete control corresponding to any detection region, the detection region and the detection region component corresponding to the detection region are deleted; in response to the user's trigger operation on the copy control corresponding to any detection region, the detection region component corresponding to the detection region is copied to generate a new detection region component in the fourth display area; in response to the user's drag operation on any detection region component, the arrangement order of the detection region component and other detection region components is adjusted; the editing operation includes trigger operations and / or drag operations on one or more of the following controls: create control, import control, export control, delete control, copy control.

[0065] See Figure 3 At the bottom of the fourth display area 310, a button control 311 is displayed. This button control 311 is a creation control. In response to a user's triggering action (e.g., clicking) on ​​the creation control, a new detection area component can be created in the fourth display area 310. For example... Figure 3 As shown, the detection region components can be displayed in a list format (labeled "Zone List" in the first editing window). Newly created detection region components can be added and displayed below the currently displayed last detection region component, following the list's order. This approach allows users to create new detection region components as needed, thereby creating new detection regions accordingly. See also... Figure 3 Each detection area component can also include a delete control 312. In response to a user's triggering of the delete control (e.g., clicking), the corresponding detection area and detection area component can be deleted. This solution allows users to delete unwanted detection area components and detection areas at any time, facilitating adjustments to the detection area. See also... Figure 3 Each detection area component may also include a copy control 313. In response to a user's triggering (e.g., clicking) operation on the copy control corresponding to any detection area, the detection area component corresponding to that detection area can be copied to generate a new detection area component in the fourth display area. As described above, detection area component 2 is obtained by copying detection area component 1. In this scheme, in addition to the detection area components, the parameters of the detection algorithm used by the detection area can also be copied synchronously. That is, in response to a user's triggering (e.g., clicking) operation on the copy control corresponding to any detection area, the parameter values ​​of the detection algorithm used by that detection area can be copied to the parameters of the detection algorithm used by the newly generated detection area. This scheme supports information copying between different detection areas within the same detection unit, facilitating users to quickly set up new detection areas.

[0066] See Figure 3The fourth display area 310 may also contain an import control 314. In response to a user's triggering of the import control (e.g., clicking), a file open dialog box (File Chooser / File Picker Dialog) can pop up. The file open dialog box can link to the storage path of a preset file library, allowing the user to select the setting information file of any detection unit stored in the preset file library. In response to the user's selection, the setting information file of the selected detection unit can be converted into setting information, and the converted setting information can be imported into the first editing window of the current target detection unit for display. Importing the converted setting information into the first editing window can include generating a new detection area component in the fourth display area based on the converted setting information. That is, the detection area components corresponding to each detection area contained in the user-selected detection unit can be imported into the fourth display area and displayed as new detection area components. For example, the preset file library stores setting information files for detection units U1, U2, and U3. The currently displayed first editing window is the first editing window for the target detection unit G1. The user selects the setting information file for the detection unit U2 from the preset file library. Since the detection unit U2 contains two detection areas, two new detection area components can be generated in the fourth display area of ​​the currently displayed first editing window based on the two detection areas of the detection unit U2. The area identifiers contained in these two area components can be the same as the area identifiers of the two detection areas of the detection unit U2, or they can be sorted together with the previously set detection areas of the target detection unit G1 to generate new area identifiers. For example, if at least some detection areas of the target detection unit have a preset detection order (i.e., the detection algorithms used by the detection areas have a preset execution order), and the arrangement order of the detection area components in the fourth display area represents the detection order of each detection area of ​​the target detection unit, the setting information of the detection unit can also include the detection order of each detection area of ​​the detection unit. In this case, when the converted setting information is imported into the first editing window for display, detection area components arranged in the same order as the detection order can also be generated in the fourth display area based on the detection order of the detection areas contained in the setting information. It is understandable that the setting information of the detection unit may also include the type of detection algorithm used in each detection area of ​​the detection unit and the parameter values ​​of the detection algorithm. When the converted setting information is imported into the first editing window for display, the type of detection algorithm and the parameter values ​​of the detection algorithm can be displayed in the corresponding display positions. For example, the type of detection algorithm can be displayed as follows: Figure 3The parameter values ​​of the detection algorithm in the "Algorithm Type" text box of the detection area component shown can be displayed in the parameter panel of the fifth display area 320. This scheme supports importing the setting information of all detection areas under other detection units into the current detection unit for setting, which helps to improve the efficiency of modeling management.

[0067] See Figure 3 The fourth display area 310 may also include an export control 315. In response to a user's triggering of the export control (e.g., clicking), a "Save As Dialog" dialog box can pop up. The "Save As" dialog box can link to the storage path of a preset file library, allowing users to export the settings information file obtained by converting the settings information of the target detection unit to the preset file library for storage. This approach supports the export of settings information for all detection areas under the entire detection unit, helping users quickly copy the settings information of the detection unit to other algorithm formulations or other detection units of the same algorithm formulation for convenient setup.

[0068] For example, in response to a user's dragging operation on any detection area component, the arrangement order of that detection area component relative to other detection area components is adjusted. The dragging operation can be performed using an input device such as a mouse or touchpad. As described above, any two detection areas can be detected independently of each other, or they can be detected in a preset detection order, with the later detection area being detected based on the detection result of the current detection area. When the detection areas have a preset detection order, the arrangement order of the detection area components can represent the detection order of the detection areas. In this case, allowing the user to adjust the arrangement order between the detection area components through dragging operations enables the user to conveniently adjust the detection order of the detection areas.

[0069] By employing the above technical solution, and providing controls for creating, importing, and exporting detection areas, as well as copying and deleting controls corresponding to each detection area, and responding to drag-and-drop operations, users can quickly create, delete, rearrange, import, and export information for detection area components or detection areas. This operation method is convenient, fast, highly interactive, and has low operational complexity. Furthermore, this free adjustment of detection areas is not supported by traditional modeling workflows. The embodiments of this invention, through the above technical solution, can effectively support and extend functions not supported by traditional modeling workflows.

[0070] According to an embodiment of the present invention, a first editing window includes a first display area, in which a first product image and a first drawing control are displayed. The first drawing control is used to set a first drawing tool, which includes a first type of drawing tool and / or a second type of drawing tool. The area inside a graphic drawn using the first type of drawing tool is a selected area, and the area outside a graphic drawn using the second type of drawing tool is a selected area. In response to a user's operation to set a region of interest (ROI) for the first product image, determining the positions of each ROI included in the detection area includes: in response to a user's trigger operation on the first drawing control, determining the first drawing tool selected by the user; and in response to a user's first drawing operation on the first product image, determining the positions of the first ROIs included in the detection area according to the graphic specified by the first drawing tool. The ROI setting operation includes a trigger operation on the first drawing control and a first drawing operation.

[0071] For example, the first display area may include a first sub-display area for displaying a first product image and a second sub-display area for displaying a first drawing control. See also Figure 3 The second sub-display area 330 is marked with a dashed box. The first sub-display area is the area containing the image in the dark part of the center of the first editing window. It should be noted that the positions of various display areas, pages, windows, and controls described in this document can be arbitrarily set as needed and are not limited to the positions shown in the attached figures. The first drawing control is used to set the first drawing tool, which is used to determine the region of interest. The first drawing tool may include a first type of drawing tool, or a second type of drawing tool, or both. The area inside a graphic drawn using a first type of drawing tool is the selected area, and the area outside a graphic drawn using a second type of drawing tool is the selected area. Either the first type of drawing tool or the second type of drawing tool can be a drawing tool of any shape, including but not limited to various regular or irregular shapes, such as circles, rectangles, arbitrary polygons, etc. Figure 3As shown, the second sub-display area 330 displays a "Basic" control and a "Shield" control, used to set the first type of drawing tools and the second type of drawing tools, respectively. The "Basic" control can set the first type of drawing tools, which includes three types: circle, rectangle, and arbitrary polygon. For example, if the user selects the circle tool, when the user drags the cursor on the first product image using an input device such as a mouse, a circle of corresponding size will be drawn based on the cursor's movement distance; the area inside this circle is the selected area. The "Shield" control can set the second type of drawing tools, which also includes three types: circle, rectangle, and arbitrary polygon. For example, if the user selects the circle tool, when the user drags the cursor on the first product image using an input device such as a mouse, a circle of corresponding size will be drawn based on the cursor's movement distance; the area outside this circle is the selected area. The descriptions of rectangle and arbitrary polygon are similar to those for the circle tool and will not be repeated here. From the above description, it can be understood that when using the second type of drawing tools, the area in the user's frame is the shielded area, and the area outside this shielded area is the actually selected area. Conversely, with the first type of drawing tools, the area in the user's frame is the actually selected area.

[0072] In response to a trigger operation on the first drawing control and a first drawing operation on the first product image, the system determines the position of the first region of interest (ROI) within the current detection area based on the graphic specified by the first drawing tool selected by the user. By providing the first drawing control, the system allows users to manually draw the position of the ROI, resulting in a more accurate position that better meets the user's needs.

[0073] According to an embodiment of the present invention, the target detection unit includes at least some detection algorithms, each corresponding to at least one region extraction function. Different region extraction functions are used to extract regions of interest according to different extraction methods. In response to a user's region of interest setting operation for a first product image, determining the position of each region of interest included in the detection area includes: performing one or more extraction operations, each extraction operation corresponding to a region extraction function. The extraction operation includes: in response to a user's function selection operation, determining the region extraction function corresponding to the current extraction operation and displaying a second editing window for the region extraction function. The editing window also includes a second editing window. In response to a user's information setting operation for extraction information in the second editing window, determining the extraction information for the region extraction function. The extraction information is used to indicate one or more of the following information for the region of interest: extraction parameters, extraction range, extraction center, extraction template, and extraction features. Based on the extraction information, according to the extraction method of the region extraction function corresponding to the current extraction operation, extracting a second region of interest included in the detection area from the first product image to determine the position of the second region of interest. The region of interest setting operation includes one or more extraction operations.

[0074] The target detection unit includes detection algorithms that correspond to at least one region extraction function. Different region extraction functions are used to extract regions of interest (ROIs) according to different extraction methods. Each detection algorithm in all or some of the detection algorithms included in the target detection unit can correspond to one or more region extraction functions. For any two different types of detection algorithms, the corresponding region extraction functions can be partially the same and partially different, or all the same or all different. For example, defect detection algorithm A can correspond to template matching and binarization functions, which can be used to extract the ROI of the detection area corresponding to defect detection algorithm A; defect detection algorithm B can correspond to binarization and feature extraction functions, which can be used to extract the ROI of the detection area corresponding to defect detection algorithm B; defect detection algorithm C can correspond to a protection line extraction function, which can be used to extract the ROI of the detection area corresponding to defect detection algorithm C. Figure 3 As shown, the detection area currently selected by the user is the detection area corresponding to detection area component 2, where the bounding box of detection area component 2 is highlighted. The algorithm type corresponding to detection area component 2 is "Surface-(Conventional Surface Defect Detection)", which has two region extraction functions: template matching and binarization.

[0075] One or more extraction operations can be performed, each corresponding to a region extraction function. The region extraction functions corresponding to any two extraction operations can be the same or different. For example, for the same detection region, two or more extraction operations can be performed using the same region extraction function, or multiple extraction operations can be performed using different region extraction functions. The region extraction functions corresponding to one or more extraction operations can include all region extraction functions corresponding to all detection algorithms included in the target detection unit, or they can include some region extraction functions corresponding to all detection algorithms included in the target detection unit. That is, all region extraction functions corresponding to all detection algorithms included in the target detection unit can be used to perform extraction operations, or only some can be used to perform extraction operations, and each region extraction function can be used to perform at least one extraction operation.

[0076] The function selection operation can be performed in any way, such as by control triggering or shortcut key input, as long as the region extraction function corresponding to each extraction operation can be determined. In response to the user's function selection operation, a second editing window for the region extraction function can be displayed. For example, at least one region extraction function includes one or more of the following: template matching function, binarization function, guard line extraction function, feature extraction function, and threshold extraction function. Figure 4-8 The second editing window shows five different region extraction functions according to embodiments of the present invention. Figure 4 A schematic diagram of a second editing window for a template matching function according to an embodiment of the present invention is shown. Figure 5 A schematic diagram showing a second editing window for binarization functionality according to an embodiment of the present invention. Figure 6 A second editing window is shown, illustrating the feature extraction function according to an embodiment of the present invention. Figure 7 A schematic diagram of a second editing window for a threshold extraction function according to an embodiment of the present invention is shown. Figure 8 A schematic diagram of a second editing window illustrating the protection line extraction function according to an embodiment of the present invention is shown.

[0077] The extraction operation may include: responding to the user's information setting operation for extraction information in the second editing window, determining the extraction information of the region extraction function, and extracting the second region of interest (ROI) included in the detected region from the first product image according to the extraction method of the region extraction function, thereby determining the location of the second ROI. The extraction information is used to indicate one or more of the following information for the ROI: extraction parameters, extraction range, extraction center, extraction template, and extraction features. Extraction parameters control the specific extraction process of the region extraction function when extracting the ROI, such as the region determination method and / or various thresholds. The type of extraction parameters depends on the type of region extraction function. For example, for the template matching function, its extraction parameters may include the type of matching method used for template matching, the minimum similarity score based on template matching, and the angle range of template matching. The extraction range is the location range of the ROI; the ROI can fall within this extraction range, i.e., the region extraction function extracts the ROI within this extraction range. The extraction template is the template used by the template matching function when extracting the ROI. The extraction features are the features used by the feature extraction function when extracting the ROI. The extraction center is the center used by the guard line extraction function when extracting the ROI. This invention provides embodiments that cover various region extraction functions, including 3D detection, for different products. The region extraction function automatically extracts repetitive or difficult-to-draw regions of interest (ROIs) using algorithms. When setting ROIs for a detection area, some scenarios cannot be manually drawn or are inconvenient to do so. For example, a single die product may have dozens of bumps (bumps or pad bumps). These bumps can be ROIs; drawing them manually one by one would be extremely labor-intensive. However, using extraction, a template can be drawn, and all bumps can be extracted. Exemplarily, each extraction operation may also include: after extracting a second ROI, highlighting the second ROI on the first product image in the second editing window of the corresponding region extraction function and / or on the first product image in the first editing window to visually display the extraction results. Exemplarily, after performing one or more extraction operations on the current detection area, all second ROIs extracted in one or more extraction operations can be highlighted on the first product image in the first editing window. Exemplarily, the second editing window can be displayed above the first editing window (i.e., at a higher display level). In this scenario, the second editing window may partially or completely obscure the first product image in the first editing window. The first product image can be displayed by dragging or closing the second editing window to view the highlighted area of ​​interest. Of course, it's understood that the above display scheme is merely an example; displaying the second editing window on the same layer (i.e., at the same display level) as the first editing window is also possible.The algorithmic modeling tool according to embodiments of the present invention allows for independent and separate maintenance of each region extraction function, facilitating clearer and more intuitive display and use of the functions. This decoupling significantly improves data logic and interactivity. Furthermore, the algorithmic modeling tool allows users to deeply design and plan the extraction of regions of interest involved in modeling, enabling independent management of the extraction information for each region extraction function. It also allows for in-depth editing and maintenance, addressing the pain point of traditional modeling management models where users cannot edit the extraction information of regions of interest at all.

[0078] In summary, by performing one or more extraction operations, the corresponding second region of interest can be automatically obtained. On products such as wafers, the number of regions of interest can be very large, their shapes may be irregular, and their areas may be very small, making drawing them very difficult. If all of these regions are drawn manually by the user, the workload will be enormous. However, the region extraction function according to embodiments of the present invention can achieve automated extraction of regions of interest. This automated extraction scheme can significantly reduce the user's workload, effectively improve the efficiency of determining regions of interest, help to broaden the application scenarios of modeling management methods, and solve the pain point that users cannot edit the extracted information of regions of interest under traditional modeling management models.

[0079] In one embodiment, only a first drawing control is provided, meaning the user is only allowed to manually draw the region of interest. In another embodiment, the region of interest is only allowed to be extracted using a region extraction function. In yet another embodiment, both a first drawing control and a region extraction function can be provided simultaneously, allowing the user to choose between manual drawing and / or extraction using the region extraction function to determine the region of interest as needed. In this embodiment, the finally determined region of interest can be entirely drawn by the user, entirely extracted by the region extraction function, or partially drawn by the user and partially extracted by the region extraction function.

[0080] According to an embodiment of the present invention, a second editing window displays a first product image and a second drawing control. The second drawing control is used to set a second drawing tool, which includes a first type of drawing tool and / or a second type of drawing tool. In response to a user's information setting operation on the second editing window, the extraction information for the region extraction function is determined, including: in response to a user's trigger operation on the second drawing control, determining the second drawing tool selected by the user; in response to a user's second drawing operation on the first product image in the second editing window, determining the extraction range of the region of interest according to the graphic specified by the second drawing tool; wherein the information setting operation includes a trigger operation on the second drawing control and a second drawing operation, and the second editing window also displays parameter values ​​for the extraction parameters of the region extraction function, where each parameter is a default parameter value or a user-set parameter value.

[0081] For example, the second editing window can display an image of the first product. See also Figure 4-8 The second editing window can display the first product image containing the target product unit. Figure 4-8 The first product image shown contains three LED chips. As mentioned above, these three LED chips can be considered as three different product units or as a single product unit. Exemplarily, the second editing window can also display a second drawing control. The second drawing control is used to set a second drawing tool, which is used to determine the extraction range. See also... Figure 4 The display area 420 of the second drawing control is marked with a dashed box. Figure 5 , 7 -8 also shows the second drawing control, and the display position of the second drawing control is the same as... Figure 4 Similarly, this will not be elaborated further. Like the first drawing tool, the second drawing tool can include the first type of drawing tool, or include the second type of drawing tool, or include both. The interior of a graphic drawn using the first type of drawing tool is the selected area, and the exterior of a graphic drawn using the second type of drawing tool is the selected area. The second drawing tools corresponding to any two different area extraction functions can be the same or different. Figure 4-7 In the middle, both "Search" and "Block" controls are shown, which are the second drawing controls. Figure 4-7 The "search" control in Figure 3 Similar to the "Basic" controls shown, the tools set are first-class drawing tools, such as circles, rectangles, and arbitrary polygons. If the user selects the circle tool through the "Search" control, when the user drags the cursor on the first product image using an input device such as a mouse, a circle of corresponding size can be drawn according to the distance the cursor moves. The area inside this circle is the selected area. Figure 4-7 The "block" control in the middle and Figure 3 Similar to the "mask" control shown, the tool set is a second-type drawing tool, such as a circle, rectangle, or arbitrary polygon. If the user selects the circle tool through the "mask" control, when the user drags the cursor on the first product image using an input device such as a mouse, a circle of corresponding size can be drawn based on the cursor's movement distance. The area outside this circle is the selected area. Therefore, in response to the user's selection of the second drawing tool through the second drawing control and combined with the user's second drawing operation on the first product image, the extraction range of the region of interest can be determined. When extracting the region of interest, it can be extracted within this range. Different region extraction functions have different extraction methods. When using any region extraction function to extract the region of interest, extraction can be performed according to the corresponding extraction method, such as template matching, binarization, or feature extraction.

[0082] For example, the second editing window can also display extraction parameters with corresponding region extraction functions. Figure 4 The display area 410 for the extracted parameters is marked with a dashed box. This area is shown as the "Extraction Parameter Settings" area in the second editing window. Figure 5-8 The display area for the extracted parameters can be found in the reference. Figure 4 Understanding, this is not explicitly indicated in the diagram. From Figure 4-8 It can be seen that each region extraction function can have corresponding extraction parameters. The extraction parameters of any region extraction function can use all or part of the default parameter values, or be set entirely or partially by the user, allowing the user to adjust previously set or unset parameter values ​​at any time. The method for determining the parameter values ​​of this extraction function can refer to the method for determining the parameter values ​​of the detection algorithm described above, and will not be elaborated here. Based on the extraction information, and according to the extraction method of the region extraction function corresponding to this extraction operation, the steps for extracting the second region of interest included in the detected region from the first product image can include: based on the extraction parameters of the corresponding region extraction function and the determined extraction range, extracting the second region of interest from the first product image according to the extraction method of the region extraction function corresponding to the second editing window.

[0083] Using the above technical solution, a first product image and a second drawing control can be displayed on the second editing window, facilitating the user's extraction of the required range when drawing the second region of interest using the second drawing control. This solution can extract the region of interest based on the user-defined extraction range, thus effectively reducing the computational load of region extraction while also catering to the user's personalized region extraction needs.

[0084] According to embodiments of the present invention, at least one region extraction function includes one or more of the following: template matching function, binarization function, feature extraction function, and threshold extraction function.

[0085] For example, for the template matching function, the corresponding second editing window also displays a third drawing control. The third drawing control is used to set a third drawing tool, which includes a first type of drawing tool and / or a second type of drawing tool. In response to the user's information setting operation on the extracted information in the second editing window, the extraction information of the region extraction function is determined. The third drawing control also includes: in response to the user's trigger operation on the third drawing control, determining the third drawing tool selected by the user; in response to the user's third drawing operation on the first product image in the second editing window, determining the extraction template on which the region of interest is based according to the graphic specified by the third drawing tool. The information setting operation also includes the trigger operation on the third drawing control and the third drawing operation. The second region of interest is the region whose matching degree with the extraction template meets the requirements of the extraction parameters of the template matching function and falls within the extraction range of the template matching function.

[0086] See Figure 4 In display area 420, a "Template" control is also displayed. In this document's editing window, "template" is referred to as "template"; note that the two have the same meaning. This "Template" control is the third drawing control, used to set the third drawing tool. This third drawing tool is used to determine the extraction template upon which the region of interest is extracted. Similar to the first and second drawing tools, the third drawing tool can also include the aforementioned first-type and / or second-type drawing tools. The usage of the third drawing control is similar to that of the first and second drawing controls. Refer to the above description to understand the meaning and usage of the third drawing control and the third drawing tool; further details are omitted here. Figure 4 The "Template" control shown is similar to the "Search" control; that is, the "Template" control is mainly used to set the first type of drawing tool. However, it should be noted that this is only an example, and the third drawing tool can also include the second type of drawing tool or include both the first and second type of drawing tools. After the user selects the third drawing tool, such as the rectangle tool, they can use this tool to draw on the first product image. The area selected by the user through the third drawing tool and the third drawing operation is the extraction template (or template area). After determining the extraction template, on the first product image, within the extraction range corresponding to the template matching function, the area where the degree of matching between the search and the extracted template meets the specified extraction parameters corresponding to the template matching function can be used as the second region of interest. For example, as shown... Figure 4 As shown, the extracted parameters can include the matching method used in template matching, i.e., the template method type. For example, the matching method used in template matching can be such as Normalized Cross-Correlation (NCC). For example, as... Figure 4As shown, the extraction parameters may also include the minimum similarity score and / or angle range of template matching. The minimum similarity score is the similarity threshold corresponding to template matching. If the score exceeds this threshold, it is determined that the matching degree between the currently participating region and the extraction template meets the requirements, and the two are considered to be matched. The angle range is the range of angles that are allowed to be rotated during the matching process based on the extraction template. Those skilled in the art can understand the meaning of the minimum similarity score and angle range of template matching, and will not be elaborated here. By adopting the above scheme and providing a third drawing control, users can customize the extraction template on which the template matching function is based. In this way, users only need to determine any image region on the first product image, and the electronic device can use the image region determined by the user as the extraction template to automatically search and determine the image region in the first product image that matches the extraction template as the second region of interest. In this way, users only need to perform a few operations to achieve the extraction of a large number of similar image regions on the product image according to the template set by the user. The template matching function works well when applied to the region of interest extraction of products such as Pad Bump.

[0087] For example, for the binarization function, the second region of interest is the region extracted according to the extraction parameters of the binarization function and falling within the extraction range of the binarization function.

[0088] For example, such as Figure 5 As shown, for the binarization function, the extraction parameters can include binarization method, minimum gray value, maximum gray value, etc. Those skilled in the art will understand the meaning of each extraction parameter of the binarization function, and will not elaborate further here. By using binarization, the first product image can be binarized, and the region that meets the threshold requirements in the extraction parameters can be determined as the second region of interest. In many detection scenarios, the shape of the region to be detected is very irregular, making it difficult to select using a regular shape. In such cases, binarization can be used to quickly extract these irregular regions, and this binarization extraction method has a wide range of applications.

[0089] For example, for the feature extraction function, the corresponding second editing window also displays a fourth drawing control. The fourth drawing control is used to set the fourth drawing tool. The third drawing tool includes a first type of drawing tool and / or a second type of drawing tool. In response to the user's information setting operation on the extracted information in the second editing window, the extraction information of the region extraction function is determined. It also includes: in response to the user's trigger operation on the fourth drawing control, determining the fourth drawing tool selected by the user; in response to the user's fourth drawing operation on the first product image in the second editing window, determining the extraction features on which the region of interest is extracted based according to the graphics specified by the fourth drawing tool; wherein, the information setting operation also includes the trigger operation on the fourth drawing control and the fourth drawing operation, and the second region of interest is the region whose matching degree with the extracted features meets the requirements of the extraction parameters of the feature extraction function and falls within the extraction range of the feature extraction function.

[0090] For example, such as Figure 6 As shown, for the feature extraction function, extraction parameters may include Pad width tolerance, Pad height tolerance, Pad extraction grayscale threshold, etc. Those skilled in the art will understand the meaning of each extraction parameter, and this article will not elaborate further. Feature extraction is similar to template matching; it searches for regions that match the user-defined extraction features as secondary regions of interest. For example... Figure 6 As shown, the second editing window can also display a "Features" control, which is the fourth drawing control used to set the fourth drawing tool. The fourth drawing is used to determine the extracted features. Similar to the first and third drawing tools, the fourth drawing tool can also include the aforementioned first-type and / or second-type drawing tools. The usage of the fourth drawing control is similar to that of the first, second, and third drawing controls. The meaning and usage of the fourth drawing control and the fourth drawing tool can be understood by referring to the description above; further details are omitted here. Figure 6 The "Feature" control shown is similar to the "Search" and "Template" controls; that is, the "Feature" control is mainly used to set the first type of drawing tool. However, it should be noted that this is only an example, and the fourth drawing tool can also include the second type of drawing tool or include both the first and second type of drawing tools. After the user selects any image area on the first product image through the fourth drawing tool and the fourth drawing operation, the electronic device can automatically extract features from the user-selected image area (i.e., extract features) and search for image areas whose image features match the extracted features as the second region of interest. This scheme can be applied to areas with relatively obvious features, such as solder pads, and can quickly and accurately locate the second region of interest.

[0091] For example, for the threshold extraction function, the second region of interest is the region extracted according to the extraction parameters of the threshold extraction function and falling within the extraction range of the threshold extraction function.

[0092] For example, such as Figure 7 As shown, for the threshold extraction function, the extraction parameters may include target extraction color, target extraction shape, whether the die height is uniform, etc. Those skilled in the art will understand the meaning of each extraction parameter of the threshold extraction function, and will not elaborate further here. Based on the extraction parameters of the threshold extraction function, the extraction range of the threshold extraction function (e.g., the user through...) can be determined. Figure 7 Within the range set by the "Search" control and / or "Block" control shown, extract the second region of interest. The threshold extraction function is primarily a measurement-type algorithm that allows for the extraction of regions of interest that meet specific requirements through flexible threshold settings.

[0093] According to an embodiment of the present invention, at least one region extraction function includes a guard line extraction function; for the guard line extraction function, the corresponding second editing window further displays a fifth drawing control, the fifth drawing control is used to set a fifth drawing tool, the border of the graphic drawn by the fifth drawing tool represents the extraction center, in response to the user's information setting operation on the extraction information in the second editing window, the extraction information of the region extraction function is determined, and the function further includes: in response to the user's trigger operation on the fifth drawing control, determining the fifth drawing tool selected by the user; in response to the user's fifth drawing operation on the first product image in the second editing window, determining the extraction center on which the region of interest is based according to the graphic specified by the fifth drawing tool; wherein, the information setting operation further includes the trigger operation on the fifth drawing control and the fifth drawing operation, and the second region of interest is the region extracted according to the extraction parameters of the guard line extraction function and located around the extraction center of the guard line extraction function.

[0094] The protection line extraction function can be applied in wafer inspection or similar fields. For example, such as... Figure 8 As shown, for the guard line extraction function, the extraction parameters may include measurement range, measurement width, gradient threshold, etc. Those skilled in the art will understand the meaning of each extraction parameter for the guard line extraction function, and will not elaborate further here. The guard line can be a wafer dicing guard line, that is, a series of special patterns or structures (such as rows of regular, dense small squares or lines) made around the edge of each chip within the wafer dicing channel. Guard lines can be used as the boundary separating different chips. In some cases, it is necessary to inspect the condition within the wafer dicing channel, such as its defects. To facilitate the identification of the dicing channel, a portion of the area around the guard line can be extracted to obtain the dicing channel. For example... Figure 8As shown, the second editing window can display a "border" control, also known as the fifth drawing control. Through this control, the user can draw the position of the border, which can represent a guard line, referred to as the extraction center in this paper. The second region of interest can be determined by expanding outwards from the user-defined border to the distance and range specified by the extraction parameters. The guard line extraction function can extract regions of interest such as cutting lines, enabling the extraction of relatively long and narrow regions of interest.

[0095] According to a first embodiment of the present invention, the first editing window further includes a second display area, which is used to display a function selection control, and the function selection operation is a triggering operation on the function selection control.

[0096] See back Figure 3 The second display area 340 is indicated by a dashed box. See also... Figure 3 The second display area 340 displays function selection controls. The current detection algorithm corresponds to template matching and binarization functions. Accordingly, the function selection controls include "Template Matching" button controls and "Binarization Function" button controls. Of course, Figure 3 The button control shown is just an example; the function selection control can also be other controls such as a menu bar control. Users can select any region extraction function corresponding to the current detection algorithm using any type of function selection control. In response to the user's triggering action on the function selection control, a second editing window for the selected region extraction function can pop up. This solution, which displays the corresponding region extraction function's editing window through a function selection control, offers strong interactivity, simple operation, and helps to further improve the user experience.

[0097] According to a second embodiment of the present invention, the second editing window further displays parameter values ​​of extraction parameters for the region extraction function. In response to the user's information setting operation on the extraction information in the second editing window, the extraction information of the region extraction function is determined, including: in response to the user's setting operation on the parameter values ​​of at least some extraction parameters in the second editing window, determining new parameter values ​​for at least some extraction parameters, wherein the information setting operation includes setting the parameter values ​​of at least some extraction parameters.

[0098] like Figure 4-8 As shown, the second editing window can also display the parameter values ​​of the extraction parameters corresponding to the region extraction function. Each extraction parameter can correspond to an input method such as a text box control. Users can set the parameter values ​​of at least some of the extraction parameters in the second editing window through any suitable input method such as a text box control or keyboard shortcuts. This technical solution allows users to set the extraction parameters themselves, providing them with greater freedom and enabling them to adjust the extraction parameters according to their actual needs, thus achieving more personalized region of interest extraction.

[0099] According to a third embodiment of the present invention, the second editing window further displays an extraction control. The step of extracting the second region of interest included in the detection area from the first product image according to the extraction information and the extraction method of the region extraction function corresponding to the current extraction operation is executed in response to the user's trigger operation on the extraction control.

[0100] like Figure 4-8 As shown, the second editing window displays an "Extract ROI" control, which is the extraction control. In this window, users can repeatedly set the extraction information for the region extraction function, such as adjusting extraction parameters and drawing the extraction range. During the setting process, users can trigger (click) the "Extract ROI" control at any time. After each click, the system responds to the user's click action, extracting the region based on the current extraction information, and optionally displays the extraction results, allowing the user to continue setting extraction information or confirm the extraction completion. This approach responds to the user's triggering of the extraction control to perform the extraction of the second region of interest, offering strong human-computer interaction and allowing users to easily control the progress of the extraction operation.

[0101] According to the fourth embodiment of the present invention, the second editing window further displays a completion control, and the extraction operation further includes: in response to the user's triggering operation on the completion control, determining that the current extraction operation is completed and saving the extraction record corresponding to the current extraction operation, the extraction record including the region extraction function used in the extraction operation, the extraction information on which it is based, and the extraction result obtained.

[0102] like Figure 4-8 As shown, the second editing window displays a "Save ROI Results" control, which is the completion control. Before triggering (clicking) the "Save ROI Results" control, the user can freely adjust extraction parameters, draw the extraction range, and other extraction information. After setting the various extraction information required for the region extraction function, the user can click the "Save ROI Results" control. In response to the user's click on the "Save ROI Results" control, the electronic device confirms the completion of this extraction operation and saves the corresponding extraction record. Using the triggering of an independent completion control as the completion marker for the extraction operation allows users to set extraction information at any time until the set extraction information meets the user's requirements. This helps users avoid problems such as improper extraction information settings. Furthermore, this solution can automatically save the extraction record corresponding to each extraction operation for easy review later.

[0103] The first, second, third, and fourth embodiments described above can be implemented by only one of them, or any two or three of them can be combined in the same embodiment.

[0104] According to an embodiment of the present invention, when an extraction control and a completion control are displayed in the second editing window, after the step of extracting the second region of interest included in the detection area from the first product image according to the extraction information and the extraction method of the region extraction function corresponding to the current extraction operation, the extraction operation further includes: displaying a supplementary drawing control in the second editing window, the supplementary drawing control being used to set supplementary drawing tools, the supplementary drawing tools including a first type of drawing tool and / or a second type of drawing tool, the inside of the graphic drawn using the first type of drawing tool being the selected area, and the outside of the graphic drawn using the second type of drawing tool being the selected area; in response to the user's triggering operation on the supplementary drawing control, determining the supplementary drawing tool selected by the user; in response to the user's supplementary drawing operation on the first product image in the second editing window, determining the supplementary second region of interest according to the graphic specified by the supplementary drawing tool; and adding the supplementary second region of interest to the extraction result of the current extraction operation.

[0105] As described above, when the extraction control is displayed in the second editing window, the step of extracting the second region of interest from the first product image based on the determined extraction range and according to the extraction method of the region extraction function corresponding to the second editing window can be executed in response to the user's trigger operation on the extraction control. Therefore, in this embodiment, after extracting the second region of interest from the first product image in response to the user's trigger operation on the extraction control, a supplementary drawing control can be added to the second editing window. Of course, optionally, the supplementary drawing control can also be displayed before extracting the second region of interest from the first product image. In this case, the supplementary drawing control can be set to an inactive mode, that is, prohibited from being triggered by the user. After extracting the second region of interest from the first product image, the supplementary drawing control can be set to an active mode, that is, allowed to be triggered by the user.

[0106] The supplementary drawing control is used to set supplementary drawing tools. Similar to the first, second, third, and fourth drawing tools, supplementary drawing tools can also include the first type of drawing tools and / or the second type of drawing tools mentioned above. The usage of the supplementary drawing control is similar to that of the first, second, third, and fourth drawing controls described above. The meaning and usage of the supplementary drawing control and supplementary drawing tools can be understood by referring to the description above; it will not be repeated here. For example, the supplementary drawing control can be displayed as shown in... Figure 4 The display area 420 shown is the same area as one or more of the second, third, fourth, and fifth drawing controls, making it convenient for users to view quickly.

[0107] Based on the user-selected supplementary drawing tools and operations, the user-added second region of interest (ROI) can be determined and added to the extraction results of this extraction operation. Since the ROI automatically extracted by the region extraction function may be incomplete or inaccurate, supplementary drawing controls are provided to allow users to fill in any gaps in the automatically extracted ROI, resulting in a more accurate and complete ROI.

[0108] According to an embodiment of the present invention, the first editing window includes a third display area, which displays an extraction management panel. The extraction management panel is used to display extraction record components. Each extraction record component corresponds to an extraction operation and is associated with and bound to the extraction record of the corresponding extraction operation. The extraction record includes the region extraction function used in the extraction operation, the extraction information on which it is based, and the extraction result obtained. The modeling management method of the detection algorithm further includes: after each extraction operation, adding the extraction record component corresponding to the current extraction operation to the extraction management panel. The extraction record component displays the corresponding region extraction function. The extraction record component also displays one or more of the following: a second editing control, a deletion control, and a visibility control. The second editing control is used to respond to the user's operation, display the corresponding extraction record, and modify at least some information in the extraction record other than the region extraction function to redetermine the second region of interest. The deletion control is used to respond to the user's operation, delete the extraction record component and the associated extraction record, and the visibility control is used to respond to the user's operation, control the display of the extraction result in the extraction record associated with the extraction record component on the first product image.

[0109] See back Figure 3 The third display area 350 is marked with a dashed box. For example... Figure 3 As shown, the third display area 350 displays the extraction management panel. Figure 3 The editing window is shown as the "ROI Extraction Management" panel. The Extraction Management panel can be a collapsible panel, meaning it can be collapsed or expanded in response to user actions. Similarly, the parameter panel mentioned above can also be a collapsible panel. Figure 3In the diagram, the extraction management panel is collapsed, while the parameter panel is expanded. Those skilled in the art will understand the implementation of the extraction management panel, and will not elaborate further here. For example, an extraction record component corresponding to each extraction operation can be added to the extraction management panel. Similar to the region detection component, the extraction record component can be displayed in a list format. The extraction record component can display one or more pieces of information, such as the region extraction function corresponding to the extraction record, and can contain one or more controls, such as a second editing control. For instance, if the region extraction function corresponding to any extraction operation is a template matching function, a record component can be added to the extraction management panel after the extraction operation is completed. This record component displays the text "template matching" so that the user can identify that the region extraction function corresponding to this extraction record is the template matching function.

[0110] For example, the record extraction component may also display one or more of the following: a second editing control, a delete control, and a visibility control. For example, the second editing control, delete control, and visibility control may be button controls. When a user triggers (e.g., clicks) the second editing control, the corresponding extraction record can be displayed in response to the user's trigger operation, and in response to the user's modification operation, at least some information in the extraction record, excluding the region extraction function, can be modified to re-extract the second region of interest. For example, when a user triggers (e.g., clicks) the second editing control, the aforementioned second editing window can be returned to display. The user can optionally re-edit one or more of the extraction information such as extraction parameters, extraction range, extraction center, extraction features, and extraction template in the displayed second editing window, and can optionally adjust the second region of interest in the extraction result. Based on the user-edited extraction information and / or the user's adjustment of the second region of interest, the second region of interest can be redefined. The aforementioned adjustment of the second region of interest may include operations such as adding a second region of interest, deleting a second region of interest, and adjusting the position of the second region of interest. Optionally, for the same extracted record, the second editing window displayed in response to the triggering operation of the second editing control can be completely or partially the same as the second editing window before saving the extracted record. For example, the second editing window displayed in response to the triggering operation of the second editing control can still display the aforementioned extraction control and completion control, etc. The delete control of the extracted record component and the delete control of the detection area component (e.g.) Figure 3Similar to the delete control 314 shown, when a user triggers (e.g., clicks) the delete control, the corresponding extraction record component and the extraction record associated with that component can be deleted in response to the user's triggering operation. The visibility control can be used to control the display and hiding of the extraction results on the first product image. For example, the second region of interest (i.e., the second region of interest contained in the extraction record associated with that extraction record component) corresponding to each extraction record component can be displayed by default on the first product image in the editing window. Clicking the visibility control of the extraction record component once will hide the second region of interest corresponding to that extraction record component; clicking the visibility control again will redisplay the second region of interest corresponding to that extraction record component. When there are many extraction records, the second regions of interest overlap, resulting in a cluttered display and affecting the user's viewing experience. Through the visibility control, the user can control whether the second regions of interest are displayed, making it easier for the user to clearly view the extraction status of the second regions of interest corresponding to each extraction record. Traditional modeling management schemes cannot edit and manage the extraction of regions of interest, while the scheme of this embodiment allows for comprehensive management, enabling viewing and re-editing of extraction records after settings, greatly facilitating user operation.

[0111] By adopting the above technical solution, the extraction records corresponding to each extraction operation can be managed through the extraction management panel, which makes it convenient for users to review the extraction effect of each extraction record and effectively improves the convenience of modeling management.

[0112] According to an embodiment of the present invention, the modeling and management method for detection algorithms further includes: displaying a preview window in response to a preview command input by a user; determining a second product image containing the product to be tested from an image obtained by real-time scanning or a preset image library in response to an image selection command input by a user; detecting the second product image using each detection algorithm under the currently set target algorithm formula to obtain the detection result of the second product image, wherein each product unit of the product to be tested is detected by the detection algorithm contained in the detection unit corresponding to the product unit; and displaying the detection result in the preview window.

[0113] Preview commands can be entered in any way, such as using keyboard shortcuts or by typing... Figure 2 The trigger input of the "Parameter Preview" control 220 shown. Figure 9 A schematic diagram of a preview window according to an embodiment of the present invention is shown. The preview window may also be referred to as an algorithm verification page. Similar to preview instructions, image selection instructions can be entered in any manner, such as via keyboard shortcuts or by... Figure 9The "Load Folder Test" control, "Single Image Test" control, and "All Image Test" control trigger inputs are shown. The product under test can be any product to be tested, which is of the same type as the standard product mentioned above, such as a wafer of the same model. The preset image library can store product images of any one or more products under test, and the user can select at least some product images from the preset image library as the second product image. Optionally, the second product image can also be obtained from images obtained by real-time scanning at the product manufacturing site. The second product image obtained by real-time scanning at the product manufacturing site is the online image, and the second product image obtained from the preset image library is the offline image. Therefore, the embodiments of the present invention support online and offline algorithm operation, and can use appropriate images for different detection scenarios to perform efficient algorithm modeling processes. Allowing offline detection allows algorithm personnel to perform algorithm modeling without entering the on-site workshop, thereby making it very convenient to preview the effect of the algorithm and identify problems in a timely manner.

[0114] After determining the second product image, various detection algorithms under the currently set target algorithm formula can be used to detect the second product image, for example... Figure 9 The defect detection shown is used to obtain the detection results of the second product image. The detection results can be displayed in the preview window in the form of image information and / or text information. Preview commands and image selection commands can be responded to at any time. This embodiment of the invention supports the verification of modeling content at any time during the modeling process, that is, allowing users to input preview commands and image selection commands at any time before, after, or during the process of setting the detection algorithm through editing operations. If the preview command and image selection command input by the user are received, the detection algorithms under the currently set target algorithm formula can be used to detect the second product image. For example, the independent setting of each detection unit during the modeling process can be realized by designing the data format and page interaction logic, and by merging the content of multiple detection units into the corresponding algorithm execution file when saving the target algorithm formula, it can help realize the joint detection of the subsequent product under test by multiple detection units. In this way, while multiple detection units can be set independently, the entire product under test can be detected as a whole when actually being tested. That is, the settings are independent, but the overall usage is possible. The editing window and the preview window are independent of each other, allowing simultaneous editing of the detection algorithm and previewing of the detection results of the product under test.

[0115] Using the above technical solution, the detection results of each detection algorithm under the currently set target algorithm formula can be displayed in the preview window, allowing users to view the detection effect of the detection algorithm in real time and adjust the detection algorithm in real time.

[0116] According to an embodiment of the present invention, the detection includes defect detection. The preview window includes one or more of the following: a sixth display area, a seventh display area, and an eighth display area. The sixth display area is used to display a first image area of ​​a second product image with a first field of view, and to highlight the defective product unit and / or the defect on the product unit in the first image area. The seventh display area is used to display a second image area of ​​a second product image with a second field of view, and to highlight the defective product unit and / or the defect on the product unit in the second image area. The second field of view is smaller than the first field of view. The eighth display area is used to display first detection information and / or second detection information of the defective product unit. The first detection information includes one or more of the following: the image identifier of the second product image where the product unit is located, the unit identifier of the detection unit corresponding to the product unit, the defect category of the product unit, the position of the product unit on the product to be tested, the position of the second product image where the product unit is located in all second product images, and the storage path of the second product image where the product unit is located. The second detection information includes one or more of the following: the defect category of each detection area of ​​the product unit with defects, the area identifier of each detection area of ​​the product unit with defects, and the size information of each detection area of ​​the product unit with defects.

[0117] like Figure 9As shown, a sixth display area 910 and a seventh display area 920 are marked, and an eighth display area 930 is marked with a dashed box. The sixth display area 910 and the seventh display area 920 display the first image area and the second image area of ​​the second product image, respectively, with different viewing angles. The first viewing angle is larger than the second viewing angle, meaning the first image area is larger than the second image area. For example, the sixth display area 910 can display the entire image of the second product image, while the seventh display area 920 can display a partial image of the second product image containing a smaller number of product units. The smaller number can be, for example, one, two, three, etc. Exemplarily and not limitingly, in either the sixth display area 910 or the seventh display area 920, the content of the currently displayed image area can be adjusted in response to a user's dragging operation of the image. That is, dragging the first product image in the sixth display area 910 and / or the seventh display area 920 can cause the first image area at different positions of the first product image to be displayed in the sixth display area 910 in real time and / or cause the second image area at different positions to be displayed in the seventh display area 920 in real time. In both the sixth and seventh display areas, defects can be highlighted in any way, such as highlighting defective product units and / or defects on product units. Highlighting may include using a mask to highlight the entire area in a color distinct from other image areas and / or highlighting the outline of the area. The sixth display area allows users to quickly understand the overall distribution of defects on the second product image. Since the product image is very large, and the area with defects is usually small, the seventh display area magnifies the local image of the defect, allowing users to easily view the precise location of the defect.

[0118] See Figure 9 The eighth display area 930 can be used to display first and / or second detection information of defective product units. For example, the first detection information of a product unit can be displayed in list form, with each product unit's first detection information shown as a row in the list. Taking a wafer containing RBG LED chips as an example, where each die is an RBG LED chip, and each die can include three product units, corresponding to the three detection units under the target algorithm formulation. Figure 9 In the "Image Test Results" list shown, each grain is paired with one detection unit, corresponding to a horizontal detection result (i.e., the first detection information). For example... Figure 9 As shown, the first detection information may include an image identifier of the second product image containing the defective product unit, which is displayed in the "Serial Number" column of the "Image Test Results" list. Figure 9As shown, the first detection information may include the unit identifier of the detection unit corresponding to the defective product unit, which is displayed in the "Inspection Unit" column of the "Image Test Results" list. Figure 9 As shown, the first detection information may include the location of the defective product unit on its corresponding product under test, which is displayed in the "Die_Column" and "Die_Row" columns of the "Image Test Results" list. Figure 9 As shown, the first detection information may include the location of the second product image containing the defective product unit within all second product images. This location is displayed in the "Image_Row" and "Image_Column" columns of the "Image Test Results" list. Figure 9 As shown, the first detection information may include the storage path of the second product image where the defective product unit is located, and the storage path is displayed in the "Current Defective Image Path" text box.

[0119] For example, the second inspection information of a product unit can be displayed in a list format, with the second inspection information of each inspection area shown as a row in the list. It can be understood that the inspection areas (Zones) corresponding to each inspection unit have been set in the editing window. When inspecting the product to be tested, the product unit corresponding to each inspection unit can be divided based on its corresponding inspection area, and the inspection areas on the product unit can be inspected. Figure 10 A schematic diagram showing a preview window according to another embodiment of the present invention is provided. Figure 10 As shown, the second inspection information may include the defect category for each inspected area of ​​the product unit where defects exist. This category is displayed in the "Category" column of the "Detailed Defect Results" list. The defect category can be determined based on the application scenario; for example, it may include bright and dark defects, or other types and quantities of defects. Figure 10 As shown, the second inspection information may include the zone identifiers of each inspection area where a defect exists in the product unit. These identifiers are displayed in the "Zone" column of the "Detailed Defect Results" list. Figure 10 As shown, the second detection information may include the size information of each detection area of ​​the product unit where a defect exists, which is displayed in the "Length (μm)", "Width (μm)" and "Area (μm)" lists in the "Detailed Defect Results" list. 2 The three columns are displayed. Note that... Figure 9 and Figure 10 The types of information contained in the first and second detection information shown, as well as the display method of each piece of information, are merely examples and not intended to limit the invention. The invention is not limited to these examples. Figure 9 and Figure 10 The example shown.

[0120] The aforementioned sixth, seventh, and eighth display areas can be displayed individually, or two or all three can be displayed simultaneously. In the simultaneous display scheme, the output of the detection results is in the form of an image plus the first detection information and / or the second detection information, which can more intuitively and clearly display the detection results of the algorithm.

[0121] By adopting the above technical solution, the defect detection results of the product under test can be visually presented through one or more of the sixth, seventh and eighth display areas, which can provide users with more intuitive defect information.

[0122] According to an embodiment of the present invention, when the preview window includes an eighth display area, the first detection information is displayed using a first list, and the second detection information is displayed using a second list. Different list units in the first list correspond to different product units, and different list units in the second list correspond to different detection areas of the same product unit. The list units are rows or columns. Displaying the detection results in the preview window includes: in response to the user's selection operation of any list unit in the first list, displaying the second detection information of the product unit corresponding to that list unit in the second list.

[0123] List cells can be rows or columns, and can be set as needed. Figure 9 and Figure 10 In the example shown, each row of the list is considered a single list unit. For example... Figure 9 As shown, the first inspection information is displayed in a first list, namely the "Image Test Results" list, and the second inspection information is displayed in a second list, namely the "Detailed Defect Results" list. The first and second lists can be two-level lists, meaning that when a user selects any list unit in the first list, the second inspection information for the corresponding product unit can be displayed in real time in the second list. In other words, the content displayed in the second list can change in real time following the list unit selected in the first list. This method of displaying macro-level inspection information in the first list and specific inspection information for a particular product unit in the second list simplifies the information arrangement on the display interface, making the display more organized and allowing users to quickly locate the defect information they need.

[0124] According to an embodiment of the present invention, when the preview window includes a sixth display area and a seventh display area, displaying the detection result in the preview window includes: in response to a user's selection operation on any product unit on the second product image in the sixth display area, switching the display of the first image area of ​​the product unit in the seventh display area; and / or, when the preview window includes a sixth display area and an eighth display area, displaying the detection result in the preview window includes: in response to a user's selection operation on any product unit on the second product image in the sixth display area, highlighting the first detection information and / or the second detection information of the product unit in the eighth display area.

[0125] For example, when the preview window includes a sixth display area and a seventh display area, displaying the detection results in the preview window includes: responding to the user's selection operation of any product unit on the second product image in the sixth display area, switching the display of the second image area of ​​that product unit in the seventh display area. The user can view the macroscopic second product image through the sixth display area and identify the product unit of interest. When the user selects any product unit on the second product image through a selection operation, such as by mouse or touch, the second image area containing the currently selected product unit can be displayed in real time in the seventh display area. That is, when the product unit selected by the user changes on the second product image, the content displayed in the seventh display area can switch and change accordingly in real time. Using this scheme, the user can select the product unit of interest in the larger image area displayed in the sixth display area and view the specific defect information of the currently selected product unit in the seventh display area. This linked display scheme offers high viewing efficiency.

[0126] For example, when the preview window includes a sixth display area and an eighth display area, displaying the detection results in the preview window includes: in response to the user's selection operation of any product unit on the second product image in the sixth display area, highlighting the first detection information and / or the second detection information of the product unit in the eighth display area.

[0127] Users can select product units in the sixth display area. When a user selects any product unit, the first detection information and / or the second detection information of that product unit can be highlighted in the eighth display area. The highlighting can be done in any way, including but not limited to using methods such as darkening the background color of the display area containing the detection information and / or darkening the border of the display area containing the detection information. Figure 9The background color of the first row in the "Image Test Results" list is deepened as shown. When the user selects a different product unit on the second product image, the highlighted inspection information in the eighth display area changes in real time. Using this scheme, users can select the product unit of interest within the larger image area displayed in the sixth display area and easily locate and view the specific defect information of the currently selected product unit in the eighth display area. This linked display scheme offers high viewing efficiency.

[0128] For example, in response to a user's selection of any product unit on the second product image in the sixth display area, the second image area of ​​that product unit can be switched to be displayed in the seventh display area, and the first detection information and / or second detection information of that product unit can be highlighted in the eighth display area. In other words, the solution where the seventh and eighth display areas are displayed in tandem with the selection of the sixth display area can be implemented in the same embodiment. This solution can further improve viewing efficiency and enhance the user experience.

[0129] According to embodiments of the present invention, the second product image is one or more; and / or, the preview window and the editing window are displayed at the same display level, or the preview window is displayed at a higher display level than the editing window, or the preview window is displayed after the editing window is closed.

[0130] The number of products to be tested can be one or more, and the number of second product images can also be one or more. Any two second product images can contain the same product to be tested or can contain different products to be tested. Figure 9 and 10 As shown, the upper right corner of the preview window displays a "Load Folder Test" control and either a "Single Image Test" control or a "All Image Test" control. When the user triggers the "Load Folder Test" control, a file open dialog box pops up, allowing the user to select any image folder or at least some product images from within an image folder. Each image folder can contain one or more product images. The user can also select one or more product images as second product images through the file open dialog box. For the one or more second product images selected by the user, the detection algorithm in the target algorithm formula can be used. For example, when the user triggers the "Single Image Test" control, the image can be detected independently in response to the user's selection of a single product image; or when the user triggers the "All Image Test" control, multiple product images can be detected collectively in response to the user's selection of them. Using this technical solution, any number of second product images can be imported for detection according to user needs, allowing users to easily view the detection results of the products under test independently or through centralized comparison.

[0131] Figure 11A schematic diagram illustrating a preview window and an editing window according to an embodiment of the present invention is shown. Exemplarily, as... Figure 11 As shown, the preview window and editing window can be displayed on the same display layer. This same-layer display method allows users to simultaneously view the preview and editing windows in a split-screen manner, eliminating the need to frequently switch between the modeling management page and the algorithm verification page. This enables the effect of previewing and editing simultaneously, which helps improve modeling efficiency. It is understandable that during algorithm modeling, it is usually necessary to frequently verify the set detection algorithm to pursue better detection results. If editing and verification cannot be performed simultaneously, frequent page switching would create a significant operational burden. The scheme of displaying the preview and editing windows on the same layer allows for simultaneous editing and rapid verification preview of multiple detection units, greatly facilitating the user's modeling. For example, the preview window can also be displayed on a higher display layer than the editing window, meaning that displaying the preview window allows at least a portion of the editing window's window area to be obscured. This makes it easier for users to clearly view the information in the preview window when the user interface display space is limited. For example, the preview window can be displayed after the editing window is closed, which reduces the rendering pressure on the user interface and helps improve the responsiveness of the preview window when the computing resources of electronic devices are limited.

[0132] The aforementioned second product image, which can be one or more, can be combined with various display schemes of the preview window and editing window in the same embodiment. Specific implementation methods and technical effects can be understood by referring to the above description, and will not be repeated here.

[0133] According to an embodiment of the present invention, the modeling management method further includes: displaying a first prompt window in response to a user-inputted command to close the editing window; saving the setting information of the target detection unit and closing the first prompt window and the editing window in response to a user-triggered operation of a save / close control in the first prompt window; closing the first prompt window and the editing window in response to a user-triggered operation of a close confirmation control in the first prompt window; and closing the first prompt window while maintaining the display of the editing window in response to a user-triggered operation of a close / cancel control in the first prompt window.

[0134] Figure 12 A schematic diagram of a first prompt window according to an embodiment of the present invention is shown. Figure 12 As shown, when a user enters a command to close the editing window in any way, such as by using a keyboard shortcut or by clicking the first close control (the "x" control) in the upper right corner of the editing window or the second close control (the "Close Window" control) at the bottom, a pop-up window will appear. Figure 12 The first prompt window shown is used to prompt the user to confirm whether to save the target detection unit settings before closing the editing window. The prompt message is, for example: Figure 12The text message displayed asks, "Save before closing the current form?". The first prompt window can display save / close controls, close confirmation controls, and close cancel controls, as shown below. Figure 12 The diagram shows three controls: "Save and Close," "Close Directly," and "Cancel Close." When a user triggers the "Save and Close" control, the settings for the target detection unit's detection algorithm are saved, and the first prompt window and the editing window are closed, exiting the editing process. When a user triggers the "Close Directly" control, the settings are not saved; the first prompt window and the editing window are closed directly. When a user triggers the "Cancel Close" control, the editing window is closed, but the first prompt window remains displayed, allowing the user to continue editing. This technical solution provides a prompt when the user closes the editing window, reminding them to save their settings to avoid data loss due to accidental closure of the editing window.

[0135] According to an embodiment of the present invention, before the step of deleting the detection area and the detection area component corresponding to the detection area in response to the user's trigger operation on the delete control corresponding to any detection area, the detection algorithm included in the target detection unit is set in response to the user's editing operation in the editing window, and the method further includes: displaying a second prompt window in response to the user's trigger operation on the delete control corresponding to any detection area; the operation of deleting the detection area and the detection area component corresponding to the detection area is executed in response to the user's trigger operation on the confirmation delete control in the second prompt window.

[0136] Figure 13 A schematic diagram of a second prompt window according to an embodiment of the present invention is shown. Figure 13 As shown, when the user triggers the delete control in any detection area, a pop-up window will appear. Figure 13 The second prompt window is shown. This second prompt window is used to prompt the user to confirm whether to delete the detection area. The prompt message is, for example,... Figure 13 The message displayed is "Are you sure you want to delete this Zone? It cannot be recovered after deletion!". The second prompt window can display a confirmation control and a cancellation control, as shown below. Figure 13The diagram shows "Confirm Delete" and "Cancel Delete" controls. When a user triggers the "Confirm Delete" control, the detection area and its corresponding components are directly deleted, and the second prompt window closes. When a user triggers the "Cancel Delete" control, the deletion of the detection area and its corresponding components is skipped, the second prompt window closes, and the editing window remains open for further editing. This technical solution provides appropriate prompts when a user deletes a detection area, reminding them of the risks involved and preventing data loss due to accidental deletion.

[0137] According to an embodiment of the present invention, the modeling management method further includes: in response to a user's trigger operation on the parameter save control in the editing window, saving the setting information of the target detection unit and displaying a third prompt window; in response to a user's trigger operation on the close confirmation control in the third prompt window, closing the third prompt window and the editing window; in response to a user's trigger operation on the close cancel control in the third prompt window, closing the third prompt window and keeping the editing window displayed.

[0138] The editing window can display a parameter saving control, for example... Figure 3-8 The "Save Parameters" control is shown at the bottom of the editing window. In response to the user's action on the parameter save control, the settings information for the target detection unit's detection algorithm configured by the user during the current setup process can be saved, and a third prompt window will pop up. Figure 14 A schematic diagram of a third prompt window according to an embodiment of the present invention is shown. Figure 14 As shown, when the user triggers the parameter saving control, a pop-up window will appear. Figure 14 The third prompt window shown. This third prompt window is used to prompt the user to confirm whether to close the editing window; the prompt message is, for example: Figure 14 The text message displayed is "Save successful, close the current editing window?". The third prompt window can display a confirmation control and a cancel control, as shown below. Figure 14 The diagram shows "Confirm Close" and "Keep On" controls. When the user triggers the "Confirm Close" control, both the third prompt window and the editing window will close directly. When the user triggers the "Keep On" control, only the third prompt window will close, while the editing window remains open for continued editing. This technical solution provides a prompt after the user saves parameters, reminding them to confirm whether to close the editing window. This allows users to quickly determine whether to exit the current target detection unit's editing, improving modeling efficiency and offering strong interactivity.

[0139] As described above, the modeling management page can be displayed in response to a user's page opening action. This page opening action may include selecting a recipe identifier for a target algorithm recipe in the startup page. The startup page can display recipe identifiers for one or more algorithm recipes, which may include all or some of the algorithm recipes stored in a preset recipe library, and may also include one or more algorithm recipes recently opened by the user. For example, when the algorithm modeling tool application starts, a management page for algorithm engineering projects, i.e., the startup page, can be displayed at the main entry point. Through the startup page, new algorithm recipes can be quickly added or previously set algorithm recipes can be edited and viewed. Figure 15 The diagram illustrates the startup page of an algorithm modeling tool according to an embodiment of the present invention. Exemplarily, this startup page can support opening local algorithm recipes, recording the opened items in a list for quick access next time, and can be sorted and displayed according to the most recent opening time. This function greatly facilitates users' recipe management needs and enables rapid maintenance of algorithm recipes.

[0140] According to another aspect of the present invention, a modeling and management device for a detection algorithm is provided. Figure 16 A schematic block diagram of a modeling and management device 1600 for a detection algorithm according to an embodiment of the present invention is shown. Figure 16 As shown, the device 1600 may include a first display module 1610, a second display module 1620, and a setting module 1630.

[0141] The first display module 1610 is used to display the modeling management page corresponding to the target algorithm formula of the product. The modeling management page contains the first editing control of each of the multiple detection units under the target algorithm formula. The multiple detection units correspond one-to-one with the multiple product units on the product.

[0142] The second display module 1620 is used to display the editing window of the target detection unit in response to a user's trigger operation on the first editing control of any target detection unit among multiple detection units.

[0143] The setting module 1630 is used to respond to the user's editing operation in the editing window and set the detection algorithm included in the target detection unit. The detection algorithm is used to detect the target product unit corresponding to the target detection unit.

[0144] According to another aspect of the present invention, an electronic device is also provided. Figure 17 A schematic block diagram of an electronic device 1700 according to an embodiment of the present invention is shown, such as Figure 17As shown, the electronic device 1700 may include a processor 1710 and a memory 1720. The memory 1720 stores a computer program, and the processor 1710 executes the computer program to implement the modeling and management method of the detection algorithm described above.

[0145] According to another aspect of the present invention, a storage medium is also provided. This medium stores a computer program / instructions, which, when executed by a processor, implement the modeling and management method of the detection algorithm described above. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0146] Those skilled in the art can understand the specific implementation schemes and beneficial effects of the above-mentioned detection algorithm modeling management device, electronic device and storage medium by reading the relevant description of the modeling management method of the detection algorithm. For the sake of brevity, they will not be described in detail here.

[0147] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention thereto. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0150] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0151] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0152] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0153] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0154] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the modeling and management apparatus for the detection algorithm according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0155] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0156] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A modeling and management method for a detection algorithm, comprising: The modeling management page displays the target algorithm formula of the product. The modeling management page includes the first editing control of each of the multiple detection units under the target algorithm formula. The multiple detection units correspond one-to-one with the multiple product units on the product. In response to a user's triggering operation on the first editing control of any of the plurality of detection units, the editing window of the target detection unit is displayed; In response to the user's editing operation in the editing window, the detection algorithm included in the target detection unit is set, and the detection algorithm is used to detect the target product unit corresponding to the target detection unit.

2. The modeling management method according to claim 1, characterized in that, The editing window includes a first editing window, which displays a first product image of the target product unit containing a standard product. The step of setting the detection algorithm included in the target detection unit in response to the user's editing operation in the editing window includes: In response to a user's region setting operation for the first product image, a detection region corresponding to the target detection unit is set, and each detection region includes at least one region of interest on the target product unit; For each detection region, in response to the user's algorithm setting operation, the detection algorithm used for that detection region is determined; The target detection unit includes a detection algorithm for each detection region corresponding to the target detection unit. The editing operation includes a region setting operation and an algorithm setting operation. The detection algorithm used in each detection region is used to detect each region of interest included in the detection region.

3. The modeling management method as described in claim 2, wherein, The step of setting the detection area corresponding to the target detection unit in response to the user's region setting operation for the first product image includes: For each detection region corresponding to the target detection unit, in response to the user's operation of setting a region of interest for the first product image, the position of each region of interest included in the detection region is determined; The region setting operation includes the region of interest setting operation.

4. The modeling management method as described in claim 3, wherein, The first editing window includes a first display area, in which the first product image and a first drawing control are displayed. The first drawing control is used to set a first drawing tool. The first drawing tool includes a first type of drawing tool and / or a second type of drawing tool. The inside of the graphic drawn using the first type of drawing tool is the selected area, and the outside of the graphic drawn using the second type of drawing tool is the selected area. The step of determining the location of each region of interest included in the detection area in response to the user's operation of setting a region of interest for the first product image includes: In response to the user's triggering operation on the first drawing control, determine the first drawing tool selected by the user; In response to the user's first drawing operation on the first product image, the location of the first region of interest included in the detection area is determined according to the graphic specified by the first drawing tool; The region of interest setting operation includes triggering the first drawing control and the first drawing operation.

5. The modeling management method as described in claim 3 or 4, wherein, The target detection unit includes at least some detection algorithms, each of which has at least one region extraction function. Different region extraction functions are used to extract regions of interest according to different extraction methods. The step of determining the location of each region of interest included in the detection area in response to the user's operation of setting a region of interest for the first product image includes: Perform one or more extraction operations, each corresponding to a region extraction function, the extraction operations including: In response to the user's function selection operation, the region extraction function corresponding to the current extraction operation is determined, and the second editing window of the region extraction function is displayed, the editing window also including the second editing window; In response to the user's information setting operation for the extracted information in the second editing window, the extraction information for the region extraction function is determined. The extraction information is used to indicate one or more of the following information for the region of interest: extraction parameters, extraction range, extraction center, extraction template, and extraction features. Based on the extracted information, and according to the extraction method of the region extraction function corresponding to this extraction operation, the second region of interest included in the detection region is extracted from the first product image to determine the location of the second region of interest; The region of interest setting operation includes the one or more extraction operations.

6. The modeling management method as described in claim 5, wherein, The second editing window displays the first product image and a second drawing control. The second drawing control is used to set a second drawing tool, which includes the first type of drawing tool and / or the second type of drawing tool. The step of responding to the user's information setting operation for the extracted information in the second editing window, and determining the extraction information for the area extraction function, includes: In response to the user's triggering operation on the second drawing control, determine the second drawing tool selected by the user; In response to the user's second drawing operation on the first product image in the second editing window, the extraction range of the region of interest is determined according to the graphic specified by the second drawing tool; The information setting operation includes triggering the second drawing control and the second drawing operation. The second editing window also displays the parameter values ​​of the extraction parameters of the region extraction function. Each parameter in the extraction parameters is a default parameter value or a user-set parameter value.

7. The modeling management method as described in claim 6, wherein, The at least one region extraction function includes one or more of the following: template matching function, binarization function, feature extraction function, and threshold extraction function; For the template matching function, the corresponding second editing window also displays a third drawing control. The third drawing control is used to set a third drawing tool. The third drawing tool includes the first type of drawing tool and / or the second type of drawing tool. The step of determining the extraction information for the region extraction function in response to the user's information setting operation on the extraction information in the second editing window further includes: determining the third drawing tool selected by the user in response to the user's trigger operation on the third drawing control; and determining the extraction template for extracting the region of interest based on the graphic specified by the third drawing tool, in response to the user's third drawing operation on the first product image in the second editing window. The information setting operation also includes the trigger operation on the third drawing control and the third drawing operation. The second region of interest is the region whose matching degree with the extraction template meets the requirements of the extraction parameters of the template matching function and falls within the extraction range of the template matching function. For the binarization function, the second region of interest is the region extracted according to the extraction parameters of the binarization function and falling within the extraction range of the binarization function; For the feature extraction function, the corresponding second editing window also displays a fourth drawing control. The fourth drawing control is used to set a fourth drawing tool. The third drawing tool includes the first type of drawing tool and / or the second type of drawing tool. The step of determining the extraction information for the region extraction function in response to the user's information setting operation on the extraction information in the second editing window further includes: determining the fourth drawing tool selected by the user in response to the user's trigger operation on the fourth drawing control; and determining the extraction features used to extract the region of interest based on the graphics specified by the fourth drawing tool, in response to the user's fourth drawing operation on the first product image in the second editing window. The information setting operation also includes triggering the fourth drawing control and the fourth drawing operation. The second region of interest is the region whose matching degree with the extracted features meets the requirements of the extraction parameters of the feature extraction function and falls within the extraction range of the feature extraction function. For the threshold extraction function, the second region of interest is the region extracted according to the extraction parameters of the threshold extraction function and falling within the extraction range of the threshold extraction function.

8. The modeling management method as described in claim 5, wherein, The at least one region extraction function includes a guard line extraction function; For the guard line extraction function, the corresponding second editing window also displays a fifth drawing control. The fifth drawing control is used to set a fifth drawing tool. The border of the graphic drawn by the fifth drawing tool represents the extraction center. The step of determining the extraction information of the region extraction function in response to the user's information setting operation on the extraction information in the second editing window further includes: determining the fifth drawing tool selected by the user in response to the user's trigger operation on the fifth drawing control; and determining the extraction center on which the region of interest is based, according to the graphic specified by the fifth drawing tool, in response to the user's fifth drawing operation on the first product image in the second editing window. The information setting operation also includes the trigger operation on the fifth drawing control and the fifth drawing operation. The second region of interest is the region extracted according to the extraction parameters of the guard line extraction function and located around the extraction center of the guard line extraction function.

9. The modeling management method as described in claim 5, wherein, The first editing window further includes a second display area, which is used to display a function selection control, and the function selection operation is a trigger operation on the function selection control; And / or, The second editing window also displays the parameter values ​​of the extraction parameters for the region extraction function. The step of determining the extraction information for the region extraction function in response to the user's information setting operation on the extraction information in the second editing window includes: In response to a user's setting operation on the parameter values ​​of at least some of the extracted parameters in the second editing window, a new parameter value for the at least some extracted parameters is determined, wherein the information setting operation includes setting the parameter values ​​of the at least some extracted parameters; And / or, The second editing window also displays an extraction control. The step of extracting the second region of interest included in the detection area from the first product image according to the extraction information and the extraction method of the region extraction function corresponding to the current extraction operation is executed in response to the user's trigger operation on the extraction control. And / or, The second editing window also displays a completion control. The extraction operation further includes: in response to the user's triggering operation on the completion control, determining that the current extraction operation is completed and saving the extraction record corresponding to the current extraction operation. The extraction record includes the region extraction function used in the extraction operation, the extraction information on which it is based, and the extraction result obtained.

10. The modeling management method as described in claim 9, wherein, When the extraction control and the completion control are displayed in the second editing window, after the step of extracting the second region of interest included in the detection area from the first product image according to the extraction information and the extraction method of the region extraction function corresponding to this extraction operation, the extraction operation further includes: The supplementary drawing control is displayed in the second editing window. The supplementary drawing control is used to set the supplementary drawing tools. The supplementary drawing tools include a first type of drawing tool and / or a second type of drawing tool. The inside of the graphic drawn using the first type of drawing tool is the selected area, and the outside of the graphic drawn using the second type of drawing tool is the selected area. In response to the user's triggering operation on the supplementary drawing control, determine the supplementary drawing tool selected by the user; In response to the user's supplementary drawing operation on the first product image in the second editing window, a supplementary second region of interest is determined according to the graphic specified by the supplementary drawing tool; The additional second region of interest is added to the extraction result of this extraction operation.

11. The modeling management method as described in claim 5, wherein, The first editing window includes a third display area, which displays an extraction management panel. The extraction management panel is used to display extraction record components. Each extraction record component corresponds to one extraction operation and is associated with the extraction record of the corresponding extraction operation. The extraction record includes the region extraction function used by the extraction operation, the extraction information on which it is based, and the extraction result obtained. The modeling and management method for the detection algorithm also includes: After each extraction operation, an extraction record component corresponding to the current extraction operation is added to the extraction management panel. The extraction record component displays the corresponding area extraction function and also displays one or more of the following: a second editing control, a deletion control, and a visibility control. The second editing control is used to respond to the user's operation, display the corresponding extraction record, and modify at least some information in the extraction record other than the region extraction function to redetermine the second region of interest; the deletion control is used to respond to the user's operation to delete the extraction record component and the associated extraction record; the visibility control is used to respond to the user's operation to control the display of the extraction results in the extraction record associated with the extraction record component on the first product image.

12. The modeling management method as described in claim 3, wherein, The first editing window includes a fourth display area, which contains detection area components corresponding to each detection area of ​​the target detection unit. Each detection area component is used to display the area identifier of the corresponding detection area and the type of detection algorithm used by the detection area. The step of setting the detection area corresponding to the target detection unit in response to the user's region setting operation on the first product image further includes: In response to the user's identifier setting operation in the detection area component, the area identifier of the detection area corresponding to the detection area component is determined. The area setting operation also includes the identifier setting operation.

13. The modeling management method as described in claim 2, wherein, The step of determining the detection algorithm used for the detection area in response to the user's algorithm setting operation includes: In response to the user's type setting operation, determine the type of detection algorithm used for the detection area; Perform parameter setting operations, which include: a first operation, or a first operation and a second operation, or a second operation and a third operation, or a first operation, a second operation, and a third operation; The first operation includes: determining, based on the determined type, the parameter values ​​of at least some parameters of the detection algorithm used in the detection area from a preset parameter library, wherein the preset parameter library stores default parameter values ​​of at least some parameters of each of at least one type of detection algorithm; The second operation includes: based on the determined type, displaying a parameter panel in the fifth display area of ​​the first editing window, the parameter panel being used to display the parameter values ​​of the detection algorithm used in the detection area; The third operation includes: in response to a user's parameter value setting operation on at least some parameters in the parameter panel, determining a new parameter value for the parameters set by the parameter setting operation; The algorithm setting operation includes the type setting operation and the parameter setting operation.

14. The modeling management method as described in claim 13, wherein, The first editing window includes a fourth display area, which contains detection area components corresponding to each detection area of ​​the target detection unit. Each detection area component is used to display the area identifier of the corresponding detection area and the type of detection algorithm used by the detection area. The type setting operation is a setting operation performed by the user in the detection area component.

15. The modeling management method as described in claim 12 or 14, wherein, The fourth display area also includes one or more of the following controls: create control, import control, export control, copy control corresponding to each detection area, and delete control corresponding to each detection area; The step of setting the detection algorithm included in the target detection unit in response to the user's editing operation in the editing window further includes: In response to the user's triggering operation on the created control, a new detection area component is created; In response to the user's triggering operation on the import control and the selection operation on the setting information file of any detection unit stored in the preset file library, the setting information file of the detection unit is converted into setting information, and the converted setting information is imported into the first editing window for display. The preset file library is used to store the setting information file of the detection unit. The setting information file contains the setting information of the detection unit. The setting information includes the area identifier of each detection area of ​​the detection unit and the type of detection algorithm used by each detection area of ​​the detection unit and the parameter values ​​of the detection algorithm parameters. In response to the user's triggering operation on the export control, the setting information of the target detection unit is converted into a setting information file in a preset format and the setting information file is exported to the preset file library for storage; In response to a user's trigger operation on the delete control corresponding to any detection area, delete the detection area and the detection area component corresponding to the detection area; In response to a user's triggering operation on the copy control corresponding to any detection area, the detection area component corresponding to that detection area is copied to generate a new detection area component in the fourth display area; In response to a user's dragging operation on any detection area component, adjust the arrangement order of that detection area component relative to other detection area components; The editing operation includes triggering operations on one or more of the following controls: the created control, the imported control, the exported control, the deleted control, and the copied control, and / or the drag operation.

16. The modeling management method according to any one of claims 1-4, wherein, The modeling and management method for the detection algorithm also includes: In response to a user's preview command, a preview window is displayed; In response to the user's image selection command, a second product image containing the product to be tested is determined from the image obtained by real-time scanning or from a preset image library; The second product image is detected using the detection algorithms under the currently set target algorithm formula to obtain the detection result of the second product image. Each product unit of the product to be tested is detected by the detection algorithm contained in the detection unit corresponding to the product unit. The detection results are displayed in the preview window.

17. The modeling management method as described in claim 16, wherein, The detection includes defect detection, and the preview window includes one or more of the following: a sixth display area, a seventh display area, and an eighth display area. The sixth display area is used to display the first image area of ​​the second product image in a first field of view, and to highlight the defective product unit and / or the defect on the product unit in the first image area. The seventh display area is used to display the second image area of ​​the second product image with a second field of view, and to highlight the defective product unit and / or the defect on the product unit in the second image area, wherein the second field of view is smaller than the first field of view; The eighth display area is used to display first and / or second detection information of the defective product unit. The first detection information includes one or more of the following: the image identifier of the second product image where the product unit is located, the unit identifier of the detection unit corresponding to the product unit, the defect category of the product unit, the position of the product unit on the product under test, the position of the second product image where the product unit is located in all the second product images, and the storage path of the second product image where the product unit is located. The second detection information includes one or more of the following: the defect category of each detection area of ​​the defective product unit, the area identifier of each detection area of ​​the defective product unit, and the size information of each detection area of ​​the defective product unit.

18. The modeling management method as described in claim 17, wherein, When the preview window includes the eighth display area, the first detection information is displayed in a first list, and the second detection information is displayed in a second list. Different list units in the first list correspond to different product units, and different list units in the second list correspond to different detection areas of the same product unit. The list unit is a row or a column. The display of the detection results in the preview window includes: In response to a user's selection of any list cell in the first list, the second detection information of the product cell corresponding to that list cell is displayed in the second list.

19. The modeling management method as described in claim 17, wherein, When the preview window includes the sixth display area and the seventh display area, displaying the detection result in the preview window includes: In response to a user's selection of any product unit on the second product image in the sixth display area, the second image area displaying that product unit is switched in the seventh display area; And / or, When the preview window includes the sixth display area and the eighth display area, displaying the detection result in the preview window includes: In response to a user's selection of any product unit on the second product image in the sixth display area, the first detection information and / or the second detection information of that product unit are highlighted in the eighth display area.

20. The modeling management method as described in claim 16, wherein, The second product image is one or more; and / or, The preview window and the editing window are displayed at the same display level, or the preview window is displayed at a higher display level than the editing window, or the preview window is displayed after the editing window is closed.

21. A modeling and management device for a detection algorithm, comprising: The first display module is used to display the modeling management page corresponding to the target algorithm formula of the product. The modeling management page includes the first editing control of each of the multiple detection units under the target algorithm formula. The multiple detection units correspond one-to-one with the multiple product units on the product. The second display module is used to display the editing window of the target detection unit in response to a user's trigger operation on the first editing control of any of the plurality of detection units; The setting module is used to respond to the user's editing operation in the editing window and set the detection algorithm included in the target detection unit. The detection algorithm is used to detect the target product unit corresponding to the target detection unit.

22. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, the computer program instructions being executed by the processor to perform a modeling and management method for a detection algorithm as described in any one of claims 1-20.

23. A storage medium storing a computer program / instructions, which, when executed, performs a modeling and management method for the detection algorithm as described in any one of claims 1-20.

Images