Defect detection method and device, model training method and device, and electronic device

Abstract

The present disclosure relates to the technical field of defect detection, and particularly relates to a defect detection method and device, a model training method and device and an electronic device. The method comprises: acquiring sample data set containing defect product data, and identifying feature information of the sample data set, wherein the feature information comprises sample quantity of the sample data set; acquiring an initial model, wherein the initial model is a neural network model; configuring training parameters based on the feature information; training the initial model based on the sample data set and the training parameters to obtain a target model; inputting real data of a product corresponding to the sample data set into the target model to obtain defect information of the product; wherein the training parameters comprise at least one of the following: a learning rate reduction strategy, a total number of training rounds and a test strategy, wherein the learning rate reduction strategy comprises a learning rate reduction number and a round number at which the learning rate is reduced; and the test strategy comprises a test number and a round number at which the test is performed. The present scheme saves human resources.

Description

Technical Field

[0001] This disclosure relates to the field of defect detection technology, and more specifically, to a defect detection method and apparatus, a model training method and apparatus, a computer-readable storage medium, and an electronic device. Background Technology

[0002] In the screen manufacturing industry, problems in equipment, parameters, operation, and environmental interference can lead to defective products. With the rise of artificial intelligence algorithms, represented by deep learning, the use of deep learning models for defect detection is becoming increasingly widespread.

[0003] However, in existing technologies, parameter adjustments in deep learning models often require manual intervention, which wastes human resources and may result in losses due to human error.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to a first aspect of this disclosure, a defect detection method is provided, comprising:

[0007] Obtain a sample dataset containing defective product data, and identify the feature information of the sample dataset, the feature information including the number of samples in the sample dataset;

[0008] Obtain an initial model, wherein the initial model is a neural network model;

[0009] Configure training parameters based on the aforementioned feature information;

[0010] Using the sample dataset and training the initial model according to the training parameters, a target model is obtained;

[0011] The actual data of the products corresponding to the sample dataset are input into the target model to obtain the defect information of the products;

[0012] The training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during which the learning rate is descented; and the testing strategy includes the number of tests and the number of epochs during which the learning rate is tested.

[0013] According to a second aspect of this disclosure, a model training method is provided, comprising:

[0014] Obtain a sample dataset containing defective product data, and identify the feature information of the sample dataset, the feature information including the number of samples in the sample dataset;

[0015] Obtain an initial model, wherein the initial model is a neural network model;

[0016] Configure training parameters based on the aforementioned feature information;

[0017] The initial model is trained using the sample dataset and the training parameters to obtain a target model, which is used to detect defects in the real data of the products corresponding to the sample dataset.

[0018] The training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during which the learning rate is descented; and the testing strategy includes the number of tests and the number of epochs during which the learning rate is tested.

[0019] According to a third aspect of this disclosure, a model training method is provided, comprising:

[0020] In response to the user's configuration operation on the parameters of the sample dataset, the system obtains a sample dataset containing defective product data and identifies the feature information of the sample dataset, including the number of samples in the sample dataset.

[0021] Obtain an initial model, wherein the initial model is a neural network model;

[0022] Configure training parameters based on the feature information and generate a training parameter display interface;

[0023] The initial model is trained using the sample dataset and the training parameters to obtain a target model, which is used to detect defects in the real data of the products corresponding to the sample dataset.

[0024] The training parameters displayed on the training parameter display interface include at least one of the following: learning rate descent strategy, total number of training epochs, and testing strategy. The learning rate descent strategy includes the number of times the learning rate is decreased and the number of epochs during which the learning rate is decreased. The testing strategy includes the number of tests and the number of epochs during which the learning rate is tested.

[0025] According to a fourth aspect of this disclosure, a defect detection system is provided, comprising:

[0026] The data management module is configured to store and manage sample data;

[0027] The training management module is configured to execute any of the defect detection methods, model training methods, or model training methods described above.

[0028] The model management module is configured to store, display, and manage the target model.

[0029] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, wherein the program, when executed by a processor, implements the defect detection method described above, the model training method described in any one of the preceding claims, or the model training method described in any one of the preceding claims.

[0030] According to a sixth aspect of this disclosure, an electronic device is provided, comprising:

[0031] Processor; and

[0032] A memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement any of the defect detection methods, model training methods, or model training methods described above.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0035] Figure 1 A schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied is shown;

[0036] Figure 2 A schematic diagram of an electronic device to which embodiments of the present disclosure may be applied is shown;

[0037] Figure 3 This schematically illustrates a flowchart of a defect detection method according to an exemplary embodiment of the present disclosure;

[0038] Figure 4 A schematic diagram illustrating a loss curve in an exemplary embodiment of the present disclosure is shown.

[0039] Figure 5A flowchart illustrating an exemplary embodiment of the present disclosure is shown schematically.

[0040] Figure 6 A flowchart illustrating another model detection method in an exemplary embodiment of this disclosure is shown schematically.

[0041] Figure 7 This is a schematic diagram illustrating the training parameter display interface in an exemplary embodiment of the present disclosure;

[0042] Figure 8 This schematically illustrates an interface diagram for determining whether to train defects in an exemplary embodiment of this disclosure;

[0043] Figure 9 This schematic diagram illustrates a training process in an exemplary embodiment of the present disclosure.

[0044] Figure 10 A schematic diagram illustrating a confusion matrix in an exemplary embodiment of this disclosure is provided.

[0045] Figure 11 This diagram schematically illustrates the target model selection interface in an exemplary embodiment of the present disclosure.

[0046] Figure 12 This schematic diagram illustrates the structure of a trap detection system in an exemplary embodiment of the present disclosure.

[0047] Figure 13 This schematic diagram illustrates a trap detection system framework diagram in an exemplary embodiment of the present disclosure;

[0048] Figure 14 A schematic diagram illustrating the interface for dataset preparation in an exemplary embodiment of this disclosure is shown.

[0049] Figure 15 This diagram schematically illustrates a training dataset management interface in an exemplary embodiment of the present disclosure.

[0050] Figure 16 The schematic diagram illustrates the interface for creating new data in an exemplary embodiment of this disclosure;

[0051] Figure 17 This diagram schematically illustrates a dataset details display interface in an exemplary embodiment of this disclosure.

[0052] Figure 18 This diagram schematically illustrates a model management interface in an exemplary embodiment of the present disclosure.

[0053] Figure 19 The diagram schematically illustrates the interface for creating a new training task in an exemplary embodiment of this disclosure;

[0054] Figure 20This diagram schematically illustrates an interface for modifying training parameters in an exemplary embodiment of this disclosure.

[0055] Figure 21 A schematic diagram illustrating the display interface of the model training process in an exemplary embodiment of the present disclosure;

[0056] Figure 22 This schematic diagram illustrates the data flow of a defect detection system in an exemplary embodiment of the present disclosure.

[0057] Figure 23 This schematic diagram illustrates the composition of a defect detection apparatus in an exemplary embodiment of the present disclosure.

[0058] Figure 24 This schematic diagram illustrates the composition of a model training apparatus according to an exemplary embodiment of the present disclosure;

[0059] Figure 25 This schematic diagram illustrates the composition of another model training apparatus in an exemplary embodiment of the present disclosure. Detailed Implementation

[0060] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0061] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0062] Figure 1 A schematic diagram of a system architecture for an exemplary application environment in which an embodiment of the defect detection method and apparatus of this disclosure can be applied is shown.

[0063] like Figure 1As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables. Terminal devices 101, 102, and 103 may be various electronic devices with defect detection functions, including but not limited to desktop computers, laptops, smartphones, and tablets. It should be understood that... Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0064] The defect detection method provided in this embodiment is generally executed by terminal devices 101, 102, and 103, and correspondingly, the defect detection device is generally disposed in terminal devices 101, 102, and 103. However, those skilled in the art will readily understand that the defect detection method provided in this embodiment can also be executed by server 105, and correspondingly, the defect detection device can also be disposed in server 105. This exemplary embodiment does not impose any special limitations on this. For example, in one exemplary embodiment, a user may obtain a sample dataset containing defective product data through terminal devices 101, 102, and 103, identify the feature information of the sample dataset, including the number of samples in the sample dataset, and then upload the original deep data sample dataset to server 105. The server obtains an initial model and configures training parameters based on the feature information; uses the sample dataset and trains the initial model according to the training parameters to obtain a target model; inputs the real data of the defective product corresponding to the sample dataset into the target model to obtain the defect information of the defective product, and transmits the defect information to terminal devices 101, 102, and 103, etc.

[0065] Exemplary embodiments of this disclosure provide an electronic device for implementing a defect detection method, which may be... Figure 1 The terminal device 101, 102, 103 or server 105 in the system. The electronic device includes at least a processor and a memory, the memory being used to store executable instructions of the processor, the processor being configured to perform a defect detection method by executing the executable instructions.

[0066] In one exemplary embodiment of this disclosure, the system architecture described above can be a distributed product defect analysis system, or a system formed by a group of computers interconnected via a network to exchange messages and communicate, and to coordinate their behavior. Components interact with each other to achieve a common goal. The network can be an Internet of Things (IoT) based on the Internet and / or telecommunications networks, which can be wired or wireless, for example, a local area network (LAN), metropolitan area network (MAN), wide area network (WAN), cellular data communication network, or other electronic networks capable of information exchange. The distributed computing system can have software components, such as software objects or other types of individually addressable isolated entities, such as distributed objects, agents, actors, virtual components, etc. Typically, each such component is individually addressable and has a unique identity within the distributed system (such as an integer, GUID, string, or opaque data structure). In geographically distributed systems, applications can be deployed and reside within a cluster. Various system, component, and network configurations exist to support distributed computing environments. For example, computing systems can be connected together via wired or wireless systems, through local networks, or widely distributed networks. Currently, many networks are coupled to the Internet, which provides the infrastructure for widely distributed computing, and include many different networks, although any network infrastructure can be used for communication that is prone to occur in the system, as described in various examples.

[0067] Distributed product defect analysis systems provide the sharing of computer resources and services through communication exchange between computing devices and systems. These resources and services include the exchange of information on objects (e.g., files), cache storage devices, and disk storage devices. They also include the sharing of processing power across multiple processing units for load balancing, resource scaling, and processing specialization. For example, a distributed product defect analysis system may include hosts with network topologies and infrastructures such as client / server, peer-to-peer, or hybrid architectures.

[0068] The following is based on Figure 2 Taking a mobile terminal 200 as an example, the construction of an electronic device will be described by way of example. Those skilled in the art will understand that, apart from components specifically designed for mobile purposes, Figure 2 The structure shown can also be applied to fixed-type devices. In other embodiments, the mobile terminal 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The components illustrated can be implemented in hardware, software, or a combination of software and hardware. The interface connections between the components are only schematic and do not constitute a limitation on the structure of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also adopt a similar design to... Figure 2 Different interface connection methods, or combinations of multiple interface connection methods.

[0069] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 210, internal memory 221, external memory interface 222, Universal Serial Bus (USB) interface 230, charging management module 240, power management module 241, battery 242, antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, audio module 270, speaker 271, receiver 272, microphone 273, headphone jack 274, sensor module 280, display screen 290, camera module 291, indicator 292, motor 293, buttons 294, and subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0070] Processor 210 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0071] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in mobile terminals, such as image recognition, facial recognition, speech recognition, and text understanding.

[0072] The processor 210 includes a memory. The memory can store instructions for implementing six modular functions: detection instructions, link instructions, information management instructions, analysis instructions, data transfer instructions, and notification instructions, and their execution is controlled by the processor 210.

[0073] The charging management module 240 receives charging input from the charger. The power management module 241 connects the battery 242, the charging management module 240, and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, internal memory 221, display screen 290, camera module 291, and wireless communication module 260, etc.

[0074] The wireless communication function of the mobile terminal 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals; mobile communication module 250 can provide wireless communication solutions including 2G / 3G / 4G / 5G for use on the mobile terminal 200; modem processor can include modulator and demodulator; wireless communication module 260 can provide wireless communication solutions including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks) and Bluetooth (BT) for use on the mobile terminal 200. In some embodiments, antenna 1 of the mobile terminal 200 is coupled to mobile communication module 250, and antenna 2 is coupled to wireless communication module 260, enabling the mobile terminal 200 to communicate with networks and other devices via wireless communication technology.

[0075] The mobile terminal 200 implements display functions through a GPU, a display screen 290, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0076] The mobile terminal 200 can perform shooting functions through an ISP, a camera module 291, a video codec, a GPU, a display screen 290, and an application processor. The ISP processes data fed back from the camera module 291; the camera module 291 captures still images or videos; the digital signal processor processes digital signals, including digital image signals and other digital signals; the video codec compresses or decompresses digital video, and the mobile terminal 200 can support one or more video codecs.

[0077] The external storage interface 222 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 200. The external storage card communicates with the processor 210 through the external storage interface 222 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0078] Internal memory 221 can be used to store executable program code, including instructions. Internal memory 221 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.). The data storage area may store data created during the use of the mobile terminal 200 (such as audio data, phonebook, etc.). Furthermore, internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, Universal Flash Storage (UFS), etc. Processor 210 executes various functional applications and data processing of the mobile terminal 200 by running instructions stored in internal memory 221 and / or instructions stored in memory located in the processor.

[0079] The mobile terminal 200 can implement audio functions, such as music playback and recording, through an audio module 270, a speaker 271, a receiver 272, a microphone 273, a headphone jack 274, and an application processor.

[0080] The depth sensor 2801 is used to acquire depth information of the scene. In some embodiments, the depth sensor may be disposed on the camera module 291.

[0081] The pressure sensor 2802 is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 2802 may be disposed on the display screen 290. There are many types of pressure sensors 2802, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0082] The gyroscope sensor 2803 can be used to determine the motion attitude of the mobile terminal 200. In some embodiments, the gyroscope sensor 2803 can determine the angular velocity of the mobile terminal 200 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 2803 can be used for image stabilization, navigation, motion-sensing game scenes, etc.

[0083] In addition, other sensors with different functions can be set in the sensor module 280 according to actual needs, such as barometric pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor, bone conduction sensor, etc.

[0084] The mobile terminal 200 may also include other devices that provide auxiliary functions. For example, buttons 294 may include a power button, volume buttons, etc., allowing users to input key signals related to user settings and function control of the mobile terminal 200. Other examples include indicators 292, motors 293, and SIM card interfaces 295.

[0085] In the field of screen manufacturing, problems with equipment, parameters, operation, and environmental interference can lead to defective products. After each process, optical image inspection (AOI) generates a large amount of image data, requiring professional operators to classify these images as defective. With the rise of artificial intelligence algorithms, such as deep learning, AI algorithms have been introduced into the defective image classification process, giving rise to the Automatic Image Detection (ADC) system.

[0086] The ADC system mainly consists of four subsystems: a data annotation system, a GPU Server (algorithm system), a TMS system, and a training system. To simplify business processes and conserve hardware resources, the first three subsystems combined can enable automatic detection of defective images using AI algorithm inference services in production lines. In other words, the system can operate normally even without the training system. However, the system cannot update the algorithm model; if an AI model update is needed, it must be developed and deployed by algorithm developers. The training system primarily facilitates algorithm training during project development and model updates and deployments during project maintenance.

[0087] In the factory process, certain adjustments are usually made to the production process and AOI equipment. Since deep learning algorithms are a data-driven technology, the adjustment of the production process or equipment will inevitably lead to changes in the AOI images, resulting in a decrease in the accuracy of the algorithm model. On the other hand, for the production of new products, the model must also be readjusted to adapt to the different AOI images corresponding to the new products.

[0088] To improve the robustness of ADC systems and ensure accuracy, the system needs to be able to promptly train new models after changes to the AOI (Area of ​​Interest) image. However, deep learning algorithms typically train models with a large number of training parameters, which often require manual adjustment for different images. This wastes human resources and carries the risk of loss due to human error.

[0089] Based on the above-mentioned shortcomings, this disclosure first provides a defect detection method, referring to... Figure 3 As shown, the above defect detection method may include the following steps:

[0090] Step S310: Obtain a sample dataset containing defective product data and identify the feature information of the sample dataset, wherein the feature information includes the number of samples in the sample dataset;

[0091] Step S320: Obtain an initial model, wherein the initial model is a neural network model;

[0092] Step S330: Configure training parameters based on the feature information;

[0093] Step S340: Using the sample dataset and the training parameters, train the initial model to obtain the target model;

[0094] Step S350: Input the real data of the product corresponding to the sample dataset into the target model to obtain the defect information of the product.

[0095] The training parameters include at least one of the following: a learning rate descent strategy, a total number of training rounds, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of rounds during which the learning rate is descent; and the testing strategy includes the number of tests and the number of rounds during which the learning rate is tested.

[0096] Compared to related technologies, the technical solution provided in this embodiment determines the model's training parameters based on feature information obtained from the training data. On one hand, by utilizing the sample size in the feature information, the learning rate descent strategy, total number of training epochs, and testing strategy in the training parameters are determined, eliminating the need for manual operation, saving human resources, and avoiding losses due to human error. On the other hand, the learning rate descent strategy includes the number of learning rate descents and the number of epochs during descent; the testing strategy includes the number of tests and the number of epochs during testing. The learning rate descent strategy and the number of epochs during descent, as well as the number of tests and the number of epochs during testing, are configured. Since the learning rate descent strategy and the testing strategy have a significant impact on defect detection, configuring the above training parameters can greatly improve the accuracy of the target model in defect detection.

[0097] In step S310, a sample dataset containing defective product data is obtained, and the feature information of the sample dataset is identified, including the number of samples in the sample dataset.

[0098] In this example implementation, a sample dataset can be obtained first, and the feature information of the sample dataset can be identified. Specifically, the number of samples in the sample dataset can be obtained, as well as the types and sizes of the defective product images included in the sample dataset. In this example implementation, no specific limitation is made. The sample dataset may include defective product data, which may include images of defective products or other product data of defective products. In this example implementation, no specific limitation is made.

[0099] In this example implementation, the feature information of the above-mentioned sample dataset may include the type and size of the images of defective products in the sample data, and may also include the number of samples in the above-mentioned sample data. The number of samples may be 10,000, 20,000, etc., or may be customized according to the user's needs. No specific limitation is made in this example implementation.

[0100] In step S320, an initial model is obtained, wherein the initial model is a neural network model;

[0101] In this example implementation, the initial model may be a convolutional neural network (CNN) model, an object detection convolutional neural network (faster-RCNN) model, a recurrent neural network (RNN) model, or a generative adversarial network (GAN) model, but is not limited to these, and may also employ other neural network models known to those skilled in the art.

[0102] In this example implementation, the initial model can be determined based on the type of defective images in the defective products. Specifically, in this example implementation, depending on actual business needs, the same or different initial models can be selected based on images generated by different processes or product types. For example, when the images in the sample dataset are intermediate site images, the initial model can be a convolutional neural network (CNN) model; when the images in the sample dataset are final site images, the initial model can be a convolutional neural network (CNN) model for object detection (Faster-RCNN) model. No specific limitations are made in this real-time example.

[0103] In step S330, training parameters are configured based on the feature information;

[0104] In this real-time example, the training parameters may include a learning rate descent strategy, a total number of training epochs, and a testing strategy. Configuring the training parameters based on the aforementioned feature information can be achieved by configuring the learning rate descent strategy, the total number of training epochs, and the testing strategy based on the number of samples in the feature information. The learning rate descent strategy includes the number of times the learning rate is decreased and the number of epochs during the decrease; the testing strategy includes the number of tests and the number of epochs during the tests.

[0105] Specifically, the total number of training rounds is positively correlated with the number of samples. For example, if the number of samples is less than or equal to 10,000, the total number of training rounds is configured to 300,000; if the number of samples is greater than 10,000, the total number of training rounds is configured using the following formula:

[0106]

[0107] Where Y represents the total number of training rounds, X represents the number of samples (greater than or equal to 10,000), INT is the floor function, and b represents the growth factor, a fixed value (greater than or equal to 30,000 and less than or equal to 70,000). In this real-time example, the value of b can be either 50,000 or 60,000; no specific limitation is made in this example implementation. In this example implementation, the mapping relationship between the number of samples and the total number of training rounds can be the optimal result obtained through multiple trials, or it can be customized according to user needs; no specific limitation is made in this example implementation.

[0108] In this example implementation, the number of epochs during which the learning rate decreases is positively correlated with the total number of training epochs. Specifically, the number of epochs during testing is greater than or equal to the number of epochs during the first learning rate decrease and less than or equal to the total number of epochs in the training group. The learning rate decreases multiple times, with at least two tests performed within a preset number of epochs for the second learning rate decrease. Two, three, or more tests can be performed, and this example implementation does not impose a specific limitation. Performing multiple learning rate decreases during training and selecting the optimal number of decreases after multiple decreases improves the accuracy of the obtained target model, thereby enhancing the accuracy of defect detection. Furthermore, by performing multiple tests on the model during training, the model with the optimal test results can be selected as the target model, further improving the accuracy of defect detection.

[0109] In this example implementation, the learning rate reduction method can be piecewise constant decay, exponential decay, natural exponential decay, cosine decay, etc., and is not specifically limited in this example implementation. The learning rate reduction magnitude is related to the learning rate reduction method and to each parameter in the configured learning rate reduction method. It can also be a constant such as 0.1 or 0.05, and is not specifically limited in this example real-time mode.

[0110] In one example embodiment of this disclosure, the aforementioned feature information may include the size and type of the images of the defective products in the sample dataset. Configuring training parameters based on the aforementioned feature information may also include configuring the size of the input image to the initial model based on the size and type of the images.

[0111] Specifically, if the type of image of the defective product is an AOI color image or a DM image, then the size of the input image is a first preset multiple of the size of the defective product image; if the type of image of the defective product is a TDI image, then the size of the input image is a second preset multiple of the size of the defective product image; wherein, the first preset multiple is less than or equal to 1, and the second preset multiple is greater than or equal to 1.

[0112] In this example implementation, the first preset multiple can be greater than or equal to 0.25 and less than or equal to 0.6; the second preset multiple can be greater than or equal to 3 and less than or equal to 6. For example, the technical indicator of input image size is mainly determined by the dataset name (i.e., the image type and site to which the dataset belongs). For AOI color images from the SD&Final&mask site, the average original image size is 2000*2000, so their input image size can be any one of 500, 688, 864, 1000, and 1200. For TDI grayscale images, the average original image size is 64*64, so their input image size can be any one of 192, 208, 224, 240, and 256. The size of the input image can also be customized according to the user's needs, and is not specifically limited in this example implementation.

[0113] In another example implementation, the input image size is primarily determined by the dataset name (i.e., the image type and website to which the dataset belongs). For AOI color images from the SD&Final&mask website, the average original image size is 2000*2000, so their input image sizes can be multiples of 500, 688, 864, 1000, and 1200. For TDI grayscale images, the average original image size is 64*64, so their input image sizes can be multiples of 192, 208, 224, 240, and 256. That is, the number of input images is greater than the number of original images; in this case, the number of samples can be the aforementioned number of input images.

[0114] In one example embodiment of this disclosure, the feature information may further include the defect level of the defective product and the number of samples corresponding to each defect. The training parameters may further include confidence levels, wherein the confidence levels during the training process can be configured according to the number of samples corresponding to each defect and the defect level.

[0115] Specifically, a preset quantity can be set first, and the sample size corresponding to each of the above defects can be compared with the preset quantity. If the sample size corresponding to the above defect is greater than the preset quantity, then a confidence level is configured according to the defect level. The above defect levels include a first defect level and a second defect level. If the defect level is the first defect level, then the confidence level is configured as the first confidence level; if the defect level is the second defect level, then the confidence level is configured as the second confidence level, wherein the second confidence level can be greater than the first confidence level.

[0116] In this example implementation, the preset quantity can be 50, 100, etc., or it can be customized according to user needs. No specific limitation is made in this example implementation. The first confidence level is greater than or equal to 0.6 and less than or equal to 0.7; the second confidence level is greater than or equal to 0.8 and less than or equal to 0.9. The specific values ​​of the first and second confidence levels can be customized according to user needs. No specific limitation is made in this example implementation.

[0117] For example, for defects with a high incidence but low importance (i.e., low defect severity), a lower confidence level can be configured. For instance, for defect-free maps (PI820) and minor defects (PI800), the confidence level is set to 0.6. This means that if the probability score of the map in PI800 or PI820 exceeds 0.6, it is considered a defect. Conversely, for defects with a low incidence but high importance (i.e., high defect severity), a higher confidence level can be configured. For instance, for severe defects (GT011 and SD011), the default confidence level is set to 0.85. This means that if the probability score of the map in GT011 or SD011 exceeds 0.6, it is considered a defect. All other maps with lower confidence levels are classified as "unknown" (not recognized by AI) and handled manually to prevent missed detections.

[0118] In one example embodiment of this disclosure, after configuring the training parameters according to the aforementioned feature information, the method may further include generating a training parameter display interface, and setting a parameter modification flag in the training parameter display interface. After the user triggers the parameter modification flag, the modifiable parameters can be displayed, and the user can modify the configured training parameters in the modification interface.

[0119] Step S340: Using the sample dataset and the training parameters, train the initial model to obtain the target model;

[0120] In one example implementation of this disclosure, after configuring and modifying the above-mentioned training parameters, the obtained initial model can be trained using the above-mentioned sample dataset to obtain the target model.

[0121] The target model is primarily based on a deep learning neural network model. For example, the target model can be based on a feedforward neural network. A feedforward network can be implemented as an acyclic graph, where nodes are arranged in layers. Typically, the feedforward network topology includes an input layer and an output layer, separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for generating the output in the output layer. Network nodes are fully connected to nodes in adjacent layers via edges, but there are no edges between nodes within a single layer. Data received at nodes in the input layer of the feedforward network is propagated (i.e., “feedforward”) to nodes in the output layer via activation functions that compute the state of nodes in each consecutive layer of the network based on coefficients (“weights”), which are associated with each of the edges connecting these layers. The output of the target model can take various forms, and this disclosure is not limiting in this regard. The target model can also include other neural network models, such as, but not limited to, convolutional neural network (CNN) models, recurrent neural network (RNN) models, generative adversarial network (GAN) models, and other neural network models known to those skilled in the art.

[0122] The initial training using sample data described above may include the following steps: selecting a network topology; using a set of training data representing the problem being modeled by the network; and adjusting the weights until the network model exhibits minimum error for all instances in the training dataset. For example, during supervised learning training for a neural network, the output generated by the network in response to inputs representing instances in the training dataset is compared to the “correct” labeled output of that instance; an error signal representing the difference between the output and the labeled output is calculated; and the weights associated with the connections are adjusted to minimize the error as the error signal is backpropagated through the layers of the network. When the error of each output generated from the instances in the training dataset is minimized, the initial model is considered “trained” and defined as the target model.

[0123] In this example implementation, when training the initial model, the defect detection method may further include acquiring the loss curve during the training process, and then adjusting the training parameters based on the loss curve. Specifically, referring to... Figure 4As shown, the horizontal axis of the loss curve represents the number of training epochs, and the vertical axis represents the loss value. During model training, this loss curve is updated in real time according to the training status. Users can observe the loss curve and adjust the training parameters based on its state. Specifically, if the loss curve remains chaotic and does not show a downward trend, it indicates that the training parameters are not configured appropriately, and training should be stopped, the learning rate parameter and the learning rate descent strategy adjusted, and training retrained. If the loss curve decreases slowly, it should be continuously observed, training stopped, or the initial learning rate increased in the next training session. If the loss curve still shows a downward trend after training (normally it should eventually smooth out), the maximum number of training epochs should be increased after training, and retraining should be performed.

[0124] In one exemplary embodiment of this disclosure, according to the testing strategy, the output of the model that has reached the required number of training epochs and the required number of testing epochs is used as a reference model. Then, a target model is selected from multiple reference models based on their accuracy and recall. Further, the accuracy and recall corresponding to each defect in each reference model can be obtained, and a confusion matrix for each reference model is constructed based on the accuracy and recall corresponding to each defect. The target model is then obtained based on the confusion matrix. When determining the target model, the F1 score of each reference model can also be obtained, and the target model is obtained by referring to both the F1 score and the confusion matrix. This exemplary embodiment does not impose specific limitations on this method.

[0125] Specifically, the optimal reference model can be selected as the target model based on the accuracy and recall of multiple reference models in the confusion matrix. For example, the reference model with the highest accuracy and recall can be selected as the target model. In this example implementation, no specific limitation is made.

[0126] In one example embodiment of this disclosure, the method may further include modifying the confidence level based on the confusion matrix. Specifically, the precision and recall of each defect in the confusion matrix can be analyzed in detail, and the confidence level can be adjusted after the model is deployed, based on specific business needs, thereby adjusting the precision and recall of the deployed model for a specific defect. For example, the recall of PI800 in the current confusion matrix is ​​0.90, which is generated with a default confidence level of 0.8. Since PI800 is a non-critical defect, a certain amount of over-judgment is permissible. In order to improve the recall of this defect in the production line, the confidence level of PI800 can be set to 0.6-0.7 when the model is deployed. This can increase the recall of PI800 in production to 0.91-0.92. Correspondingly, the precision of PI800 in production will decrease by 0.01-0.02, while the increase in recall will reduce the workload of the image interpreters. Therefore, users can perform a detailed analysis of the confusion matrix before the model goes live, based on the confusion matrix and production requirements, and thus customize the confidence level of each defect.

[0127] Step S350: Input the real data of the defective products corresponding to the sample dataset into the target model to obtain the defect information of the defective products.

[0128] In this example implementation, after obtaining the target model, the real data of the product corresponding to the sample data is transmitted to the target model, and the defect information of the product is obtained using the target model. The real data of the product may include data of the product to be tested corresponding to the product defect data in the sample dataset.

[0129] This disclosure also provides a model training method, referring to... Figure 5 As shown, it may include the following steps:

[0130] Step S510: Obtain a sample dataset containing defective product data and identify the feature information of the sample dataset, wherein the feature information includes the number of samples in the sample dataset;

[0131] Step S520: Obtain an initial model, wherein the initial model is a neural network model;

[0132] Step S530: Configure training parameters based on the feature information;

[0133] Step S540: Using the sample dataset, train the initial model according to the training parameters to obtain a target model. The target model is used to detect defects in the real data of the products corresponding to the sample dataset.

[0134] The training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during which the learning rate is descented; and the testing strategy includes the number of tests and the number of epochs during which the learning rate is tested.

[0135] In step S510, a sample dataset is acquired, and the feature information of the sample dataset is identified, including the number of samples in the sample dataset.

[0136] In one example embodiment of this disclosure, the feature information of the sample dataset may include the type and size of the images of defective products in the sample data, and may also include the number of samples in the sample data. The number of samples may be 10,000, 20,000, etc., or may be customized according to the user's needs. No specific limitation is made in this example embodiment.

[0137] In step S520, an initial model is obtained, wherein the initial model is a neural network model;

[0138] In this example implementation, the initial model may be a convolutional neural network (CNN) model, an object detection convolutional neural network (faster-RCNN) model, a recurrent neural network (RNN) model, or a generative adversarial network (GAN) model, but is not limited to these, and may also employ other neural network models known to those skilled in the art.

[0139] In this example implementation, the initial model can be determined based on the type of defective images in the defective products. Specifically, in this example implementation, three types of images can be involved, depending on actual business needs: final site images (SD_final site), intermediate site images (mask site), and TDI grayscale images. Different initial models can be selected based on different images. For example, when the images in the sample dataset are intermediate site images, the initial model can be a convolutional neural network (CNN) model; when the images in the sample dataset are final site images, the initial model can be a convolutional neural network (CNN) model for object detection (Faster-RCNN) model. In this example of real-time implementation, no specific limitations are made.

[0140] Step S530: Configure training parameters based on the feature information.

[0141] In this real-time example, the training parameters may include a learning rate descent strategy, a total number of training epochs, and a testing strategy. Configuring the training parameters based on the aforementioned feature information can be achieved by configuring the learning rate descent strategy, the total number of training epochs, and the testing strategy based on the number of samples in the feature information. The learning rate descent strategy includes the number of times the learning rate is decreased and the number of epochs during the decrease; the testing strategy includes the number of tests and the number of epochs during the tests.

[0142] Specifically, the total number of training rounds is positively correlated with the number of samples. For example, if the number of samples is less than or equal to 10,000, the total number of training rounds is configured to 300,000; if the number of samples is greater than 10,000, the total number of training rounds is configured using the following formula:

[0143]

[0144] Where Y represents the total number of training rounds, X represents the number of samples (greater than or equal to 10,000), INT is the floor function, and b represents the growth factor, a fixed value (greater than or equal to 30,000 and less than or equal to 70,000). In this real-time example, the value of b can be either 50,000 or 60,000; no specific limitation is made in this example implementation. In this example implementation, the mapping relationship between the number of samples and the total number of training rounds can be the optimal result obtained through multiple trials, or it can be customized according to user needs; no specific limitation is made in this example implementation.

[0145] In this example implementation, the number of epochs during which the learning rate decreases is positively correlated with the total number of training epochs. Specifically, the number of epochs during testing is greater than or equal to the number of epochs during the first learning rate decrease and less than or equal to the total number of epochs in the training group. The learning rate decreases multiple times, with at least two tests performed within a preset number of epochs for the second learning rate decrease. Two, three, or more tests can be performed, and this example implementation does not impose a specific limitation. Performing multiple learning rate decreases during training and selecting the optimal number of decreases after multiple decreases improves the accuracy of the obtained target model, thereby enhancing the accuracy of defect detection. Furthermore, by performing multiple tests on the model during training, the model with the optimal test results can be selected as the target model, further improving the accuracy of defect detection.

[0146] In this example implementation, the learning rate reduction method can be piecewise constant decay, exponential decay, natural exponential decay, cosine decay, etc., and is not specifically limited in this example implementation. The learning rate reduction magnitude is related to the learning rate reduction method and to each parameter in the configured learning rate reduction method. It can also be a constant such as 0.1 or 0.05, and is not specifically limited in this example real-time mode.

[0147] For details, please refer to the configuration method in the defect detection method above, which will not be repeated here.

[0148] Step S540: Using the sample dataset, train the initial model according to the training parameters to obtain a target model. The target model is used to detect defects in the real data of the products corresponding to the sample dataset.

[0149] In one example implementation of this disclosure, after configuring and modifying the above-mentioned training parameters, the obtained initial model can be trained using the above-mentioned sample dataset to obtain the target model.

[0150] The target model is primarily based on a deep learning neural network model. For example, the target model can be based on a feedforward neural network. A feedforward network can be implemented as an acyclic graph, where nodes are arranged in layers. Typically, the feedforward network topology includes an input layer and an output layer, separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for generating the output in the output layer. Network nodes are fully connected to nodes in adjacent layers via edges, but there are no edges between nodes within a single layer. Data received at nodes in the input layer of the feedforward network is propagated (i.e., “feedforward”) to nodes in the output layer via activation functions that compute the state of nodes in each consecutive layer of the network based on coefficients (“weights”), which are associated with each of the edges connecting these layers. The output of the target model can take various forms, and this disclosure is not limiting in this regard. The target model can also include other neural network models, such as, but not limited to, convolutional neural network (CNN) models, recurrent neural network (RNN) models, generative adversarial network (GAN) models, and other neural network models known to those skilled in the art.

[0151] The initial training using sample data described above may include the following steps: selecting a network topology; using a set of training data representing the problem being modeled by the network; and adjusting the weights until the network model exhibits minimum error for all instances in the training dataset. For example, during supervised learning training for a neural network, the output generated by the network in response to inputs representing instances in the training dataset is compared to the “correct” labeled output of that instance; an error signal representing the difference between the output and the labeled output is calculated; and the weights associated with the connections are adjusted to minimize the error as the error signal is backpropagated through the layers of the network. When the error of each output generated from the instances in the training dataset is minimized, the initial model is considered “trained” and defined as the target model.

[0152] In this example implementation, when training the initial model, the defect detection method may further include acquiring the loss curve during the training process, and then adjusting the training parameters based on the inspection curve. Specifically, referring to... Figure 4 As shown, the horizontal axis of the loss curve represents the number of training epochs, and the vertical axis represents the loss value. During model training, this loss curve is updated in real time according to the training status. Users can observe the loss curve and adjust the training parameters based on its state. Specifically, if the loss curve remains chaotic and does not show a downward trend, it indicates that the training parameters are not configured appropriately, and training should be stopped, the learning rate parameter and the learning rate descent strategy adjusted, and training retrained. If the loss curve decreases slowly, it should be continuously observed, training stopped, or the initial learning rate increased in the next training session. If the loss curve still shows a downward trend after training (normally it should eventually smooth out), the maximum number of training epochs should be increased after training, and retraining should be performed.

[0153] In one exemplary embodiment of this disclosure, according to the testing strategy, the output of the model that has reached the required number of training epochs and the required number of testing epochs is used as a reference model. Then, a target model is selected from multiple reference models based on their accuracy and recall. Further, the accuracy and recall corresponding to each defect in each reference model can be obtained, and a confusion matrix for each reference model is constructed based on the accuracy and recall corresponding to each defect. The target model is then obtained based on the confusion matrix. When determining the target model, the F1 score of each reference model can also be obtained, and the target model is obtained by referring to both the F1 score and the confusion matrix. This exemplary embodiment does not impose specific limitations on this method.

[0154] Specifically, the optimal reference model can be selected as the target model based on the accuracy and recall of multiple reference models in the confusion matrix. For example, the reference model with the highest accuracy and recall can be selected as the target model. This example implementation does not impose any specific limitations. The specific details of training the initial model can be found in the section on training multiple initial models in the defect detection method above, and will not be repeated here.

[0155] This disclosure also provides a model training method, referring to... Figure 6 As shown, the model training method may include the following steps:

[0156] Step S610: In response to the user's configuration operation on the parameters of the sample dataset, obtain the sample dataset containing defective product data, and identify the feature information of the sample dataset, the feature information including the number of samples in the sample dataset;

[0157] Step S620: Obtain an initial model, wherein the initial model is a neural network model;

[0158] Step S630: Configure training parameters according to the feature information and generate a training parameter display interface;

[0159] Step S640: Using the sample dataset, train the initial model according to the training parameters to obtain a target model. The target model is used to detect defects in the real data of the products corresponding to the sample dataset.

[0160] The training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during which the learning rate is descented; and the testing strategy includes the number of tests and the number of epochs during which the learning rate is tested.

[0161] The steps described above are explained in detail below.

[0162] In step S610, in response to the user's configuration operation on the parameters of the sample dataset, a sample dataset containing defective product data is obtained, and the feature information of the sample dataset is identified, including the number of samples in the sample dataset.

[0163] In one example implementation of this disclosure, sample data can be acquired in response to a user's configuration operation on the parameters of the sample dataset. For example, multiple sample data corresponding to defects and acquisition identifiers corresponding to the sample data can be displayed in a graphical user interface. When the user triggers the acquisition identifier, the sample data corresponding to the acquisition identifier can be acquired.

[0164] In this example implementation, refer to Figure 20 As shown, in response to the user's task creation operation, a training task is created, and a parameter configuration interface for the sample dataset is generated. The user can configure parameters for the sample dataset in this interface. These parameters may include department, section, site, image type, product, training type, etc. Then, in response to the user's parameter configuration operation, the system automatically obtains the sample dataset corresponding to the sample dataset parameters and identifies the feature information in the sample data. In another example implementation, after obtaining the sample data, a training task corresponding to the sample dataset can be created based on the parameters of the sample dataset; however, this example implementation does not impose specific limitations.

[0165] The specific details of extracting feature information from the above sample dataset have already been explained in detail above, so they will not be repeated here.

[0166] In step S620, an initial model is obtained, wherein the initial model is a neural network model.

[0167] In this example implementation, the specific details of obtaining the initial model can be found in the description of the defect detection method, and will not be repeated here.

[0168] In step S630, training parameters are configured according to the feature information, and a training parameter display interface is generated;

[0169] In one example embodiment of this disclosure, the specific details of configuring training parameters based on the feature information have been described in detail in the above-described defect detection method, and therefore will not be repeated here.

[0170] In one exemplary embodiment of this disclosure, reference is made to Figure 20 As shown, the sample configuration interface includes a training parameter viewing identifier. The server can generate the training parameter display interface in response to the user's trigger operation on the training parameter viewing identifier. In another example embodiment of this invention, refer to... Figure 7 As shown, after configuring the above training parameters, a training parameter display interface can be directly generated. The training parameter display interface includes the configuration information of each of the above training parameters and the parameter modification identifier.

[0171] The training parameters mentioned above may include the total number of training rounds, learning rate reduction strategy, testing strategy, confidence level, and the size of the image input to the initial model. Other parameters may also be included, but are not specifically limited in this example implementation.

[0172] In this example implementation, the specific details of the total number of training rounds, the learning rate reduction strategy, the testing strategy, the confidence level, and the size of the image input to the initial model have been explained in detail in the above defect detection method, and therefore will not be repeated here.

[0173] In this example implementation, refer to Figure 7 As shown, the server can respond to user triggers of the above parameter modification flags, modify the above training parameters, and after the modification flag is triggered, configure each training parameter to a modifiable state and generate a confirmation flag. After the user triggers the confirmation flag, the modification of the above training parameters is completed. Training parameters can be modified via an interactive interface without directly modifying the code, making it easier for system administrators without programming skills to operate and improving operational convenience.

[0174] In one example embodiment of this disclosure, the feature information may further include the defect level of the defective product and the number of samples corresponding to each defect. The training parameters may further include confidence level. The confidence level during the training process can be configured according to the number of samples corresponding to each defect and the defect level. The parameter display interface also includes a confidence level configuration identifier.

[0175] In this example implementation, refer to Figure 8 As shown, a confidence configuration interface can be generated in response to a user's trigger operation on the confidence configuration identifier. This interface includes the sample quantity for each defect and the selection identifier for each defect. The confidence configuration interface is used to respond to the user's confirmation operation on the selection identifier, configuring the confidence level of the defect corresponding to the selection operation. Specifically, in response to the user's confirmation operation on the selection identifier of a defect whose sample quantity in the confidence configuration interface is greater than a preset quantity, the confidence level is configured according to the defect level. The defect level includes a first defect level and a second defect level. If the defect level is the first defect level, the confidence level is configured as the first confidence level; if the defect level is the second defect level, the confidence level is configured as the second confidence level. The second confidence level can be greater than the first confidence level.

[0176] In this example implementation, the preset quantity can be 50, 100, etc., or it can be customized according to user needs. No specific limitation is made in this example implementation. The first confidence level is greater than or equal to 0.6 and less than or equal to 0.7; the second confidence level is greater than or equal to 0.8 and less than or equal to 0.9. The specific values ​​of the first and second confidence levels can be customized according to user needs. No specific limitation is made in this example implementation.

[0177] For example, for defects with a high incidence but low importance, a lower confidence level can be configured. For instance, for defect-free maps (PI820) and minor defects (PI800), the confidence level is set to 0.6, meaning the map is considered a defect if its probability score in PI800 or PI820 exceeds 0.6. Conversely, for defects with a low incidence but high importance, a higher confidence level can be configured. For example, for severe defects (GT011 and SD011), the default confidence level is set to 0.85, meaning the map is considered a defect only if its probability score in GT011 or SD011 exceeds 0.6. All other maps with lower confidence levels are considered unidentified and processed manually to prevent missed detections.

[0178] Step S640: Using the sample dataset, train the initial model according to the training parameters to obtain a target model. The target model is used to detect defects in the real data of the products corresponding to the sample dataset.

[0179] In this example implementation, the specific training process has been described in detail in the above defect detection method, so it will not be repeated here.

[0180] In one exemplary embodiment of this disclosure, reference is made to Figure 9As shown, when training the initial model, a training progress table can be generated and displayed. The training progress table can include task detail flags and task cancellation flags. When the user triggers the task detail flag, a loss curve during the training process is generated and displayed. The user can then adjust the training parameters based on the loss curve.

[0181] The specific details of adjusting the training parameters based on the loss curve have been explained in detail in the defect detection method described above, and therefore will not be repeated here. In this example implementation, training of the initial model is stopped when the user triggers the task cancellation flag.

[0182] In one exemplary embodiment of this disclosure, reference is made to Figure 10 As shown,

[0183] In one exemplary embodiment of this disclosure, according to the testing strategy, the model output that has reached the required number of training epochs and testing epochs is used as a reference model. Then, a target model is selected from multiple reference models based on their accuracy and recall. Further, the accuracy and recall corresponding to each defect in each reference model can be obtained. Then, a confusion matrix for each reference model is constructed based on the accuracy and recall corresponding to each defect. The target model is obtained based on the confusion matrix, and a confusion matrix display interface is generated. When determining the target model, the F1 score of each reference model can also be obtained, and the target model is obtained by referring to both the F1 score and the confusion matrix. This exemplary embodiment does not impose specific limitations on this method.

[0184] Specifically, refer to Figure 11 As shown, based on the accuracy and recall of multiple reference models in the confusion matrix, the optimal reference model is selected as the target model. Specifically, in response to the user's selection operation of the multiple reference models, the reference model corresponding to the selection operation is determined as the target model. For example, the reference model with the highest accuracy and recall is selected as the target model. This example implementation does not impose specific limitations. The user can select the target model by clicking the "OK" button after making a selection in the selection bar.

[0185] In one example implementation of this disclosure, the user can respond to the user's modification operation on the aforementioned confidence level by updating the confidence level according to the confusion matrix. Specifically, the precision and recall of each defect in the confusion matrix can be analyzed in detail, and the confidence level can be adjusted after the model goes live, based on specific business needs, thereby adjusting the precision and recall of the live model for specific defects. For example, the recall of PI800 in the current confusion matrix is ​​0.90, which is generated with a default confidence level of 0.8. Since PI800 is a non-critical defect, a certain amount of over-judgment is permissible. In order to improve the recall of this defect in the production line, the confidence level of PI800 can be set to 0.6-0.7 when the model goes live. This can increase the recall of PI800 in production to 0.91-0.92. Correspondingly, the precision of PI800 in production will decrease by 0.01-0.02, while the increase in recall will reduce the workload of the image interpreters. Therefore, users can perform a detailed analysis of the confusion matrix before the model goes live, based on the confusion matrix and production requirements, and thus customize the confidence level of each defect.

[0186] Furthermore, this disclosure also provides a defect detection system, referring to... Figure 12 As shown, the above system may include a data management module 1202, a model management module 1204, and a training management module 1203. The data management module 1202 is configured to store and manage sample data; the training management module 1203 is configured to execute the above defect detection method and model training method; and the model management module 1204 is configured to store, display, and manage the target model.

[0187] In one example embodiment of this disclosure, the defect detection system may further include a user management module, wherein the user management module 1201 is configured to perform CRUD operations on user information, as well as permission management and / or password management.

[0188] Reference Figure 13 As shown, the system adopts a B / S or C / S architecture, consisting of a backend server 1306, a frontend server 1303, and shared storage 1304. The operation is handled by the factory PC 1307 via a web browser. As a subsystem responsible for training tasks within the ADC system, the training system communicates with the ADC system's data annotation system, TMS system, and GPU Server (algorithm system). The training system can be externally connected to the data annotation system and TMS system, providing the TMS system 1305 with updated target models and other related services.

[0189] The training system and data annotation system 1302 interact with each other via a database and shared storage 1304 (NAS network storage) for data and image exchange. The training system communicates with the GPU Server (algorithm system) 1301 via TCP / IP protocol to control the GPU Server 1301 for model training and automatic testing. The training system transfers model data with the TMS system 1305 via FTP protocol and exchanges model information via the database. Internally, the training system uses HTTP protocol for interaction between front-end and back-end services and the web interface.

[0190] The following is a detailed explanation of each of the above modules.

[0191] Specifically, the user management module manages the system's users and system information. It handles the creation, deletion, modification, and querying of user information, access control, password management, and provides CRUD functions for information such as applicable departments, offices, and sites. The user management module includes user information and system information. Users can enter their username and password to access the training system. Upon entry, they are directed to the completed training module, allowing users to directly view the training status of existing models. Users can view all currently managed users and manage system permissions, enabling the addition, modification, and deletion of users.

[0192] In one example embodiment of this disclosure, the data management module is configured to store and manage sample data. Since deep learning AI algorithms are data-driven, the differences in production processes and AOI (Automated Optical Inspection) devices in factories lead to significant intra-class and inter-class variations in data, making it difficult to solve all problems using a single general model. Therefore, separate models can be trained for specific data to comprehensively cover real-time inference services in factory production.

[0193] This dataset management module processes these datasets according to a unified standard, thus facilitating model training. Dataset management is divided into training dataset management and preparation dataset management. The data in preparation dataset management consists of raw data labeled by the data annotation system. After statistical verification by the user, it can be imported into training dataset management. Data in training dataset management can be directly submitted for corresponding training tasks to train the model.

[0194] Specifically, refer to Figure 14 As shown, data annotated by the data annotation system can be automatically synchronized (or manually synchronized) to the dataset management preparation dataset management. In the dataset management, the original dataset can be synchronized and statistically displayed. The dataset management mainly displays detailed information such as the product, defect type and quantity, number of images, the defect with the fewest images, and update time for each dataset.

[0195] Dataset management can be manually maintained. Clicking the synchronization icon will directly synchronize the data labeled by the data annotation system. If manual synchronization is not performed, the system can automatically synchronize the data daily. Clicking the statistics icon will provide detailed information for each dataset, including the defect categories in each dataset, the number of images in each category, and a distribution table.

[0196] When managing the training dataset, refer to Figure 15 As shown, the prepared dataset can be statistically validated and then used for functions such as creating new rows, modifying existing rows, and managing images. This allows for a series of operations on the dataset to generate a sample dataset that can be used to train a model.

[0197] There is a correspondence between sample datasets and models. Each sample dataset, paired with different training parameters, can train a different model, allowing users to select a model based on relevant metrics. Therefore, the first step in preparing for model training is to create a new sample dataset for the model to be trained on this interface.

[0198] It should be noted that the naming of sample datasets in the defect detection system follows specific rules. The system automatically configures training parameters based on the feature information of the sample datasets. Since the name of the sample dataset is also a feature of the dataset, it must adhere to specific rules. These rules are as follows: For models on the SD / Final site, the Sub-defect model is named "product_product_name", and the main defect model is named "maindefect_product_name" (where the product name is two letters or numbers). For example, the sub-defect dataset for ak is named product_ak; the main defect dataset for ak is named maindefect_ak.

[0199] In this real-time example, for the Mask site model, the Mask site sub-defect model is named "mask_site name" (where the site name is four letters or numbers). The Mask site does not have a main defect model. For example, the model dataset for site 1500 is named mask_1500.

[0200] In this example of real-time mode, for the TDI model, which is currently applied to the SD / Final site and has no main defect model, the naming convention is "tdi_tdiX" (where X is a number). For example, the model name for a regular product is tdi_tdi1.

[0201] Reference Figure 16As shown, after selecting the department, section, and site, the user needs to manually enter the sample dataset name into the system according to the naming rules mentioned above. Once entered, the dataset creation is complete. After clicking the change icon, enter the new dataset name and click OK to complete the dataset name change. Clicking the delete icon next to the corresponding sample dataset will delete the sample dataset. Here, deletion refers to deleting the data imported into this module. (Refer to...) Figure 17 As shown, clicking the details icon displays detailed information about each sample dataset, including its location, internal structure, and previews of the images. The middle column shows the directory structure of the images in the sample dataset, and the rightmost column lists the images in the selected defect category folder, allowing users to preview and delete images.

[0202] The model management module is configured to store, display, and manage the target model; specifically, refer to... Figure 18 As shown, this can include changing the name of the target model, deleting it, and displaying details. After the user clicks the details icon, all version information for a specific model will be displayed, including the model name, model version number, whether the model is online, the trainer, the training completion time, the model's accuracy, recall, and path. Clicking the delete icon is used to delete a trained target model.

[0203] The training management module enables the submission of training tasks, adaptive configuration of training parameters, management of training tasks, and display of automatic training and testing results. Among these, adaptive parameter configuration and automatic training and testing are features not found in traditional training systems. These two functions significantly reduce the difficulty of training algorithm models by incorporating the parameter tuning experience of algorithm users during development into the system logic to achieve adaptive parameter adjustment and automatic testing. This allows even maintenance personnel and users without algorithm development backgrounds to use this module to train models that achieve production line accuracy. The training management module is mainly divided into three sub-modules: trained, training in progress, and canceled.

[0204] In this example implementation, refer to Figure 19 As shown. Regardless of whether you are in the "Trained," "Training," or "Cancelled" screen, you can click the "Create New" icon to submit a training task. It's important to note that submitting a training task requires either creating a corresponding sample dataset or having the system directly obtain a sample dataset related to the training task described above.

[0205] To facilitate model training, refer to Figure 20As shown, considering the complex production scenarios where the defect detection system operates, and the data characteristics of large intra-class and inter-class differences, the training system has the function of automatically configuring training parameters based on the dataset features. After selecting "Site," "Image Type," "Product" (product name naming rules are shown in the table below), and "Training Type," click "View / Modify" after "Training Parameters." After a few seconds, the relevant configuration parameters for the model will automatically pop up. For details, please refer to... Figure 7 The content shown.

[0206] Table 1

[0207]

[0208] The naming rules for the product name can be found in Table 1. Select the relevant dataset, initial model (optional), and input model name (optional). Select the training type as main defect and sub defect (the correspondence between training type and image type is shown in Table 2). After modifying the training parameters, click OK.

[0209] Table 2

[0210]

[0211] Reference Figure 7 and Figure 8 As shown, the defect settings icon on the interface needs to be clicked. In the pop-up window, select the options based on the number of images in the dataset and modify the confidence level as needed. The specific selection rule is that if the number of samples is greater than or equal to the preset number, the option should be selected. The explanation of the preset number has been detailed in the defect detection method section above, so it will not be repeated here.

[0212] In this example implementation, in such Figure 8 When the displayed page pops up, the system will provide default values ​​for the confidence levels of each defect based on the algorithm user's experience. These default values ​​are determined through multiple tests based on the severity and occurrence rate of defects provided by the business. For some defects with high occurrence rates but low severity, the confidence levels are appropriately relaxed.

[0213] For example, for defect-free PI820 maps and slightly defective PI800 maps, the confidence level is set to 0.6. This means that if the probability score of the map in PI800 or PI820 exceeds 0.6, it is considered a defect. However, for certain defects with low occurrence rates but high importance, the confidence level is strictly controlled. For example, for severe defects GT011 and SD011, the default confidence level is set to 0.85. This means that only maps with a probability score exceeding 0.6 in GT011 or SD011 are considered defects. All other maps with low confidence levels are classified as "unknown" (not recognized by AI) and handled manually to prevent missed detections.

[0214] As mentioned above, for all the defects that pop up, the algorithm user conducts multiple experiments based on business needs, and selects the optimal confidence level for the default configuration. During the training process, the user can adjust the confidence level configuration according to the above conditions.

[0215] Reference Figure 9 As shown, the training management module allows you to view tasks in training, including the dataset name, trainer, training reason, current training epoch, and total number of training epochs. The operations section includes two icons: Cancel and Details. Clicking the Cancel icon will cancel the current training task; see reference... Figure 4 As shown, clicking the details icon will generate and display the training loss curve for the current training task. The trend of the loss curve can be used to judge the training effect of the model to a certain extent.

[0216] In this example implementation, the horizontal axis of the loss curve represents the number of training epochs, and the vertical axis represents the loss value. During model training, this loss curve is updated in real time according to the training status. Users can observe the loss curve and adjust the training parameters based on its state. Specifically, if the loss curve remains chaotic and does not show a downward trend, it indicates that the training parameters are not configured appropriately, and training should be stopped, the learning rate parameter and the learning rate descent strategy adjusted, and retraining performed. If the loss curve decreases slowly, it should be continuously observed, training stopped, or the initial learning rate increased in the next training session. If the loss curve still shows a downward trend after training (normally it should eventually smooth out), the maximum number of training epochs should be increased after training is completed, and retraining should be performed.

[0217] Confidence setting principle: In the confidence input box of the training parameters interface, you can set the default confidence level for all defects. If you select "Train Defects" and leave the confidence levels for each defect in the confidence setting interface blank, the default values ​​from the previous page will be used. If you enter a value, the confidence level from that interface will be used.

[0218] Reference Figure 21 As shown, the training management module can also display trained models, completed training tasks, including the model name generated for each dataset (this model is a model with different training rounds corresponding to a dataset, not just a single model; the models selected for testing when submitting training will all be displayed here), the training completion time, and the reason for training.

[0219] Reference Figure 19As shown, the operation includes two icons: "View Results" and "Select Optimal." Clicking "View Results" will pop up confusion matrices for multiple models (the number of test rounds depends on the initial training parameter settings; the default is 6 models). This function supplements the traditional training system by converting algorithm-related metrics into more intuitive tables for data display, making it easier for operations personnel or users to view the data in a familiar table format and select models based on their preferred metrics. Clicking the "Select Optimal" button will display the test results for each model, with the most important metrics being accuracy, recall, and F1 score. Based on accuracy, recall, F1 score, and the aforementioned confusion matrices, users can decide whether to deploy a model.

[0220] Beyond guiding model deployment, users can also analyze the precision and recall of each defect in the confusion matrix in detail. Based on specific business needs, they can adjust the confidence level after model deployment, thereby adjusting the precision and recall of the deployed model for specific defects. For example, if the current recall of PI800 in the confusion matrix is ​​0.90, this recall is generated with a default confidence level of 0.8. Since PI800 is a non-critical defect, a certain amount of over-judgment is permissible. To improve the recall of this defect in the production line, the confidence level of PI800 can be set to 0.6-0.7 when the model is deployed. This will increase the recall of PI800 in production to 0.91-0.92. Correspondingly, the precision of PI800 in production will decrease by 0.01-0.02, while the increased recall will reduce the workload of the image interpreters. Therefore, users can conduct a detailed analysis of the confusion matrix before model deployment based on the confusion matrix and production requirements, thereby customizing the confidence level of each defect. After selecting and confirming the model, you can see the model in the model management interface of the TMS system and then put it into production (it can also be seen in the model management function module).

[0221] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0222] In this example implementation, refer to Figure 22As shown, the sample dataset produced by the data annotation system 2201 is used to call the defect detection system 2202 of this disclosure to train the corresponding model, which is then uploaded to the TMS system 2203 for model inference service. The specific process is detailed below, with users organizing and annotating training data in the data annotation system. By embedding AI supervised algorithms and traditional unsupervised algorithms, automatic and semi-automatic data annotation functions are achieved, greatly reducing the workload of data annotation. The data annotated by this module is automatically synchronized to the relevant directory in the training management module. The training sample dataset is imported into the training management module. A new sample dataset name needs to be created according to the training task, and the annotated data from the data annotation system needs to be imported. Simultaneously, relevant information is written to the database. After the data import is complete, details can be viewed, and data can be modified or deleted. The training task is submitted. After the training set is imported, the corresponding training task is submitted in the training task management module. During the submission of training tasks, users need to select information such as the site, image type, and training type based on the feature information of the sample dataset. After selecting the relevant information, the training management module can adaptively adjust the training parameters, displaying the configured training parameters on the interface for user reference. Users can choose to use the default parameters or modify them themselves. After completion, the training task can be submitted. Automatic model training and testing: After submitting the training task, the training management module sends the training task and configured training parameters to the AI ​​algorithm system. Subsequently, the AI ​​algorithm system automatically trains the model according to the received task and performs automatic model testing after training. Various indicator changes during the training process are plotted as icons and displayed in the system so users can monitor the training status at any time. Processing training test results: After training, the AI ​​algorithm system stores the model in shared storage and sends the relevant training results to the training management module. After receiving the message from the AI ​​algorithm system, the training management module displays the training results in the "Trained" section. Users can view various indicators, confusion matrix, and some necessary training indicator analyses after automatic model testing. Model pre-deployment. Based on the above metrics, users can select the optimal model for pre-deployment, which will then be synchronized to the model management module of TMS system 2203. The pre-deployed model can be easily viewed and managed in the model management module of the training management module. Model Deployment: The pre-deployed model has been synchronized to the model management database of TMS system 2203, and users can perform offline testing and formal deployment of the model within the TMS system.

[0223] Further reference Figure 23As shown, this example embodiment also provides a defect detection device 2300, including a first acquisition module 2310, a first configuration module 2320, a first training module 2330, and a detection module 2340. The first acquisition module 2310 is used to acquire a sample dataset containing defective product data and identify feature information of the sample dataset, the feature information including the number of samples in the sample dataset; and acquire an initial model, wherein the initial model is a neural network model; the first configuration module 2320 is used to train parameters according to the feature information; the first training module 2330 is used to use the sample dataset and, according to the training parameters, train the initial model to obtain a target model; and the detection module 2340 is used to input the real data of the product corresponding to the sample dataset into the target model to obtain the defect information of the product; wherein the training parameters include at least one of the following: a learning rate descent strategy, a total number of training rounds, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of rounds during descent; the testing strategy includes the number of tests and the number of rounds during testing.

[0224] Further reference Figure 24 As shown, this example embodiment also provides a model training device 2400, including a second acquisition module 2410, a second configuration module 2420, and a second training module 2430. The second acquisition module 2410 is used to acquire a sample dataset containing defective product data and identify feature information of the sample dataset, the feature information including the number of samples in the sample dataset; acquire an initial model, wherein the initial model is a neural network model; the second configuration module 2420 is used to configure training parameters based on the feature information; the second training module 2430 is used to train the initial model using the sample dataset according to the training parameters to obtain a target model, the target model being used to perform defect detection on real data of the products corresponding to the sample dataset; wherein the training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during descent; the testing strategy includes the number of tests and the number of epochs during testing.

[0225] Further reference Figure 25As shown, this example embodiment also provides a model training device 2500, including a third acquisition module 2510, a third configuration module 2520, and a third training module 2530. The third acquisition module 2510 responds to a user's configuration operation on parameters of a sample dataset, acquires a sample dataset containing defective product data, and identifies feature information of the sample dataset, including the number of samples in the sample dataset; acquires an initial model, wherein the initial model is a neural network model; the third configuration module 2520 configures training parameters according to the feature information and generates a training parameter display interface; the third training module 2530 trains the initial model using the sample dataset and the training parameters to obtain a target model, the target model being used to perform defect detection on real data of the products corresponding to the sample dataset; wherein the training parameters displayed in the training parameter display interface include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during descent; the testing strategy includes the number of tests and the number of epochs during testing.

[0226] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0227] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0228] Exemplary embodiments of this disclosure also provide a computer-readable storage medium having a program product stored thereon capable of implementing the methods described above in this specification. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0229] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0230] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0231] Furthermore, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0232] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0233] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A defect detection method, wherein, include: Obtain a sample dataset containing defective product data and identify the feature information of the sample dataset, wherein the defective product data includes images of defective products, and the feature information includes the number of samples in the sample dataset, as well as the size and type of the images of defective products in the sample dataset; Obtain an initial model, wherein the initial model is a neural network model; Configure training parameters based on the aforementioned feature information; Using the sample dataset and training the initial model according to the training parameters, a target model is obtained; The actual data of the products corresponding to the sample dataset are input into the target model to obtain the defect information of the products; The training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during which the learning rate is descent; and the testing strategy includes the number of tests and the number of epochs during which the learning rate is tested. The configuration of training parameters based on the feature information further includes: adjusting the size of the input image of the initial model according to the size and type of the defective product image; The step of adjusting the size of the input image of the initial model according to the size and type of the defective product images includes: If the image of the defective product is an AOI color image or a DM image, then the size of the input image is a first preset multiple of the size of the image of the defective product. If the image of the defective product is a TDI image, then the size of the input image is a second preset multiple of the size of the image of the defective product; wherein the first preset multiple is less than or equal to 1, and the second preset multiple is greater than or equal to 1.

2. The method according to claim 1, wherein, The total number of training rounds is positively correlated with the number of samples.

3. The method according to claim 1, wherein, The total number of training rounds is configured according to the number of samples and preset rules, including: If the number of samples is less than or equal to 10,000, then the total number of training rounds is configured to 300,000; If the number of samples is greater than 10,000, the total number of training rounds is configured using the following formula: The formula for determining the total number of rounds is: ； Where Y represents the total number of training rounds, X represents the number of samples, INT is the floor function, b represents the growth factor, which is a fixed value, and b is greater than or equal to 30000 and less than or equal to 70000.

4. The method according to claim 1, wherein, The number of rounds in which the learning rate decreases is positively correlated with the total number of training rounds; the number of rounds in the test is greater than or equal to the number of rounds in which the learning rate first decreases and less than or equal to the total number of training rounds.

5. The method according to claim 1, wherein, The learning rate decreases multiple times, with at least two tests conducted within a preset number of rounds from the second learning rate decrease round.

6. The method according to claim 5, wherein, The learning rate decreases three times, and at least three tests are conducted within a preset number of rounds from the second learning rate decrease round.

7. The method according to claim 1, wherein, The learning rate reduction strategy includes the method of reducing the learning rate and the magnitude of the reduction.

8. The method according to claim 1, wherein, The first preset multiple is greater than or equal to 0.25 and less than or equal to 0.6; the second preset multiple is greater than or equal to 3 and less than or equal to 6.

9. The method according to claim 1 or 8, wherein, The input images include multiple sizes of the same image of the defective product.

10. The method according to claim 1, wherein, The feature information also includes the defect level of the defective product, and configuring training parameters based on the feature information includes: The confidence level during the training process is configured according to the defect level corresponding to the defective product.

11. The method according to claim 10, wherein, The defect levels include a first defect level and a second defect level. The confidence level during the training process is configured according to the defect level corresponding to the defective product, including: If the defect level is the first defect level, then the confidence level is configured to the first confidence level; If the defect level is the second defect level, then the confidence level is configured to the second confidence level; The second confidence level is greater than the first confidence level.

12. The method according to claim 11, wherein, The first confidence level is greater than or equal to 0.6 and less than or equal to 0.7; the second confidence level is greater than or equal to 0.8 and less than or equal to 0.

9.

13. The method according to claim 1, wherein, The method further includes: The target model is labeled using the size, type, and process of the images of the defective products.

14. The method according to claim 1, wherein, include: The initial model is obtained based on the type of image of the defective product.

15. The method according to claim 1, wherein, After configuring the training parameters based on the feature information, the method further includes: Generate a training parameter display interface, which includes parameter modification identifiers; The training parameters are updated in response to a user's trigger action on the parameter modification flag.

16. The method according to claim 1, wherein, The method further includes: Obtain the loss curve during the training process; The training parameters are updated based on the loss curve.

17. The method according to claim 1, wherein, The target model is obtained by training the initial model according to the training parameters, including: According to the test strategy, multiple reference models are obtained, and the accuracy and recall of the multiple reference models are obtained; The target model is determined from the reference models based on the accuracy and recall of each reference model.

18. The method according to claim 1, wherein, The target model is obtained by training the initial model according to the training parameters, including: Multiple reference models are obtained according to the test strategy, and the confusion matrix of each reference model is determined. The target model is determined from the reference model based on the confusion matrix.

19. The method according to claim 18, wherein, The method further includes: The confidence level is updated based on the confusion matrix.

20. A model training method, wherein, include: Obtain a sample dataset containing defective product data and identify the feature information of the sample dataset, wherein the defective product data includes images of defective products, and the feature information includes the number of samples in the sample dataset, as well as the size and type of the images of defective products in the sample dataset; Obtain an initial model, wherein the initial model is a neural network model; Configure training parameters based on the aforementioned feature information; The initial model is trained using the sample dataset and the training parameters to obtain a target model, which is used to detect defects in the real data of the products corresponding to the sample dataset. The training parameters include at least one of the following: a learning rate descent strategy, a total number of training epochs, and a testing strategy, wherein the learning rate descent strategy includes the number of learning rate descents and the number of epochs during which the learning rate is descent; and the testing strategy includes the number of tests and the number of epochs during which the learning rate is tested. The configuration of training parameters based on the feature information further includes: adjusting the size of the input image of the initial model according to the size and type of the defective product image; The step of adjusting the size of the input image of the initial model according to the size and type of the defective product images includes: If the image of the defective product is an AOI color image or a DM image, then the size of the input image is a first preset multiple of the size of the image of the defective product. If the image of the defective product is a TDI image, then the size of the input image is a second preset multiple of the size of the image of the defective product; wherein the first preset multiple is less than or equal to 1, and the second preset multiple is greater than or equal to 1.

21. The method according to claim 20, wherein, The total number of training rounds is positively correlated with the number of samples.

22. The method according to claim 20, wherein, The total number of training rounds is configured according to the number of samples and preset rules, including: If the number of samples is less than or equal to 10,000, then the total number of training rounds is configured to 300,000; If the number of samples is greater than 10,000, the total number of training rounds is configured using the following formula: The formula for determining the total number of rounds is: ； Where Y represents the total number of training rounds, X represents the number of samples, INT is the floor function, b represents the growth factor, which is a fixed value, and b is greater than or equal to 30000 and less than or equal to 70000.

23. The method of claim 20, wherein, The number of rounds in which the learning rate decreases is positively correlated with the total number of training rounds; the number of rounds in the test is greater than or equal to the number of rounds in which the learning rate first decreases and less than or equal to the total number of training rounds.

24. The method of claim 20, wherein, The learning rate decreases multiple times, with at least two tests conducted within a preset number of rounds from the second learning rate decrease round.

25. The method according to claim 20, wherein, The learning rate decreases three times, and at least three tests are conducted within a preset number of rounds from the second learning rate decrease round.

26. The method of claim 20, wherein, The learning rate reduction strategy includes the method of reducing the learning rate and the magnitude of the reduction.

27. A model training method, wherein, include: In response to the user's configuration operation on the parameters of the sample dataset, the system obtains a sample dataset containing defective product data and identifies the feature information of the sample dataset. The defective product data includes images of defective products, and the feature information includes the number of samples in the sample dataset, as well as the size and type of the images of defective products in the sample dataset. Obtain an initial model, wherein the initial model is a neural network model; Configure training parameters based on the aforementioned feature information and generate a training parameter display interface; The initial model is trained using the sample dataset and the training parameters to obtain a target model, which is used to detect defects in the real data of the products corresponding to the sample dataset. The training parameters displayed on the training parameter display interface include at least one of the following: learning rate descent strategy, total number of training epochs, and testing strategy, wherein the learning rate descent strategy includes the number of times the learning rate is decreased and the number of epochs during the decrease; the testing strategy includes the number of tests and the number of epochs during the test. The configuration of training parameters based on the feature information further includes: adjusting the size of the input image of the initial model according to the size and type of the defective product image; The step of adjusting the size of the input image of the initial model according to the size and type of the defective product images includes: If the image of the defective product is an AOI color image or a DM image, then the size of the input image is a first preset multiple of the size of the image of the defective product. If the image of the defective product is a TDI image, then the size of the input image is a second preset multiple of the size of the image of the defective product; wherein the first preset multiple is less than or equal to 1, and the second preset multiple is greater than or equal to 1.

28. The method according to claim 27, wherein, The method further includes: A training task corresponding to the sample dataset is established based on the parameters of the sample dataset.

29. The method according to claim 27, wherein, Before obtaining the sample dataset containing defective product data in response to a user's configuration operation on the parameters of the sample dataset, the method further includes: In response to the user's task creation operation, a training task is created, and a parameter configuration interface for the sample dataset is generated.

30. The method according to claim 29, wherein, The parameter configuration interface also includes a training parameter viewing identifier. Training parameters are configured based on the feature information, and a training parameter display interface is generated, including: In response to the user's trigger operation on the training parameter viewing identifier, a training parameter display interface is generated.

31. The method according to claim 27, wherein, The training parameter display interface includes parameter modification indicators, and the method further includes: The training parameters are updated in response to a user's trigger action on the parameter modification flag.

32. The method according to claim 27, wherein, The training parameters also include confidence level and the size of the image input to the initial model.

33. The method according to claim 32, wherein, The parameter display interface includes a confidence level configuration identifier, and the method further includes: In response to the user's trigger operation on the confidence configuration identifier, a confidence configuration interface is generated.

34. The method according to claim 33, wherein, The confidence configuration interface includes the number of samples corresponding to each defect and the selection identifier corresponding to each defect. The confidence configuration interface is used to respond to the user's confirmation operation on the selection identifier and configure the confidence level of the defect corresponding to the confirmation operation.

35. The method according to claim 27, wherein, The method further includes: Generate and display a model training progress table, which includes a task cancellation flag and a task detail flag. In response to the user's triggering of the task cancellation flag, training of the initial model is stopped.

36. The method according to claim 35, wherein, The method further includes: In response to the user's trigger operation on the task details identifier, generate and display the loss curve of the training process.

37. The method of claim 27, wherein, The target model is obtained by training the initial model according to the training parameters, including: According to the testing strategy, multiple reference models are obtained, and the accuracy and recall of the multiple reference models are obtained, and a list of reference models is generated; so that the target model can be determined from the reference models based on the accuracy and recall of each reference model.

38. The method according to claim 27, wherein, The target model is obtained by training the initial model according to the training parameters, including: Multiple reference models are obtained according to the test strategy, and the confusion matrix of each reference model is determined. A confusion matrix display interface is generated so that the target model can be determined from the reference models based on the confusion matrix.

39. The method according to claim 38, wherein, The method further includes: Respond to the user's confidence modification operation and update the confidence based on the confusion matrix.

40. The method of claim 38, wherein, Determining the target model from the reference model based on the confusion matrix includes: In response to the user's selection of the reference model, the reference model corresponding to the selection operation is determined as the target model.

41. The method according to claim 27, wherein, The total number of training rounds is positively correlated with the number of samples.

42. The method according to claim 27, wherein, The total number of training rounds is configured according to the number of samples and preset rules, including: If the number of samples is less than or equal to 10,000, then the total number of training rounds is configured to 300,000; If the number of samples is greater than 10,000, the total number of training rounds is configured using the following formula: The formula for determining the total number of rounds is: ； Where Y represents the total number of training rounds, X represents the number of samples, INT is the floor function, b represents the growth factor, which is a fixed value, and b is greater than or equal to 30000 and less than or equal to 70000.

43. The method according to claim 27, wherein, The number of rounds in which the learning rate decreases is positively correlated with the total number of training rounds; the number of rounds in the test is greater than or equal to the number of rounds in which the learning rate first decreases and less than or equal to the total number of training rounds.

44. The method according to claim 43, wherein, The learning rate decreases multiple times, with at least two tests conducted within a preset number of rounds from the second learning rate decrease round.

45. The method according to claim 44, wherein, The learning rate decreases three times, and at least three tests are conducted within a preset number of rounds from the second learning rate decrease round.

46. ​​The method according to claim 27, wherein, The learning rate reduction strategy includes the method of reducing the learning rate and the magnitude of the reduction.

47. A defect detection system, wherein, include: The data management module is configured to store and manage sample data; The training management module is configured to execute the defect detection method as described in any one of claims 1 to 19, the model training method as described in any one of claims 20 to 26, or the model training method as described in any one of claims 27 to 46. The model management module is configured to store, display, and manage the target model.

48. The defect detection system according to claim 47, wherein, The system also includes: The user management module is configured to perform CRUD operations on user information, as well as permission management and / or password management.

49. An electronic device, wherein, include: processor; as well as A memory for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the defect detection method as described in any one of claims 1 to 19, the model training method as described in any one of claims 20 to 26, or the model training method as described in any one of claims 27 to 46.

Images