Model training methods, defect target detection methods, devices and storage media

Abstract

This application provides a model training method, a defect target detection method, an apparatus, and a storage medium, relating to the fields of intelligent manufacturing and artificial intelligence technology, which can improve the detection accuracy of small targets in liquid crystal panels. The method includes: determining multiple defect samples and a preset loss function; the parameters of the preset loss function include: coordinate information of the actual bounding box of the preset defect target, coordinate information of any predicted bounding box of the preset defect target, the actual area of ​​the preset defect target, and multiple predicted areas of the preset defect target; constructing a first neural network model based on the preset loss function; inputting multiple defect samples into the first neural network model for training to determine a defect target detection model; the defect target detection model is used to determine the coordinate information of the bounding box of the defect target in the acquired image of the target panel. The embodiments of this application are used in the model training process.

Description

Technical Field

[0001] This application relates to the fields of intelligent manufacturing and artificial intelligence technology, and in particular to a model training method, a defect target detection method, a device, and a storage medium. Background Technology

[0002] To improve panel quality, those skilled in the art have proposed a neural network model (e.g., the YOLOv5 model) to detect defects on the panel. This facilitates maintenance personnel in optimizing defective LCD panels, thereby improving the overall quality of the LCD panels.

[0003] Currently, existing YOLOv5 models are trained based on the compatible intersection-over-union (CIOU) loss function. However, the CIOU loss function includes the aspect ratio (length to width ratio) of the bounding box in its parameters. For smaller targets, which may be a single point, the aspect ratio is not available. This makes existing YOLOv5 models unsuitable for such smaller targets, or leads to significant discrepancies between the detection results and the actual values, thus reducing the accuracy of YOLOv5 models in detecting smaller targets in the panel. Summary of the Invention

[0004] This application provides a model training method, a defect target detection method, an apparatus, and a storage medium, which can improve the detection accuracy of small targets in a liquid crystal panel.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, this application provides a model training method, which includes: determining multiple defect samples and a preset loss function; a defect sample is a captured image of a defect panel; the parameters of the preset loss function include: coordinate information of the actual bounding box of a preset defect target, coordinate information of any predicted bounding box of the preset defect target, the actual area of ​​the preset defect target, and multiple predicted areas of the preset defect target; the actual area is determined based on the coordinate information of the actual bounding box; the multiple predicted areas are determined based on the coordinate information of the multiple predicted bounding boxes of the preset defect target; the preset defect target is a defect target in any one of the multiple defect samples; a first neural network model is constructed based on the preset loss function; multiple defect samples are input into the first neural network model for training to determine a defect target detection model; the defect target detection model is used to determine the coordinate information of the bounding box of the defect target in the captured image of the target panel.

[0007] The above technical solution brings at least the following beneficial effects: The model training method provided in this application constructs a first neural network model through a preset loss function, thus enabling the application to optimize the first neural network model using the preset loss function. The parameters of the preset loss function include: the coordinate information of the actual bounding box of the defective target, the coordinate information of any predicted bounding box of the defective target, the actual area of ​​the defective target, and multiple predicted areas of the defective target. As can be seen from the above, the preset loss function of this application relies on the coordinate information of the actual and predicted bounding boxes of the defective target, as well as the actual area and multiple predicted areas of the defective target, without excessively relying on the aspect ratio of the defective target's bounding box. Therefore, for some smaller targets, even if no aspect ratio exists, they can still be calculated and will not be lost. Thus, compared to the CIOU loss function of the prior art, the preset loss function of this application is more suitable for detecting smaller defective targets, improving the detection accuracy of smaller targets in the panel.

[0008] In one possible implementation, the pre-defined loss function satisfies the following formula:

[0009]

[0010] Where PDA is the loss function value; L is the Euclidean distance between the center point of the actual bounding box of the preset defect target and the center point of any predicted bounding box of the preset defect target; S is the sum of the actual area and multiple predicted areas;

[0011] L satisfies the following formula:

[0012]

[0013] Among them, (A) x1 A y1 ) and (A x2 A y2 (B) represents the two endpoints of any diagonal line within the actual bounding box; x1 B y1 ) and (B x2 B y2 () represents the two endpoints of any diagonal line in any predicted bounding box;

[0014] S satisfies the following formula:

[0015]

[0016] Where W is the width of any predicted bounding box; h is the length of any predicted bounding box; and R is the radius of any predicted bounding box.

[0017] In one possible implementation, a sampled image of a defect panel is acquired; the size of the sampled image is greater than or equal to a preset size; and the coordinate information of the actual bounding box of the defect target in the sampled image of the defect panel is determined by a target tool.

[0018] In one possible implementation, step 1 involves obtaining multiple training samples for the i-th training iteration; when i is 1, each training sample represents a defect sample; when i is greater than 1, each training sample includes: a defect sample and the coordinate information of the predicted bounding box of the defect target in a defect sample during the (i-1)-th training iteration; the predicted bounding box is the predicted bounding box among the multiple predicted bounding boxes whose loss function value is greater than or equal to a preset threshold; i is a positive integer; step 2 involves performing the following operation on each training sample among the multiple training samples to obtain the predicted bounding box of the defect target in each defect sample during the i-th training iteration. The coordinate information and the target value of the defect target in each defect sample during the i-th training are obtained. A training sample is input into the first neural network model during the i-th training to obtain the coordinate information of multiple first predicted bounding boxes. The coordinate information of these multiple first predicted bounding boxes represents the coordinate information of multiple predicted bounding boxes of the defect target in a defect sample during the i-th training. The loss function value of each first predicted bounding box is determined according to a preset loss function. The coordinate information of the first predicted bounding boxes whose loss function value is greater than or equal to a preset threshold is determined as the target prediction of the defect target in a defect sample during the i-th training. The steps are as follows: Step 3: Optimize the first neural network model trained in the i-th training session based on the target value of the defect target in each defect sample during the i-th training session, and determine the second neural network model trained in the i-th training session; Step 4: Determine the evaluation metrics of the second neural network model trained in the (i-1)-th training session and the second neural network model trained in the i-th training session, and determine the evaluation metrics of the second neural network model trained in the (i-1)-th training session and the second neural network model trained in the i-th training session. The neural network model with the better evaluation index is the third neural network model trained in the i-th training session; Step 5: If the current training round number is less than the preset training round number, then the coordinate information of the target prediction bounding box of the defect target in each defect sample during the i-th training session, and each defect sample are determined as multiple training samples for the (i+1)-th training session, and the third neural network model trained in the i-th training session is determined as the first neural network model trained in the (i+1)-th training session. Steps 1, 2, 3, 4, and 5 are executed in sequence; If the current training round number is equal to the preset training round number, then the third neural network model obtained in the last training session is determined as the defect target detection model.

[0019] In one possible implementation, multiple validation samples and the coordinate information of the actual bounding box of each validation sample are obtained; the multiple validation samples are input into a fourth neural network model to determine the coordinate information of the target predicted bounding box of the multiple validation samples; the fourth neural network model is either a first neural network model or a second neural network model; and the evaluation index of the fourth neural network model is determined based on the coordinate information of the actual bounding box of the multiple validation samples and the coordinate information of the target predicted bounding box of the multiple validation samples.

[0020] Secondly, this application provides a defect target detection method, applied to a defect target detection model including the first aspect and any possible implementation thereof, the method comprising: acquiring a sampled image of a target panel; inputting the sampled image into the defect target detection model; and determining the coordinate information of the bounding box of the defect target in the sampled image.

[0021] The above technical solution brings at least the following beneficial effects: The defect target detection method provided in this application can determine the coordinate information of the bounding box of the defect target in the acquisition image by constructing and optimizing the defect target detection model based on a preset loss function. The preset loss function of this application relies on the coordinate information of the actual bounding box and the predicted bounding box of the defect target, as well as the actual area and multiple predicted areas of the defect target. It does not overly depend on the aspect ratio of the bounding box of the defect target. In this way, even if some smaller targets do not have an aspect ratio, they can still be calculated and will not be lost. Therefore, compared with the CIOU loss function of the prior art, the preset loss function of this application is more suitable for the detection of smaller defect targets and improves the detection accuracy of smaller targets in the panel.

[0022] Thirdly, this application provides a model training apparatus, comprising: a processing unit; the processing unit being configured to: determine multiple defect samples and a preset loss function; a defect sample being an acquired image of a defect panel; the parameters of the preset loss function including: coordinate information of the actual bounding box of a preset defect target, coordinate information of any predicted bounding box of the preset defect target, the actual area of ​​the preset defect target, and multiple predicted areas of the preset defect target; the actual area being determined based on the coordinate information of the actual bounding box; the multiple predicted areas being determined based on the coordinate information of the multiple predicted bounding boxes of the preset defect target; the preset defect target being a defect target in any one of the multiple defect samples; constructing a first neural network model based on the preset loss function; inputting multiple defect samples into the first neural network model for training to determine a defect target detection model; the defect target detection model being used to determine the coordinate information of the bounding box of the defect target in the acquired image of the target panel.

[0023] In one possible implementation, the pre-defined loss function satisfies the following formula:

[0024]

[0025] Where PDA is the loss function value; L is the Euclidean distance between the center point of the actual bounding box of the preset defect target and the center point of any predicted bounding box of the preset defect target; S is the sum of the actual area and multiple predicted areas;

[0026] L satisfies the following formula:

[0027]

[0028] Among them, (A) x1 A y1 ) and (A x2 A y2 (B) represents the two endpoints of any diagonal line within the actual bounding box; x1 B y1 ) and (B x2 B y2 () represents the two endpoints of any diagonal line in any predicted bounding box;

[0029] S satisfies the following formula:

[0030]

[0031] Where W is the width of any predicted bounding box; h is the length of any predicted bounding box; and R is the radius of any predicted bounding box.

[0032] In one possible implementation, the processing unit is further configured to: acquire a sampled image of a defect panel; the size of the sampled image is greater than or equal to a preset size; and determine the coordinate information of the actual bounding box of the defect target in the sampled image of the defect panel using a target tool.

[0033] In one possible implementation, the processing unit is further configured to: Step 1, acquire multiple training samples for the i-th training iteration; when i is 1, one training sample is one defect sample; when i is greater than 1, one training sample includes: one defect sample, and the coordinate information of the target predicted bounding box of the defective target in one defect sample during the (i-1)-th training iteration; the target predicted bounding box is the predicted bounding box of the multiple predicted bounding boxes whose loss function value is greater than or equal to a preset threshold; i is a positive integer; Step 2, perform the following operation on each training sample among the multiple training samples to obtain the target bounding box of the defective target in each defect sample during the i-th training iteration. The coordinate information of the predicted bounding boxes and the target value of the defect target in each defect sample during the i-th training are used. A training sample is input into the first neural network model during the i-th training to obtain the coordinate information of multiple first predicted bounding boxes. The coordinate information of these multiple first predicted bounding boxes represents the coordinate information of multiple predicted bounding boxes of the defect target in a defect sample during the i-th training. A loss function value is determined for each of the multiple first predicted bounding boxes based on a preset loss function. The coordinate information of the first predicted bounding boxes whose loss function value is greater than or equal to a preset threshold is determined as the coordinate information of the defect target in a defect sample during the i-th training. Step 1: Calculate the coordinates of the target prediction bounding box and determine the loss function value of the target prediction bounding box of the defective target in a defective sample during the i-th training iteration. Step 2: Optimize the first neural network model trained in the i-th training iteration based on the target value of the defective target in each defective sample during the i-th training iteration, thus determining the second neural network model trained in the i-th training iteration. Step 3: Determine the evaluation metrics for the second neural network model trained in the (i-1)-th and i-th training iterations, and determine the second neural network model trained in the (i-1)-th and i-th training iterations. In the process, the neural network model with better evaluation metrics is the third neural network model trained in the i-th training session; Step 5: If the current training round number is less than the preset training round number, then the coordinate information of the target prediction bounding box of the defect target in each defect sample during the i-th training session, and each defect sample are determined as multiple training samples for the (i+1)-th training session, and the third neural network model trained in the i-th training session is determined as the first neural network model trained in the (i+1)-th training session. Steps 1, 2, 3, 4, and 5 are executed sequentially; If the current training round number is equal to the preset training round number, then the third neural network model obtained in the last training session is determined as the defect target detection model.

[0034] In one possible implementation, the processing unit is further configured to: acquire multiple validation samples and the coordinate information of the actual bounding box of each validation sample; input the multiple validation samples into a fourth neural network model to determine the coordinate information of the target predicted bounding box of the multiple validation samples; the fourth neural network model is either a first neural network model or a second neural network model; and determine the evaluation index of the fourth neural network model based on the coordinate information of the actual bounding box of the multiple validation samples and the coordinate information of the target predicted bounding box of the multiple validation samples.

[0035] Fourthly, this application provides a defect target detection device, applied in the defect target detection model described in the third aspect and any possible implementation of the third aspect. The device includes: a communication unit and a processing unit; the communication unit is used to acquire a sampled image of a target panel; the processing unit is used to input the sampled image into the defect target detection model and determine the coordinate information of the bounding box of the defect target in the sampled image.

[0036] Fifthly, this application provides a model training apparatus, which includes: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the model training method as described in the first aspect and any possible implementation of the first aspect.

[0037] Fifthly, this application provides a defect target detection device, which includes: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the defect target detection method as described in the second aspect.

[0038] In a seventh aspect, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform the model training method described in the first aspect and any possible implementation thereof, or the defect target detection method described in the second aspect.

[0039] Eighthly, this application provides a computer program product containing instructions that, when run on a model training device or a defect target detection device, causes the model training device or the defect target detection device to execute the model training method described in the first aspect and any possible implementation thereof, or the defect target detection method described in the second aspect.

[0040] Ninthly, this application provides a chip including a processor and a communication interface, the communication interface and the processor being coupled together, the processor being used to run computer programs or instructions to implement the model training method described in the first aspect and any possible implementation of the first aspect, or the defect target detection method described in the second aspect.

[0041] Specifically, the chip provided in this application also includes a memory for storing computer programs or instructions. Attached Figure Description

[0042] Figure 1 A structural diagram of a computing device provided in an embodiment of this application;

[0043] Figure 2 A flowchart illustrating a model training method provided in this application embodiment;

[0044] Figure 3 A schematic diagram of an actual bounding box #1 and a predicted bounding box #2 provided for an embodiment of this application;

[0045] Figure 4 A flowchart illustrating another model training method provided in this application embodiment;

[0046] Figure 5 A flowchart illustrating another model training method provided in this application embodiment;

[0047] Figure 6 A flowchart illustrating another model training method provided in this application embodiment;

[0048] Figure 7 A flowchart of a defect target detection method provided in an embodiment of this application;

[0049] Figure 8 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0050] Figure 9 This is a schematic diagram of the structure of a defect target detection device provided in an embodiment of this application. Detailed Implementation

[0051] The model training method, defect target detection method, apparatus, and storage medium provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0052] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0053] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0054] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0055] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0056] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0057] The method provided in this application can be executed by an electronic device. The electronic device can be a terminal device, such as a wireless terminal in a banking system; it can also be a desktop computer, personal computer, or other similar device. The electronic device can also be a server, which can be a single server device or a server cluster. The server can also be a cloud server. This application does not limit the specific type of electronic device.

[0058] Figure 1 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Figure 1 As shown, the computing device 100 includes at least one processor 101, a communication line 102, and at least one communication interface 104, and may also include a memory 103. The processor 101, memory 103, and communication interface 104 are connected to each other via the communication line 102.

[0059] The processor 101 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0060] Communication line 102 may include a path for transmitting information between the aforementioned components.

[0061] The communication interface 104 is used to communicate with other devices or communication networks. It can use any transceiver-like device, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0062] The memory 103 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of including or storing desired program code having the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0063] In one possible design, the memory 103 can exist independently of the processor 101, meaning the memory 103 can be an external memory of the processor 101. In this case, the memory 103 can be connected to the processor 101 via a communication line 102 to store execution instructions or application code, and its execution is controlled by the processor 101 to implement the software upgrade method provided in the following embodiments of this application. In another possible design, the memory 103 can also be integrated with the processor 101, meaning the memory 103 can be an internal memory of the processor 101. For example, the memory 103 can be a cache, which can be used to temporarily store some data and instruction information.

[0064] As one possible implementation, processor 101 may include one or more CPUs, for example Figure 1 CPU0 and CPU1 in the example. Alternatively, the computing device 100 may include multiple processors, such as CPU0 and CPU1. Figure 1 The computing device 100 includes processors 101 and 107. Alternatively, the computing device 100 may also include output device 105 and input device 106.

[0065] Furthermore, the communication system described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new communication systems, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0066] With the rapid development of smart machines, the demand for their panels (such as LCD panels) has increased significantly, and the quality requirements for these panels are also rising. However, everyday operational errors, material damage, and accidental impacts can all lead to different types of defects on the panels, thereby reducing their quality.

[0067] To improve panel quality, those skilled in the art have proposed a display panel micro-defect detection method based on a two-stage detection network. This method is based on a two-stage network model detection (e.g., Faster region-convolutional neural networks (Faster R-CNN)). Although this method can effectively detect defects on LCD panels, the detection time is relatively long because it consists of two stages, and it cannot meet the requirements of real-time detection.

[0068] To improve both the efficiency and quality of defect detection, a defect detection method based on the YOLOv5 model has been proposed by those skilled in the art. This method comprises five stages: acquiring the target defect map, dividing the target defect map dataset, training the neural network model, validating the trained neural network model, and application of the detection. This method detects defects using only one stage of the neural network model, thus improving detection efficiency. Furthermore, the loss function in the YOLOv5 model training includes: defect target classification prediction, defect target bounding box prediction, and defect target confidence prediction. The comprehensive prediction of these three categories within the loss function enhances training capabilities, thereby improving the detection accuracy of the YOLOv5 model.

[0069] Currently, existing YOLOv5 models are trained based on the CIOU loss function. However, the CIOU loss function includes the aspect ratio (i.e., the ratio of length to width) of the bounding box in its parameters. For smaller targets, which may be a single point, the aspect ratio is not available. This makes existing YOLOv5 models unsuitable for such smaller targets, or leads to significant discrepancies between the detection results and the actual values, thus reducing the accuracy of YOLOv5 models in detecting smaller targets in the panel.

[0070] To address the problems existing in the prior art, this application proposes a model training method that can improve the detection accuracy of small targets in a liquid crystal panel. For example... Figure 2 As shown, the method includes:

[0071] S201. The computing device determines multiple defect samples and a preset loss function.

[0072] In this model, a defect sample is a captured image of a defect panel. The parameters of the preset loss function include: the coordinates of the actual bounding box of the preset defect target, the coordinates of any predicted bounding box of the preset defect target, the actual area of ​​the preset defect target, and multiple predicted areas of the preset defect target. The actual area is determined based on the coordinates of the actual bounding box. The multiple predicted areas are determined based on the coordinates of the multiple predicted bounding boxes of the preset defect target. The preset defect target is any defect target in any of the multiple defect samples.

[0073] In one possible implementation, the preset loss function satisfies the following formula 1:

[0074]

[0075] Where PDA is the loss function value. L is the Euclidean distance between the center point of the actual bounding box of the preset defect target and the center point of any predicted bounding box of the preset defect target. S is the sum of the actual area and multiple predicted areas.

[0076] As can be seen from Formula 1 above, the preset loss function is based on e, which can control the normalization of the loss function value (i.e., the loss function value is controlled within the range of [0, 1] by the preset loss function). The normalization of the loss function value can help optimize the first neural network model, which can improve the accuracy of the subsequently determined missing target detection model.

[0077] L satisfies the following formula 2:

[0078]

[0079] Among them, (A) x1 A y1 ) and (A x2 A y2 (B) represents the two endpoints of any diagonal line within the actual bounding box. x1 B y1 ) and (B x2 B y2 ) represents the two endpoints of any diagonal line in any prediction bounding box.

[0080] For example, such as Figure 3As shown, Figure 3 An actual bounding box #1 and a predicted bounding box #2 are shown. Figure 3 In this context, L represents the Euclidean distance between the center point of the actual bounding box #1 and the center point of the predicted bounding box #2.

[0081] S satisfies the following formula 3:

[0082]

[0083] Where W is the width of any predicted bounding box, h is the length of any predicted bounding box, and R is the radius of any predicted bounding box.

[0084] In some optional examples, when the area of ​​the defect target is large, the computing device can determine the bounding box of the defect target as a rectangle. When the area of ​​the defect target is small, the computing device can determine the bounding box of the defect target as a circle.

[0085] S202. The computing device constructs a first neural network model based on a preset loss function.

[0086] In one feasible implementation, the computing device also needs to configure model parameters during the process of building the first neural network model.

[0087] For example, model parameters include at least one of the following: batch size, number of training epochs, and learning rate.

[0088] Here, "batch" refers to the number of defective samples input into the first neural network model at one time.

[0089] For example, let's take a scenario with 10 defective samples and a batch size of 2 for the neural network model: The computing device needs to input the 10 defective samples in 5 separate inputs, with 2 defective samples input each time.

[0090] It's important to note that batch size can affect the optimization rate of the first neural network model. If the batch size is too small, the optimization will be slow, requiring more training epochs to obtain a satisfactory defect detection model. Conversely, if the batch size is too large, it will place a heavy processing burden on the first neural network model, potentially causing it to crash. Furthermore, inputting a large number of samples at once will also prolong the optimization time for that batch.

[0091] The training epoch refers to the number of training rounds of the first neural network model.

[0092] For example, the computing device may set the training epochs to 300. The above is merely an exemplary illustration of the number of training epochs, and the number of training epochs may also be other values ​​(e.g., 310), which are not limited in this application.

[0093] The learning rate is used to control the learning progress of the first neural network model.

[0094] It should be noted that an excessively large learning rate can cause the gradient of the loss function of the first neural network model to explode, resulting in oscillations, while an excessively small learning rate can cause the loss function of the first neural network model to overfit and reduce the convergence speed. Therefore, the computing device needs to set the learning rate within a reasonable range.

[0095] S203. The computing device inputs multiple defect samples into the first neural network model for training to determine the defect target detection model.

[0096] Among them, the defect target detection model is used to determine the coordinate information of the bounding box of the defect target in the acquisition image of the target panel.

[0097] As an optional implementation, the computing device can train and validate the first neural network model multiple times until the current training rounds reach the preset number of training rounds, at which point the computing device determines the last obtained neural network model as the defect target detection model.

[0098] In one example, the target panel described above can be a liquid crystal panel.

[0099] The above technical solution brings at least the following beneficial effects: The model training method provided in this application involves a computing device constructing a first neural network model using a preset loss function. This allows the computing device to optimize the first neural network model using the preset loss function. The parameters of the preset loss function include: the coordinate information of the actual bounding box of the defective target, the coordinate information of any predicted bounding box of the defective target, the actual area of ​​the defective target, and multiple predicted areas of the defective target. As can be seen from the above, the preset loss function of this application relies on the coordinate information of the actual and predicted bounding boxes of the defective target, as well as the actual area and multiple predicted areas of the defective target. It does not overly depend on the aspect ratio of the defective target's bounding box. Therefore, for some smaller targets, even if no aspect ratio exists, they can still be calculated and will not be lost. Thus, compared to the CIOU loss function of the prior art, the preset loss function of this application is more suitable for detecting smaller defective targets, improving the detection accuracy of smaller targets in the panel.

[0100] It should be noted that the parameters of the preset loss function include the coordinates of the actual bounding box of the defective target. Therefore, in order to construct the first neural network model based on the preset loss function, the computing device needs to determine the coordinates of the actual bounding box of the defective target beforehand. Figure 2 ,like Figure 4 As shown, the specific implementation process of the computing device determining the coordinate information of the actual bounding box of the defect target can be determined by the following steps S401 to S402.

[0101] S401. The computing device acquires a sample image of a defect panel.

[0102] The size of the acquired image is greater than or equal to the preset size.

[0103] In one feasible implementation, the specific implementation process of S401 is as follows: production personnel can obtain multiple images of panels from an image acquisition device (e.g., a high-definition camera) installed on the panel production line, filter out images of panels with defects from the multiple images of panels based on experience, and transmit the images of the filtered defective panels to a computing device.

[0104] Optionally, the computing device can classify the captured images of the aforementioned defective panels according to defect type to obtain at least one defect set (or defect folder). One defect set corresponds to one defect type.

[0105] For example, the preset size can be 1024×1024.

[0106] It should be noted that the above is only an exemplary description of a preset size, and the computing device may also set the preset size to other sizes, which is not limited in this application.

[0107] For example, the resolution of the acquired image is 2400×2400.

[0108] S402. The computing device uses a target tool to determine the coordinate information of the actual bounding box of the defect target in the acquisition image of a defect panel.

[0109] For example, the target tool could be the LabelImg tool.

[0110] Based on the above example, the specific implementation process of S402 is as follows: the computing device uses the LabelImg tool to annotate the defect target in the acquisition image of the above defect panel, generates an XML file with the actual bounding box location information of the defect target, and converts the above XML file format into a format adapted by the neural network model.

[0111] The above technical solution brings at least the following beneficial effects: In the model training method provided by this application, the size of the defect panel acquisition image obtained by the computing device needs to be greater than or equal to a preset size (e.g., 1024×1024). Compared with the 640×640 size acquisition image in the prior art, the defect panel acquisition image obtained by this application is less likely to be lost even after multiple downsampling processes. This allows the neural network model to still detect the features of the defect target in the defect panel acquisition image after multiple downsampling processes, thus ensuring the clarity of the defect target features as much as possible during the neural network model training stage. In an optional embodiment, such as in S203, the computing device needs to train the first neural network model to obtain a defect target detection model. Figure 2 Based on the illustrated method embodiments, this embodiment provides a possible implementation, such as... Figure 5 As shown, the specific implementation process of the computing device training the first network model may include the following steps S501 to S507.

[0112] S501, The computing device acquires multiple training samples for the i-th training session.

[0113] Where i is 1, a training sample is a single defect sample. When i is greater than 1, a training sample includes: a defect sample, and the coordinates of the predicted bounding box of the defect target in a defect sample during the (i-1)th training iteration. The predicted bounding box is the predicted bounding box among multiple predicted bounding boxes whose loss function value is greater than or equal to a preset threshold. i is a positive integer.

[0114] Understandably, when the first neural network model is initially trained, the computing device only needs to input multiple defect samples. During subsequent training, the computing device needs to input the coordinates of the predicted bounding boxes of each defect sample from the previous training iterations, along with the coordinates of all the defect samples themselves. This allows the first neural network model to refer to the bounding box coordinates from the previous training iterations when predicting the bounding boxes of the defect targets within these multiple defect samples, thus improving the accuracy of the bounding box coordinates obtained in this training iteration.

[0115] S502. The computing device performs the following operation on each of the multiple training samples to obtain the coordinate information of the target prediction bounding box of the defect target in each defect sample during the i-th training and the target value of the defect target in each defect sample during the i-th training.

[0116] The specific implementation process of the computing device determining the coordinate information of the target prediction bounding box of the defect target in a defect sample during the i-th training and the target value of the defect target in a defect sample during the i-th training can be determined by the following S5021-S5023.

[0117] S5021. The computing device inputs a training sample into the first neural network model during the i-th training to obtain the coordinate information of multiple first predicted bounding boxes.

[0118] Among them, the coordinate information of multiple first predicted bounding boxes is the coordinate information of multiple predicted bounding boxes of the defect target in a defect sample during the i-th training.

[0119] It should be noted that the aforementioned "one training sample" refers to any one of the multiple training samples mentioned above. Each training sample, when input into the first neural network model during the i-th training iteration, will yield the coordinate information of multiple predicted bounding boxes for the defective target within that training sample.

[0120] In one alternative scenario, the computing device needs to first set up a YOLOv5-based training environment before the first neural network model can be trained within this environment. This YOLOv5-based training environment uses PyTorch as its basic framework and Python as its programming language. The computing device needs to first build the YOLOv5-based training environment using PyTorch, then modify the input to train the first neural network model.

[0121] S5022. The computing device determines the loss function value of each of the multiple first prediction bounding boxes according to the preset loss function.

[0122] As an optional implementation, the specific implementation process of S5022 is as follows: The computing device can first obtain the coordinate information of the actual bounding box of the training sample, and then determine the actual area of ​​the training sample based on the coordinate information of the actual bounding box of the training sample and Formula 3 above. The computing device determines multiple predicted areas of the training sample based on the coordinate information of multiple first predicted bounding boxes and Formula 3 above. The computing device determines the Euclidean distance (denoted as the target Euclidean distance) between the center point of the actual bounding box of the training sample and the center point of any first predicted bounding box based on the coordinate information of the actual bounding box of the training sample, the coordinate information of any first predicted bounding box, and Formula 2 above. The computing device determines the loss function value of any first predicted bounding box based on the actual area of ​​the training sample, the multiple predicted areas of the training sample, the target Euclidean distance, and Formula 1 above.

[0123] S5023. The computing device determines that the coordinate information of the first predicted bounding box with a loss function value greater than or equal to a preset threshold in multiple first predicted bounding boxes is the coordinate information of the target predicted bounding box of the defect target in a defect sample during the i-th training, and determines that the loss function value of the target predicted bounding box of the defect target in a defect sample during the i-th training is the target value of the defect target in a defect sample during the i-th training.

[0124] As a feasible implementation, the specific implementation process of S5023 is as follows: The computing device determines the coordinate information of the first predicted bounding box with the highest loss function value from multiple first predicted bounding boxes as the coordinate information of the target predicted bounding box at the i-th training time, and determines the loss function value of the target predicted bounding box at the i-th training time as the target value of the defect target in the defect sample at the i-th training time. In this case, the preset threshold is the loss function value of the target predicted bounding box at the i-th training time.

[0125] S503. The computing device optimizes the first neural network model for the i-th training based on the target value of the defect target in each defect sample during the i-th training, and determines the second neural network model for the i-th training.

[0126] In one feasible scenario, the computing device optimizes the first neural network model trained in the i-th training by adjusting the parameters of the first neural network model trained in the i-th training, and determines the first neural network model trained in the i-th training with adjusted parameters as the second neural network model trained in the i-th training.

[0127] S504. The computing device determines the evaluation index of the second neural network model trained in the (i-1)th training session and the evaluation index of the second neural network model trained in the ith training session, and determines that the neural network model with the better evaluation index among the second neural network model trained in the (i-1)th training session and the second neural network model trained in the ith training session is the third neural network model trained in the ith training session.

[0128] It should be noted that after the second neural network model has undergone a new round of training, the computing device can determine the evaluation index of the second neural network model optimized during the current training and the evaluation index of the second neural network model optimized during the previous training by using verification samples. By comparing the evaluation indexes of the two second neural network models, the device can determine the second neural network model with the better evaluation index as the third neural network model for the current training.

[0129] S505. The computing device determines whether the current number of training rounds has reached the preset number of training rounds.

[0130] If the current number of training rounds has not reached the preset number of training rounds (i.e., the current number of training rounds is less than the preset number of training rounds), the computing device executes S506.

[0131] S506, The computing device determines the coordinate information of the target prediction bounding box of the defect target in each defect sample during the i-th training, and each defect sample as multiple training samples for the i+1-th training, and determines the third neural network model of the i-th training as the first neural network model for the i+1-th training, and executes S501-S505 in sequence.

[0132] It should be noted that if the current training rounds have not reached the preset number of training rounds, the computing device needs to control the neural network model to start a new round of training.

[0133] If the current training rounds reach the preset training rounds (i.e., the current training rounds are equal to the preset training rounds), the computing device executes S507.

[0134] S507. The computing device determines that the third neural network model obtained from the last training is the defect target detection model.

[0135] Optionally, after the computing device determines the defect target detection model, the computing device can also acquire test samples and input the test samples into the defect target detection model to determine the accuracy of the defect target detection model.

[0136] For example, if the defect target detection model based on the CIOU loss function and the defect target detection model of this application are tested with the same test samples, the accuracy of the defect target detection model based on the CIOU loss function (e.g., 0.91) is lower than the accuracy of the defect target detection model of this application (e.g., 0.92); the recall of the defect target detection model based on the CIOU loss function (e.g., 0.87) is lower than the recall of the defect target detection model of this application (e.g., 0.89).

[0137] The above technical solution brings at least the following beneficial effects: The model training method provided in this application requires the computing device to train multiple times until the current training round reaches the preset training round before the defect target detection model is determined. This makes the final determined defect target detection model more stable and has better performance.

[0138] In an optional embodiment, such as in S504, the computing device determines the evaluation metrics of the first neural network model trained in the i-th training iteration and the evaluation metrics of the second neural network model trained in the i-th training iteration. Figure 5 Based on the illustrated method embodiments, this embodiment provides a possible implementation, such as... Figure 6 As shown, the specific implementation process of the computing device determining the evaluation index of the first neural network model and the evaluation index of the second neural network model may include the following steps S601 to S603.

[0139] S601, The computing device acquires the coordinate information of multiple verification samples and the actual bounding box of each verification sample.

[0140] One of the verification samples is also a sampled image of a defect panel.

[0141] It should be noted that the coordinates of the actual bounding box of each validation sample in the multiple validation samples can be determined by referring to the above. Figure 4 The method shown is for understanding purposes and will not be elaborated further here.

[0142] S602. The computing device inputs multiple verification samples into the fourth neural network model to determine the coordinate information of the target prediction bounding boxes of the multiple verification samples.

[0143] The fourth neural network model is either the (i-1)th trained second neural network model or the i-th trained second neural network model.

[0144] It should be noted that the coordinate information of the target predicted bounding box of the validation sample can be understood by referring to the coordinate information of the target predicted bounding box of the training sample mentioned above, and will not be repeated here.

[0145] S603. The computing device determines the evaluation index of the fourth neural network model based on the coordinate information of the actual bounding boxes of multiple validation samples and the coordinate information of the target predicted bounding boxes of multiple validation samples.

[0146] As a feasible implementation, the specific implementation process of S603 is as follows: The computing device can determine the accuracy of a verification sample based on the degree of difference between the coordinate information of the actual bounding box of the verification sample and the coordinate information of the predicted bounding box of the target of the verification sample. The computing device determines the evaluation index of the fourth neural network model based on the accuracy of each verification sample among the multiple verification samples.

[0147] The above technical solution brings at least the following beneficial effects: The model training method provided in this application uses a computing device to determine the evaluation index of the second neural network model trained in the (i-1)th training stage and the evaluation index of the second neural network model trained in the ith stage through multiple verification samples. This ensures that the performance of the third neural network model determined subsequently is that of the neural network model with better performance in each training session, thereby improving the accuracy of the finally determined defect target detection model.

[0148] To address the problems existing in the prior art, this application proposes a defect target detection method, applicable to the above-mentioned... Figure 2 , Figures 4-6 The described defect detection model can improve the detection accuracy of smaller targets in LCD panels. For example... Figure 7As shown, the method includes:

[0149] S701, The computing device acquires the image of the target panel.

[0150] It should be noted that the specific implementation process of how the computing device acquires the target panel image can be understood by referring to the specific implementation process of S401 mentioned above, and will not be repeated here.

[0151] S702. The computing device inputs the acquired image into the defect target detection model and determines the coordinate information of the bounding box of the defect target in the acquired image.

[0152] In an alternative scenario, the computing device combines the defect target detection model file (e.g., best.pt) with the YOLOv5 model, and installs the resulting neural network model with the defect target detection model and the YOLOv5 model into the computing device of the factory that actually produces the target panel.

[0153] The above technical solution brings at least the following beneficial effects: The defect target detection method provided in this application allows the computing device to determine the coordinate information of the bounding box of the defect target in the acquisition image through a defect target detection model constructed and optimized based on a preset loss function. The preset loss function of this application relies on the coordinate information of the actual bounding box and predicted bounding box of the defect target, as well as the actual area and multiple predicted areas of the defect target. It does not overly depend on the aspect ratio of the bounding box of the defect target. In this way, even if some smaller targets do not have an aspect ratio, they can still be calculated and will not be lost. Therefore, compared with the CIOU loss function of the prior art, the preset loss function of this application is more suitable for the detection of smaller defect targets, improving the detection accuracy of smaller targets in the panel, without affecting the detection of larger defect targets.

[0154] It is understood that the above-described model training method can be implemented by a model training device. To achieve the above functions, the model training device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments disclosed in this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments disclosed in this application.

[0155] The embodiments disclosed in this application can divide the model training device generated by the above method examples into functional modules. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the embodiments disclosed in this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0156] Figure 8 This is a schematic diagram of a model training device provided in an embodiment of the present invention. Figure 8 As shown, the model training device 80 can be used to perform... Figure 2 , Figures 4-6 The model training method shown is illustrated. The model training apparatus 80 includes a processing unit 801. Optionally, the model training apparatus 80 further includes a communication unit 802.

[0157] Processing unit 801 is used for: determining multiple defect samples and a preset loss function; a defect sample is a captured image of a defect panel; the parameters of the preset loss function include: the coordinate information of the actual bounding box of the preset defect target, the coordinate information of any predicted bounding box of the preset defect target, the actual area of ​​the preset defect target, and multiple predicted areas of the preset defect target; the actual area is determined based on the coordinate information of the actual bounding box; the multiple predicted areas are determined based on the coordinate information of the multiple predicted bounding boxes of the preset defect target; the preset defect target is a defect target in any one of the multiple defect samples; constructing a first neural network model based on the preset loss function; inputting multiple defect samples into the first neural network model for training to determine a defect target detection model; the defect target detection model is used to determine the coordinate information of the bounding box of the defect target in the captured image of the target panel.

[0158] In one possible implementation, the pre-defined loss function satisfies the following formula:

[0159]

[0160] Where PDA is the loss function value; L is the Euclidean distance between the center point of the actual bounding box of the preset defect target and the center point of any predicted bounding box of the preset defect target; S is the sum of the actual area and multiple predicted areas;

[0161] L satisfies the following formula:

[0162]

[0163] Among them, (A) x1 A y1 ) and (Ax2 A y2 (B) represents the two endpoints of any diagonal line within the actual bounding box; x1 B y1 ) and (B x2 B y2 () represents the two endpoints of any diagonal line in any predicted bounding box;

[0164] S satisfies the following formula:

[0165]

[0166] Where W is the width of any predicted bounding box; h is the length of any predicted bounding box; and R is the radius of any predicted bounding box.

[0167] In one possible implementation, the processing unit 801 is further configured to: acquire a sampled image of a defect panel; the size of the sampled image is greater than or equal to a preset size; and determine the coordinate information of the actual bounding box of the defect target in the sampled image of the defect panel using a target tool.

[0168] In one possible implementation, the processing unit 801 is further configured to: Step 1, acquire multiple training samples for the i-th training; when i is 1, one training sample is one defect sample; when i is greater than 1, one training sample includes: one defect sample, and the coordinate information of the target predicted bounding box of the defect target in one defect sample during the (i-1)-th training; the target predicted bounding box is the predicted bounding box of the multiple predicted bounding boxes whose loss function value is greater than or equal to a preset threshold; i is a positive integer; Step 2, perform the following operation on each training sample among the multiple training samples to obtain the defect target in each defect sample during the i-th training. The system obtains the coordinate information of the predicted bounding boxes and the target value of the defect target in each defect sample during the i-th training iteration; it inputs a training sample into the first neural network model during the i-th training iteration to obtain the coordinate information of multiple first predicted bounding boxes; the coordinate information of the multiple first predicted bounding boxes is the coordinate information of multiple predicted bounding boxes of the defect target in a defect sample during the i-th training iteration; it determines the loss function value of each first predicted bounding box in the multiple first predicted bounding boxes according to a preset loss function; it determines the coordinate information of the first predicted bounding boxes in the multiple first predicted bounding boxes whose loss function value is greater than or equal to a preset threshold as the coordinate information of the defect target in a defect sample during the i-th training iteration. Step 1: Calculate the coordinates of the target prediction bounding box and determine the loss function value of the target prediction bounding box of the defective target in a defective sample during the i-th training iteration. Step 2: Optimize the first neural network model trained in the i-th training iteration based on the target value of the defective target in each defective sample during the i-th training iteration, thus determining the second neural network model trained in the i-th training iteration. Step 3: Determine the evaluation metrics for the second neural network model trained in the (i-1)-th and i-th training iterations, and determine the evaluation metrics for the second neural network model trained in the (i-1)-th and i-th training iterations. In the model, the neural network model with better evaluation metrics is the third neural network model trained in the i-th training session; Step 5: If the current training round number is less than the preset training round number, then the coordinate information of the target prediction bounding box of the defect target in each defect sample during the i-th training session, and each defect sample are determined as multiple training samples for the i+1-th training session, and the third neural network model trained in the i-th training session is determined as the first neural network model trained in the i+1-th training session. Steps 1, 2, 3, 4, and 5 are executed sequentially; If the current training round number is equal to the preset training round number, then the third neural network model obtained in the last training session is determined as the defect target detection model.

[0169] In one possible implementation, the processing unit 801 is further configured to: acquire multiple validation samples and the coordinate information of the actual bounding box of each validation sample; input the multiple validation samples into a fourth neural network model to determine the coordinate information of the target predicted bounding box of the multiple validation samples; the fourth neural network model is a first neural network model or a second neural network model; and determine the evaluation index of the fourth neural network model based on the coordinate information of the actual bounding box of the multiple validation samples and the coordinate information of the target predicted bounding box of the multiple validation samples.

[0170] It is understood that the aforementioned defect detection method can be implemented by a defect detection device. To achieve the above functions, the defect detection device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments disclosed in this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments disclosed in this application.

[0171] The embodiments disclosed in this application can divide the defect target detection device generated by the above method example into functional modules. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in the embodiments disclosed in this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0172] Figure 9 This is a schematic diagram of a defect target detection device provided in an embodiment of the present invention. Figure 9 As shown, the defect target detection device 90 can be used to perform... Figure 7 The defect target detection method shown is illustrated. The defect target detection device 90 includes a communication unit 901 and a processing unit 902.

[0173] The communication unit 901 is used to acquire the image of the target panel.

[0174] The processing unit 902 is used to input the acquired image into the defect target detection model and determine the coordinate information of the bounding box of the defect target in the acquired image.

[0175] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0176] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0177] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method, characterized in that, The method comprises: determining a plurality of defect samples and a preset loss function; one defect sample is a collection image of one defect panel; parameters of the preset loss function include: coordinate information of an actual bounding box of a preset defect target, coordinate information of any one predicted bounding box of the preset defect target, an actual area of the preset defect target, and a plurality of predicted areas of the preset defect target; the actual area is determined according to the coordinate information of the actual bounding box; the plurality of predicted areas are determined according to the coordinate information of a plurality of predicted bounding boxes of the preset defect target; the preset defect target is a defect target in any one of the plurality of defect samples; constructing a first neural network model based on the preset loss function; inputting the plurality of defect samples into the first neural network model for training to determine a defect target detection model; the defect target detection model is used to determine coordinate information of a bounding box of a defect target in a collection image of a target panel; the preset loss function satisfies the following formula: wherein, the PDA is a loss function value; the L is the Euclidean distance between the center point of the actual bounding box of the preset defect target and the center point of any one predicted bounding box of the preset defect target; the S is the sum of the actual area and the plurality of predicted areas; the L satisfies the following formula: wherein (A x1 , A y1 ) and (A x2 , A y2 ) are two endpoints of any diagonal line in the actual bounding box; (B x1 , B y1 ) and (B x2 , B y2 ) are two endpoints of any diagonal line in the any predicted bounding box; the S satisfies the following formula: S= wherein, the W is the width of the any one predicted bounding box; the h is the length of the any one predicted bounding box; and the R is the radius of the any one predicted bounding box.

2. The method of claim 1, wherein, The method further comprises: obtaining a collection image of the one defect panel; the size of the collection image is greater than or equal to a preset size; determining coordinate information of an actual bounding box of a defect target in the collection image of the one defect panel by a target tool.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: Step 1, obtaining a plurality of training samples of i-th training; in the case that the i is 1, one training sample is the one defect sample; in the case that the i is greater than 1, the one training sample includes: the one defect sample, and coordinate information of a target predicted bounding box of a defect target in the one defect sample in i-1-th training; the target predicted bounding box is a predicted bounding box in the plurality of predicted bounding boxes, and the loss function value is greater than or equal to a preset threshold; the i is a positive integer; Step 2, performing the following operation on each training sample in the plurality of training samples to obtain coordinate information of a target predicted bounding box of a defect target in each defect sample in the i-th training and a target value of the defect target in each defect sample in the i-th training; inputting the one training sample into the first neural network model of the i-th training to obtain coordinate information of a plurality of first predicted bounding boxes; the coordinate information of the plurality of first predicted bounding boxes is coordinate information of a plurality of predicted bounding boxes of a defect target in the one defect sample in the i-th training; determining a loss function value of each first predicted bounding box in the plurality of first predicted bounding boxes according to the preset loss function; determining that coordinate information of a first predicted bounding box in the plurality of first predicted bounding boxes, whose loss function value is greater than or equal to the preset threshold value, is coordinate information of a target predicted bounding box of the defect target in the one defect sample in the i-th training, and determining that a loss function value of the target predicted bounding box of the defect target in the one defect sample in the i-th training is a target value of the defect target in the one defect sample in the i-th training; Step 3, optimizing the first neural network model of the i-th training according to the target value of the defect target in each defect sample in the i-th training to determine a second neural network model of the i-th training; Step 4, determining an evaluation index of the second neural network model of the i-1-th training and an evaluation index of the second neural network model of the i-th training, and determining that a neural network model with a better evaluation index in the second neural network model of the i-1-th training and the second neural network model of the i-th training is a third neural network model of the i-th training; Step 5, if the current training round number is less than a preset training round number, then coordinate information of the target predicted bounding box of the defect target in each defect sample in the i-th training and each defect sample are determined as a plurality of training samples of the i+1-th training, the third neural network model of the i-th training is determined as a first neural network model of the i+1-th training, and the step 1, the step 2, the step 3, the step 4 and the step 5 are sequentially executed; if the current training round number is equal to the preset training round number, then a third neural network model obtained through the last training is determined as the defect target detection model.

4. The method of claim 3, wherein, The method further comprises: obtaining a plurality of verification samples and coordinate information of an actual bounding box of each verification sample in the plurality of verification samples; inputting the plurality of verification samples into a fourth neural network model to determine coordinate information of a target predicted bounding box of the plurality of verification samples; the fourth neural network model is the first neural network model or the second neural network model; determining an evaluation index of the fourth neural network model according to the coordinate information of the actual bounding box of the plurality of verification samples and the coordinate information of the target predicted bounding box of the plurality of verification samples.

5. A defect target detection method characterized by, application in the defect target detection model in any one of the above claims 1-4, comprising: obtaining a collection image of a target panel; inputting the collection image into the defect target detection model to determine coordinate information of a bounding box of a defect target in the collection image.

6. A model training apparatus characterized by comprising: comprising: a processing unit; the processing unit is used to: determine a plurality of defect samples and a preset loss function; The defect sample is a collection image of a defect panel; parameters of the preset loss function include: coordinate information of an actual bounding box of a preset defect target, coordinate information of any one predicted bounding box of the preset defect target, an actual area of the preset defect target, and a plurality of predicted areas of the preset defect target; the actual area is determined according to the coordinate information of the actual bounding box; the plurality of predicted areas are determined according to coordinate information of a plurality of predicted bounding boxes of the preset defect target; the preset defect target is a defect target in any one of the plurality of defect samples; a first neural network model is constructed based on the preset loss function; the plurality of defect samples are input into the first neural network model for training to determine a defect target detection model; the defect target detection model is used to determine coordinate information of a bounding box of a defect target in a collection image of a target panel; the preset loss function satisfies the following formula: wherein, the PDA is a loss function value; the L is an Euclidean distance between a center point of the actual bounding box of the preset defect target and a center point of any one predicted bounding box of the preset defect target; the S is a sum of the actual area and the plurality of predicted areas; the L satisfies the following formula: wherein (A x1 , A y1 ) and (A x2 , A y2 ) are two endpoints of any diagonal line in the actual bounding box; (B x1 , B y1 ) and (B x2 , B y2 ) are two endpoints of any diagonal line in the any predicted bounding box; the S satisfies the following formula: S= wherein, the W is a width of the any one predicted bounding box; the h is a length of the any one predicted bounding box; and the R is a radius of the any one predicted bounding box.

7. The apparatus of claim 6, wherein, The processing unit is further configured to: obtain a collection image of the one defect panel; the size of the collection image is greater than or equal to a preset size; determine, by a target tool, coordinate information of an actual bounding box of a defect target in the collection image of the one defect panel.

8. The apparatus of claim 6 or 7, wherein, The processing unit is further configured to: Step 1: obtain a plurality of training samples of i-th training; in a case where the i is 1, one training sample is the one defect sample; in a case where the i is greater than 1, the one training sample includes: the one defect sample, and coordinate information of a target predicted bounding box of a defect target in the one defect sample in i-1-th training; the target predicted bounding box is a predicted bounding box in the plurality of predicted bounding boxes, for which the loss function value is greater than or equal to a preset threshold; the i is a positive integer; Step 2: perform the following operation on each training sample in the plurality of training samples to obtain coordinate information of a target predicted bounding box of a defect target in each defect sample in the i-th training and a target value of the defect target in each defect sample in the i-th training. inputting the one training sample into the first neural network model of the i-th training to obtain coordinate information of a plurality of first predicted bounding boxes; the coordinate information of the plurality of first predicted bounding boxes is coordinate information of a plurality of predicted bounding boxes of a defect target in the one defect sample in the i-th training; determining a loss function value of each first predicted bounding box in the plurality of first predicted bounding boxes according to the preset loss function; determining that coordinate information of a first predicted bounding box in the plurality of first predicted bounding boxes, whose loss function value is greater than or equal to the preset threshold, is coordinate information of a target predicted bounding box of the defect target in the one defect sample in the i-th training, and determining that a loss function value of the target predicted bounding box of the defect target in the one defect sample in the i-th training is a target value of the defect target in the one defect sample in the i-th training; Step 3, optimizing the first neural network model of the i-th training according to the target value of the defect target in each defect sample in the i-th training to determine a second neural network model of the i-th training; Step 4, determining an evaluation index of the second neural network model of the i-1-th training and an evaluation index of the second neural network model of the i-th training, and determining that a neural network model with a better evaluation index in the second neural network model of the i-1-th training and the second neural network model of the i-th training is a third neural network model of the i-th training; Step 5, if the current training round number is less than the preset training round number, then the coordinate information of the target predicted bounding box of the defect target in each defect sample in the i-th training and each defect sample are determined as a plurality of training samples of the i+1-th training, the third neural network model of the i-th training is determined as a first neural network model of the i+1-th training, and the steps 1, 2, 3, 4 and 5 are sequentially executed; if the current training round number is equal to the preset training round number, then a third neural network model obtained through the last training is determined as the defect target detection model.

9. The apparatus of claim 8, wherein, The processing unit is further configured to: obtain a plurality of verification samples and coordinate information of actual bounding boxes of each verification sample in the plurality of verification samples; input the plurality of verification samples into a fourth neural network model to determine coordinate information of target predicted bounding boxes of the plurality of verification samples; the fourth neural network model is the first neural network model or the second neural network model; determine an evaluation index of the fourth neural network model according to the coordinate information of the actual bounding boxes of the plurality of verification samples and the coordinate information of the target predicted bounding boxes of the plurality of verification samples.

10. A defect target detection apparatus characterized by comprising: The model training device is applied to any one of the preceding claims 6-9, and comprises a communication unit and a processing unit; The communication unit is configured to obtain an acquisition image of a target panel; The processing unit is configured to input the acquisition image into a defect target detection model to determine coordinate information of a bounding box of a defect target in the acquisition image.

11. A model training apparatus, comprising: The model training device comprises: A processor and a communication interface; the communication interface and the processor are coupled, and the processor is configured to run computer programs or instructions to implement the model training method as claimed in any one of claims 1-4.

12. A defect target detection apparatus characterized by comprising: comprising: A processor and a communication interface; the communication interface and the processor are coupled, and the processor is configured to run computer programs or instructions to implement the defect target detection method as claimed in claim 5.

13. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium being characterized by, When a computer executes the instructions, the computer executes the model training method as claimed in any one of claims 1-4, or the defect target detection method as claimed in claim 5.

Images