Guardrail verification model for automated optical inspection models

Abstract

The present disclosure provides a guardrail verification model of an automatic optical detection model, and relates to the field of defect detection, which comprises a configuration input module, a configuration access module and a running protection module; the configuration input module receives a configuration object of the automatic optical detection model; the configuration access module performs legality verification on the configuration object before the configuration object enters an execution module of the automatic optical detection model, so as to confirm that the configuration object has executability and traceability; and the running protection module sets a guardrail protection mechanism based on line quality related parameters, beat related parameters, resource related parameters, safety related parameters, version related parameters and audit requirement related parameters of the automatic optical detection model, so as to perform running protection on the automatic optical detection model in the running process of the automatic optical detection model. The problem that the accuracy of the output result of the automatic optical detection model is affected due to the error configuration, expired configuration, mutual exclusion configuration and the like is improved.

Description

Technical Field

[0001] This disclosure relates to the field of defect detection technology, specifically to a guardrail verification model based on an automatic optical inspection model. Background Technology

[0002] In the manufacturing processes of semiconductor devices and display panels, including but not limited to semiconductor devices, automated optical inspection equipment is typically used to automatically detect manufacturing defects in these devices. However, the accuracy of the automated optical inspection model's output can be affected by misconfigurations, outdated configurations, or mutually exclusive configurations when performing defect detection tasks. Summary of the Invention

[0003] The main objective of this disclosure is to provide a guardrail verification model for an automatic optical inspection model, so as to improve the accuracy of the output results of the automatic optical inspection model affected by misconfiguration, outdated configuration, mutual exclusion configuration, etc.

[0004] To achieve the above objectives, the first aspect of this disclosure provides an automatic optical inspection model for guardrail verification, the guardrail verification model comprising: A configuration input module is used to receive a configuration object of the automatic optical detection model; the configuration object includes at least some of the expert agent recipe, expert agent routing, threshold-related parameters, expert agent version, and budget-related parameters of the automatic optical detection model. The configuration access module is used to perform a legality check on the configuration object before it enters the execution module of the automatic optical detection model, so as to confirm that the configuration object has executability and traceability; wherein, the legality check includes at least some of the following: summary check, required field check, value range check, dependency check, semantic consistency check, version compatibility check, and scope of application check of the configuration object; The operation protection module is used to set up a guardrail protection mechanism based on the production line quality-related parameters, cycle time-related parameters, resource-related parameters, safety-related parameters, version-related parameters, and audit requirement-related parameters of the production line where the automatic optical inspection model is located, so as to protect the automatic optical inspection model during its operation.

[0005] In some embodiments of this disclosure, the operation protection module includes: The pre-barrier module is used to intercept pre-set pre-barrier protection indicators before the execution module of the automatic optical detection model is executed; wherein, the pre-barrier protection indicators include: illegal configuration indicators, missing evidence configuration indicators, unapproved configuration indicators, and unauthorized configuration indicators; The resource fence module is used to constrain resource fence protection indicators during the execution of the automatic optical detection model; wherein, the resource fence protection indicators include: computing power resource indicators, memory resource indicators, P95 latency indicators, and unprocessed detection task queue indicators. The quality guardrail module is used to control the quality indicators of the execution results when the execution module of the automatic optical inspection model generates execution results; wherein, the quality indicators include: whether OOD occurs, confidence conflict, conflict between expert agents, and result drift; The output guardrail module is used to control the output result indicators of the automatic optical detection model; wherein, the output result indicators include: whether the output result is missing a version number, whether the evidence field is missing, whether it is mixed across versions, and whether the primary key is inconsistent; The guardrail module is modified to control the safety indicators of the automatic optical detection model; wherein, the safety indicators include whether gray-scale release has not been completed, whether dual signature has not been achieved, and whether playback verification has not been passed.

[0006] In some embodiments of this disclosure, the resource fence module is configured as follows: When it detects that the computing power resource index, the memory resource index, the P95 latency index, or the pending detection task queue index exceeds the respective set thresholds for these indexes, it automatically reduces the parallelism of the automatic optical detection model, reduces the number of expert agents added to the automatic optical detection model, switches the automatic optical detection model to a lightweight path, or enters a pre-set conservative path mode.

[0007] In some embodiments of this disclosure, the mass guardrail module is configured as follows: When the system detects that the execution result generated by the execution module of the automatic optical detection model has an Out-of-Depth (OOD) error, the confidence conflict exceeds the confidence conflict limit set for it, the expert agent conflict exceeds the agent conflict limit set for it, or the execution result exceeds the allowable drift range set for it, the automatic optical detection model is prohibited from directly outputting the execution result as the final conclusion. Instead, the automatic optical detection model is triggered to perform additional verification or manual review process.

[0008] In some embodiments of this disclosure, the guardrail modification module is configured to: When the system detects that the automatic optical inspection model has not completed grayscale release, has not achieved dual signature, or has not passed playback verification, the automatic optical inspection model is prohibited from entering the full production line where the automatic optical inspection model is to be run.

[0009] In some embodiments of this disclosure, the guardrail verification model further includes: A guardrail arrangement module is used to configure the guardrail module combination formula included in the operation protection module based on the current operating status of the automatic optical detection model.

[0010] In some embodiments of this disclosure, the guardrail modules included in the guardrail module combination recipe are coupled and linked with the expert agent routing in the configuration object, so that the expert agent routing adjusts the expert agent recipe according to the guardrail actions of the guardrail modules included in the guardrail module combination recipe.

[0011] In some embodiments of this disclosure, the expert agent routing adjusts the expert agent formula based on the guardrail actions of the guardrail modules included in the guardrail module combination formula, including: If the resource guardrail module detects congestion in the current queue of pending detection tasks, the expert agent routing will downgrade the heavy expert agent included in the expert agent recipe to a lightweight expert agent and a rule review agent, wherein the heavy expert agent and the lightweight expert agent are used to process the same detection task. If the quality guardrail module detects that the result drift exceeds the allowable drift range, the expert agent routing adds other expert agents set up for the same detection task to the expert agent recipe to perform a multi-expert agent mutual verification mechanism, or / and adds the expert agent recipe to the manual review pool.

[0012] In some embodiments of this disclosure, the prohibition conditions of the guardrail protection mechanism are set based on the risk level of the detection task currently being performed by the automated optical detection model; The higher the risk level of the detection task currently being performed by the automatic optical detection model, the more stringent the prohibition conditions. The lower the risk level of the detection task currently being performed by the automated optical inspection model, the more lenient the prohibition conditions.

[0013] In some embodiments of this disclosure, the configuration admission module performs a validity check on the configuration object, including: The configuration object is subjected to a feed validation to confirm whether the fields of the configuration object are complete, whether the field types are correct, whether there are any missing key primary keys, and whether there are any unrecognizable fields. The configuration object is subjected to range verification to confirm whether the threshold of the configuration object is out of bounds, whether the ROI size exceeds the input image boundary of the automatic optical detection model, whether the number of parallel expert agents exceeds the computing power resource limit of the device running the automatic optical detection model, and whether the magnification and scaling parameters of the configuration object are within the executable range. Dependency checks are performed on the configuration objects to confirm whether the expert agent version depends on a specific feature extractor version, whether the expert agent routing strategy depends on the OOD expert agent, and whether the output format of the automatic optical detection model depends on a specified table structure. Perform semantic and compatibility checks on the configuration objects to confirm whether there are mutually exclusive items in the same configuration object; The configuration object is validated for validity and permissions to confirm whether it is valid, approved, and allowed to be applied to the current production line, machine, and process stage.

[0014] The second aspect of this disclosure provides an automated optical inspection device, which includes a guardrail verification model of any of the automated optical inspection models provided in the first aspect of this disclosure.

[0015] The guardrail verification model of the automatic optical inspection model provided in this embodiment of the disclosure sets up a configuration access module and an operation protection module. The configuration access module verifies the legality of the configuration object before it enters the execution module of the automatic optical inspection model to confirm its executability and traceability. The operation protection module sets up a guardrail protection mechanism based on production line quality-related parameters, cycle time-related parameters, resource-related parameters, safety-related parameters, version-related parameters, and audit requirement-related parameters. This mechanism protects the automatic optical inspection model during its operation, thereby identifying misconfigurations, expired configurations, and mutually exclusive configurations before outputting results, thus preventing these situations from affecting the accuracy of the automatic optical inspection model's output and improving the accuracy of the output results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1A schematic block diagram and a flowchart illustrating an automatic optical inspection model for guardrail verification provided in an embodiment of this disclosure. Figure 2 A schematic block diagram of a configuration admission module provided in an embodiment of this disclosure; Figure 3 A schematic diagram illustrating the workflow of a configuration admission module provided in an embodiment of this disclosure; Figure 4 A schematic diagram illustrating the workflow of a configuration admission module provided in another embodiment of this disclosure; Figure 5 A schematic diagram illustrating the workflow of the routing coupling between the guardrail module and the expert intelligent agent provided in an embodiment of this disclosure; Figure 6 A schematic diagram illustrating the process of setting the prohibition conditions in a guardrail protection mechanism provided in an embodiment of this disclosure; Figure 7 A schematic diagram of a legality verification process provided in an embodiment of this disclosure; Figure 8 A schematic diagram of the workflow of a guardrail verification model provided in another embodiment of this disclosure for an automated optical inspection model; Figure 9 A schematic diagram of the core items for legality verification provided in an embodiment of this disclosure; Figure 10 This is a schematic diagram of the core components of a guardrail provided in an embodiment of this disclosure. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] In this disclosure, the terms “upper,” “lower,” “left,” “right,” “front,” “rear,” “top,” “bottom,” “inner,” “outer,” and “middle,” etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. These terms are primarily for the purpose of better describing this disclosure and its embodiments, and are not intended to limit the indicated devices, elements, or components to having a specific orientation, or to be constructed and operated in a specific orientation.

[0021] Furthermore, in addition to indicating direction or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in certain circumstances to indicate a dependency or connection. Those skilled in the art can understand the specific meaning of these terms in this disclosure according to the specific circumstances.

[0022] Furthermore, the terms "set up," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection via an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this disclosure according to the specific circumstances.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Example 1 To address at least some of the aforementioned problems, embodiments of this disclosure provide an automatic optical inspection model for guardrail verification, with reference to... Figure 1 and Figure 8 The guardrail verification model mainly includes the following modules: The configuration input module receives a configuration object for the automated optical detection model. This configuration object includes at least some of the following: expert agent recipe, expert agent routing, threshold-related parameters, expert agent version, and budget-related parameters. Specifically, it receives a configuration object that may include a Recipe, expert routing strategy, threshold table, budget-related parameters, canary release strategy, rollback strategy, model version reference, and evidence field definitions.

[0025] The configuration admission module performs a validity check on the configuration object before it enters the execution module of the automated optical inspection model. This check ensures the configuration object is executable and traceable. The validity check includes at least some of the following: configuration object summary verification, required field verification, value range verification, dependency verification, semantic consistency verification, version compatibility verification, and scope of application verification. Specifically, the configuration admission module verifies the validity of the configuration object by checking its summary, required fields, value range, dependencies, semantic consistency, version compatibility, and scope of application before it enters the execution module of the automated optical inspection model. This ensures the configuration is executable and traceable.

[0026] The operation protection module is used to set up a guardrail protection mechanism based on parameters related to production line quality, cycle time, resources, safety, version, and audit requirements of the automated optical inspection model. This mechanism protects the automated optical inspection model during its operation. For example, the operation protection module is configured as an industrial-grade guardrail, consisting of a set of rigid constraints and conservative strategies (guardrail protection mechanism) around production line quality, cycle time, resources, safety, version, and audit requirements, used to limit the behavior of the automated optical inspection model system in risky scenarios.

[0027] In the above solution, a configuration access module and a runtime protection module are set up. The configuration access module performs a legality check on the configuration object before it enters the execution module of the automated optical inspection model to confirm its executability and traceability. The runtime protection module sets up a guardrail protection mechanism based on the production line quality-related parameters, cycle time-related parameters, resource-related parameters, safety-related parameters, version-related parameters, and audit requirement-related parameters of the automated optical inspection model. This mechanism protects the automated optical inspection model during its operation, thereby identifying incorrect, expired, and mutually exclusive configurations before outputting results, thus preventing these situations from affecting the accuracy of the automated optical inspection model's output and improving the accuracy of the output results. It should be noted that the legality check solves the problem of "whether the configuration can be executed," while the guardrail solves the problem of "even if it can be executed, is it allowed to execute in the current running state?" Legality checks mostly occur before release or during loading; the guardrail runs throughout the entire runtime and is a dynamic constraint.

[0028] The following is in conjunction with the appendix Figures 1 to 10 The automatic optical inspection model of this disclosure provides a detailed description of the guardrail verification model.

[0029] The device being inspected can be, but is not limited to, semiconductor devices such as display panels and wafers. The aforementioned inspection tasks can be, but are not limited to, any defect inspection task that can be visually inspected, such as, conductive line defects, contact hole defects, void defects, flatness defects, groove defects, etc.

[0030] There are several ways to configure the operation protection module. Some of these methods are illustrated below.

[0031] For example, refer to Figure 2 , Figure 3 and Figure 10 The operation protection module may include: a front guardrail module, which is used to intercept pre-set front guardrail protection indicators before the execution module of the automatic optical detection model is executed; the front guardrail protection indicators include: illegal configuration indicators, missing evidence configuration indicators, unapproved configuration indicators, and over-authorization configuration indicators.

[0032] For example, refer to Figure 2 , Figure 3 and Figure 10 The operation protection module may include a resource fence module, which is used to constrain resource fence protection indicators during the execution of the automatic optical inspection model. The resource fence protection indicators include: computing power resource indicators, memory resource indicators, P95 latency indicators, and unprocessed inspection task queue indicators.

[0033] For example, refer to Figure 2 , Figure 3 , Figure 4 and Figure 10 The resource guardrail module can be configured to automatically reduce the parallelism of the automatic optical detection model, reduce the number of expert agents added to the automatic optical detection model, switch the automatic optical detection model to a lightweight path, or enter a pre-set conservative path mode when it detects that the computing power resource indicators, memory resource indicators, P95 latency indicators, or pending detection task queue indicators exceed the respective set thresholds for these indicators.

[0034] For example, refer to Figure 2 , Figure 3 and Figure 10 The operation protection module may include a quality guardrail module, which is used to control the quality indicators of the execution results when the execution module of the automatic optical inspection model generates execution results; among which, the quality indicators include: whether OOD (sample phenomenon that does not appear in the training samples of the automatic optical inspection model) occurs, confidence conflict, conflict between expert agents, and result drift.

[0035] For example, refer to Figure 2 , Figure 3 , Figure 4 and Figure 10 The quality guardrail module can be configured to prevent the automatic optical inspection model from directly outputting the execution result as the final conclusion when it detects that the execution result generated by the execution module of the automatic optical inspection model has an OOD, the confidence conflict exceeds the confidence conflict limit set for it, the expert agent conflict exceeds the agent conflict limit set for it, or the execution result exceeds the allowable drift range set for it. Instead, it will trigger the automatic optical inspection model to perform additional verification or manual review process.

[0036] For example, refer to Figure 2 , Figure 3 and Figure 10 The operation protection module may include an output guardrail module, which is used to control the output result indicators of the automatic optical detection model; among which, the output result indicators include: whether the output result is missing a version number, whether it is missing an evidence field, whether it is mixed across versions, and whether the primary key is inconsistent.

[0037] For example, refer to Figure 2 , Figure 3 and Figure 10 The operation protection module may include: a change guardrail module, which is used to control the safety indicators of the automatic optical detection model; among which, the safety indicators include whether the gray-scale release has not been completed, whether the dual signature has not been achieved, and whether the playback verification has not been passed.

[0038] For example, refer to Figure 2 , Figure 3 , Figure 4 and Figure 10 The guardrail module can be configured to prevent the automatic optical inspection model from entering the full production line where it is about to run when it detects that the automatic optical inspection model has not completed grayscale release, has not achieved dual signature, or has not passed playback verification.

[0039] For example, refer to Figure 2 and Figure 10 The guardrail verification model may also include: a guardrail arrangement module, which is used to configure the guardrail module combination formula contained in the running protection module based on the current running status of the automatic optical detection model.

[0040] It should be noted that the aforementioned guardrail module is typically not implemented at a single point, but rather exists in three locations: before configuration, during execution, and after output. The guardrail before configuration is responsible for access control, the guardrail during execution is responsible for preventing incidents, and the guardrail after output is responsible for preventing erroneous conclusions from entering downstream systems.

[0041] For ease of understanding, the following table presents the key points and detailed descriptions of each guardrail module. Please refer to Table 1 below for specific details.

[0042] Table 1 - Comparison of Key Considerations and Detailed Explanations for Each Guardrail Module

[0043] For example, refer to 1. Figure 2 and Figure 5 The guardrail modules included in the guardrail module combination recipe are coupled and linked with the expert agent routing in the configuration object, so that the expert agent routing adjusts the expert agent recipe according to the guardrail actions of the guardrail modules included in the guardrail module combination recipe.

[0044] For example, refer to 1. Figure 2 and Figure 5 The expert agent routing adjusts the expert agent recipe based on the guardrail actions of the guardrail modules included in the guardrail module combination recipe. This may include: if the resource guardrail module detects congestion in the current queue of pending detection tasks, the expert agent routing will downgrade the heavy expert agent included in the expert agent recipe to a lightweight expert agent and a rule review agent. The heavy expert agent and the lightweight expert agent are used to handle the same detection task.

[0045] For example, refer to 1. Figure 2 and Figure 5 The expert agent routing adjusts the expert agent recipe based on the guardrail actions of the guardrail modules included in the guardrail module combination recipe. This may include: if the quality guardrail module detects that the result drift exceeds the allowable drift range, the expert agent routing adds other expert agents set for the same detection task to the expert agent recipe to perform a multi-expert agent mutual verification mechanism, or / and adds the expert agent recipe to the manual review pool.

[0046] For example, the guardrail module can work in conjunction with MoE / Router (expert agent routing). For instance, if the Router (expert agent routing) originally planned to execute three expert agents, but the resource guardrail module detects congestion in the current queue of pending detection tasks, it can downgrade to one lightweight expert agent + one rule-based expert agent for review. If the quality guardrail module detects excessive drift, it can reverse the process to "multi-expert mutual verification + human pool" to reduce drift and improve the accuracy of detection results.

[0047] For example, refer to Figure 6The prohibition conditions of the guardrail protection mechanism are set based on the risk level of the detection task currently being performed by the automatic optical inspection model. The higher the risk level of the current detection task, the stricter the prohibition conditions; conversely, the lower the risk level, the more lenient the prohibition conditions. For example, for high-risk processes, each guardrail module can be set with stricter prohibition conditions.

[0048] For example, in the critical defect detection scenario, since the critical defect detection scenario has a high risk level and belongs to a high-risk process, it can follow the prohibition condition that any "low confidence + rule conflict + OOD alarm" will not be allowed to proceed automatically. That is, the prohibition condition is very strict at this time. If any guardrail module fails the guardrail verification, it will not be allowed to proceed.

[0049] For example, refer to Figure 7 , Figure 8 and Figure 9 The configuration admission module performs validity checks on configuration objects, which may include: performing a schema (fab) check on the configuration object to confirm whether the fields are complete, whether the field types are correct, whether any key primary keys are missing, and whether any unrecognizable fields exist. In other words, it performs a schema (fab) check on the configuration object to confirm whether the configuration fields are complete, whether the field types are correct, whether any key primary keys are missing, and whether any unrecognizable fields exist.

[0050] For example, refer to Figure 7 , Figure 8 and Figure 9 The configuration admission module performs legality verification on the configuration object, which may include: range verification of the configuration object to confirm whether the threshold of the configuration object is out of bounds, whether the ROI size exceeds the input image boundary of the automatic optical inspection model, whether the number of expert agents in parallel exceeds the computing power resource limit of the device running the automatic optical inspection model, and whether the magnification and scaling parameters of the configuration object are within the executable range. In other words, range verification of the configuration object confirms whether the threshold is out of bounds, whether the ROI size exceeds the image boundary, whether the number of parallel agents exceeds the resource limit, and whether the magnification / scaling parameters are within the executable range.

[0051] For example, refer to Figure 7 , Figure 8 and Figure 9The configuration admission module performs validity checks on configuration objects, which may include: performing dependency checks on the configuration objects to confirm whether the expert agent version depends on a specific feature extractor version, whether the expert agent routing strategy depends on an OOD expert agent, and whether the output format of the automatic optical detection model depends on a specified table structure. In other words, dependency checks are performed, such as whether a model version depends on a specific feature extractor version, whether a routing strategy depends on the existence of an OOD expert, and whether an output format depends on a specified table structure.

[0052] For example, refer to Figure 7 , Figure 8 and Figure 9 The configuration admission module performs validity checks on configuration objects, which may include semantic and compatibility checks to confirm whether there are mutually exclusive items within the same configuration object. Specifically, semantic / compatibility checks determine whether there are mutually exclusive items within the same configuration, such as enabling "rule priority forced fallback" while simultaneously disabling rule output; or enabling a segmentation model but the tag set does not match the current product family.

[0053] For example, refer to Figure 7 , Figure 8 and Figure 9 The configuration access module performs legality verification on configuration objects, which may include: verifying the validity and permissions of the configuration objects to confirm whether the configuration objects are valid, approved, and permitted for application to the current production line, machine, and process stage. In other words, it performs validity and permission verification to confirm whether the configuration is valid, approved, and permitted for application to the current production line, machine, and customer stage.

[0054] For example, the guardrail verification model may also include: a verification report generation module, which can output pass / fail status, error items, warning items, suggested repair items and configuration fingerprints as prerequisites for subsequent release and auditing.

[0055] For example, the guardrail verification model may also include a release and audit module, which is used to determine the gray-scale release of the adjusted automatic optical inspection model version based on the verification results of the guardrail module at runtime.

[0056] It should be noted that the guardrail module shown in the above embodiments of this disclosure is an industrial-grade "guardrail," which focuses on the operational boundaries and risk control of the automatic optical inspection model system, answering the question "which behaviors are not allowed to occur, and how to block them if they occur." The "legality verification" focuses on whether the configuration object itself is correct, complete, compatible, and publishable, answering the question "can the configuration enter the execution system?" The former (guardrail verification) leans towards operational governance, while the latter (legality verification) leans towards configuration access control; together, they constitute an industrial-grade security mechanism.

[0057] It should be noted that the above-mentioned functional modules can be configured through hardware such as memory and processor, as well as software code running on them.

[0058] In summary, the automatic optical inspection model for guardrail verification provided in this disclosure offers the following advantages over related technologies: it prevents incorrect, outdated, and mutually exclusive configurations from entering the production line. In cases of abnormal cycle time, model anomalies, or distribution anomalies, it prioritizes system security and result reliability. It provides verifiable, traceable, and rollback-compatible engineering evidence for customer acceptance.

[0059] It should be noted that, in addition to the structure described above, the guardrail verification model of the automatic optical inspection model in this embodiment may also include the related structures described in part of the embodiment, all of which are within the protection scope of the guardrail verification model of the automatic optical inspection model provided in this embodiment.

[0060] Example 2 This disclosure provides an automatic optical inspection device, which includes a guardrail verification model based on any of the automatic optical inspection models provided in Embodiment 1 of this disclosure.

[0061] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A guardrail verification model based on an automatic optical inspection model, characterized in that, include: A configuration input module is used to receive a configuration object of the automatic optical detection model; the configuration object includes at least some of the expert agent recipe, expert agent routing, threshold-related parameters, expert agent version, and budget-related parameters of the automatic optical detection model. The configuration access module is used to perform a legality check on the configuration object before it enters the execution module of the automatic optical detection model, so as to confirm that the configuration object has executability and traceability; wherein, the legality check includes at least some of the following: summary check, required field check, value range check, dependency check, semantic consistency check, version compatibility check, and scope of application check of the configuration object; The operation protection module is used to set up a guardrail protection mechanism based on the production line quality-related parameters, cycle time-related parameters, resource-related parameters, safety-related parameters, version-related parameters, and audit requirement-related parameters of the production line where the automatic optical inspection model is located, so as to protect the automatic optical inspection model during its operation.

2. The guardrail verification model as described in claim 1, characterized in that, The operation protection module includes: The pre-barrier module is used to intercept pre-set pre-barrier protection indicators before the execution module of the automatic optical detection model is executed; wherein, the pre-barrier protection indicators include: illegal configuration indicators, missing evidence configuration indicators, unapproved configuration indicators, and unauthorized configuration indicators; The resource fence module is used to constrain resource fence protection indicators during the execution of the automatic optical detection model; wherein, the resource fence protection indicators include: computing power resource indicators, memory resource indicators, P95 latency indicators, and unprocessed detection task queue indicators. The quality guardrail module is used to control the quality indicators of the execution results when the execution module of the automatic optical inspection model generates execution results; wherein, the quality indicators include: whether OOD occurs, confidence conflict, conflict between expert agents, and result drift; The output guardrail module is used to control the output result indicators of the automatic optical detection model; wherein, the output result indicators include: whether the output result is missing a version number, whether the evidence field is missing, whether it is mixed across versions, and whether the primary key is inconsistent; The guardrail module is modified to control the safety indicators of the automatic optical detection model; wherein, the safety indicators include whether gray-scale release has not been completed, whether dual signature has not been achieved, and whether playback verification has not been passed.

3. The guardrail verification model as described in claim 2, characterized in that, The resource guardrail module is configured as follows: When it detects that the computing power resource index, the memory resource index, the P95 latency index, or the pending detection task queue index exceeds the respective set thresholds for these indexes, it automatically reduces the parallelism of the automatic optical detection model, reduces the number of expert agents added to the automatic optical detection model, switches the automatic optical detection model to a lightweight path, or enters a pre-set conservative path mode.

4. The guardrail verification model as described in claim 2, characterized in that, The quality guardrail module is configured as follows: When the system detects that the execution result generated by the execution module of the automatic optical detection model has an Out-of-Depth (OOD) error, the confidence conflict exceeds the confidence conflict limit set for it, the expert agent conflict exceeds the agent conflict limit set for it, or the execution result exceeds the allowable drift range set for it, the automatic optical detection model is prohibited from directly outputting the execution result as the final conclusion. Instead, the automatic optical detection model is triggered to perform additional verification or manual review process.

5. The guardrail verification model as described in claim 2, characterized in that, The guardrail modification module is configured as follows: When the system detects that the automatic optical inspection model has not completed grayscale release, has not achieved dual signature, or has not passed playback verification, the automatic optical inspection model is prohibited from entering the full production line where the automatic optical inspection model is to be run.

6. The guardrail verification model as described in any one of claims 2 to 5, characterized in that, Also includes: A guardrail arrangement module is used to configure the guardrail module combination formula included in the operation protection module based on the current operating status of the automatic optical detection model.

7. The guardrail verification model as described in claim 6, characterized in that, The guardrail modules included in the guardrail module combination recipe are coupled and linked with the expert agent routing in the configuration object, so that the expert agent routing adjusts the expert agent recipe according to the guardrail actions of the guardrail modules included in the guardrail module combination recipe.

8. The guardrail verification model as described in claim 7, characterized in that, The expert agent routing adjusts the expert agent formula based on the guardrail actions of the guardrail modules included in the guardrail module combination formula, including: If the resource guardrail module detects congestion in the current queue of pending detection tasks, the expert agent routing will downgrade the heavy expert agent included in the expert agent recipe to a lightweight expert agent and a rule review agent, wherein the heavy expert agent and the lightweight expert agent are used to process the same detection task. If the quality guardrail module detects that the result drift exceeds the allowable drift range, the expert agent routing adds other expert agents set up for the same detection task to the expert agent recipe to perform a multi-expert agent mutual verification mechanism, or / and adds the expert agent recipe to the manual review pool.

9. The guardrail verification model as described in claim 2, characterized in that, The prohibition conditions of the guardrail protection mechanism are set based on the risk level of the detection task currently being performed by the automatic optical detection model; The higher the risk level of the detection task currently being performed by the automatic optical detection model, the more stringent the prohibition conditions. The lower the risk level of the detection task currently being performed by the automated optical inspection model, the more lenient the prohibition conditions.

10. The guardrail verification model as described in claim 1, characterized in that, The configuration access module performs a validity check on the configuration object, including: The configuration object is subjected to a feed validation to confirm whether the fields of the configuration object are complete, whether the field types are correct, whether there are any missing key primary keys, and whether there are any unrecognizable fields. The configuration object is subjected to range verification to confirm whether the threshold of the configuration object is out of bounds, whether the ROI size exceeds the input image boundary of the automatic optical detection model, whether the number of parallel expert agents exceeds the computing power resource limit of the device running the automatic optical detection model, and whether the magnification and scaling parameters of the configuration object are within the executable range. Dependency checks are performed on the configuration objects to confirm whether the expert agent version depends on a specific feature extractor version, whether the expert agent routing strategy depends on the OOD expert agent, and whether the output format of the automatic optical detection model depends on a specified table structure. Perform semantic and compatibility checks on the configuration objects to confirm whether there are mutually exclusive items in the same configuration object; The configuration object is validated for validity and permissions to confirm whether it is valid, approved, and allowed to be applied to the current production line, machine, and process stage.

Images