Automated optical inspection system

Abstract

This application discloses an automatic optical inspection system. The system includes: a data access module, an agent orchestration module, a large language model decision module, a path execution module, a result fusion module, and an early warning module. The data access module acquires image data of the object to be inspected. The agent orchestration module generates a set of candidate detection paths based on the image data. The large language model decision module determines the main detection path and backup detection paths from the set of candidate detection paths. The path execution module executes the main detection path and backup detection paths to obtain defect detection results. The result fusion module verifies and fuses the defect detection results generated by multiple detection paths and outputs the final detection result. The early warning module performs root cause attribution based on the final detection result and data in a historical knowledge base and outputs a first early warning processing suggestion, or outputs a second early warning processing suggestion based on monitoring indicators in the current inspection scenario.

Description

Technical Field

[0001] This application relates to the field of industrial automated optical inspection technology, and more specifically, to an automated optical inspection system. Background Technology

[0002] Industrial automated optical inspection (AOI) is widely used in mass production inspection scenarios such as panels, wafers, and precision components. The inspection task needs to adapt to changes in different defect types, production line cycle times, equipment status, and material batches. Under the premise of ensuring detection rate, measurement accuracy, and processing delay, it can achieve stable, interpretable, and traceable automated inspection.

[0003] Existing AOI inspection systems generally adopt a fixed inspection link mode, which means that after image acquisition, a single preset image processing flow and defect model inference are directly executed. The detection path, algorithm combination and model calling method all rely on manual pre-configuration and cannot be dynamically adjusted according to task requirements and changes in working conditions.

[0004] However, a single detection process is difficult to adapt to diverse defects and complex working conditions, and is prone to problems such as unstable detection and large measurement errors. The combination of multiple models and multiple preprocessing steps relies on manual experience for selection, and the path optimization is costly and inefficient. The detection process lacks an intelligent path decision-making mechanism. At the same time, the results of multiple paths are difficult to unify and integrate, the detection basis is uninterpretable, anomalies cannot be predicted, and experience cannot be accumulated, which cannot meet the actual needs of modern industry for high-precision, high-robustness, and auditable detection. Summary of the Invention

[0005] The main objective of this application is to provide an automated optical inspection system to solve the problems of fixed AOI inspection links, reliance on manual path selection, difficulty in fusing results from multiple models, lack of interpretability and root cause warning. The system can achieve adaptive multi-path decision-making, reliable execution of primary and backup paths, unified fusion of multiple results, traceability auditing and intelligent warning, thereby significantly improving inspection accuracy, stability and deployment efficiency.

[0006] To achieve the above objectives, this application proposes an automatic optical inspection system, comprising: a data access module, an intelligent agent orchestration module, a large language model decision module, a path execution module, a result fusion module, and an early warning module; the data access module is used to acquire image data of the object to be inspected; the intelligent agent orchestration module is used to generate a candidate detection path set based on the image data; the large language model decision module is used to determine a primary detection path and a backup detection path from the candidate detection path set; the path execution module is used to execute the primary detection path and the backup detection path to obtain defect detection results; the result fusion module is used to verify and fuse the defect detection results generated by multiple detection paths and output a final detection result; the early warning module is used to perform root cause attribution based on the final detection result and data in a historical knowledge base and output a first early warning processing suggestion, or, based on monitoring indicators under the current detection scenario, output a second early warning processing suggestion.

[0007] According to the automatic optical inspection system provided in this application, the intelligent agent orchestration module is used to: acquire a structured inspection task corresponding to the image data, the structured inspection task including inspection requirements, defect types, and execution constraints; select a target processing step that matches the current inspection task from multiple image processing steps based on the image attributes of the image data and the inspection requirements; match the inspection requirements and defect types by calling a capability tag library to select a target calling step that matches the current inspection task from multiple expert model calling steps, the capability tag library being a pre-built structured database used to characterize the adaptation characteristics of each expert model; combine the target processing step with the target calling step to generate at least two candidate inspection paths, and filter the at least two candidate inspection paths according to the execution constraints to obtain a candidate inspection path set.

[0008] According to the automatic optical inspection system provided in this application, the candidate detection path includes at least one image processing step and at least one expert model invocation step; the image processing step includes one or more of the following: image cropping, image stitching, image alignment, downsampling texture enhancement, super-resolution enhancement, brightness equalization, contrast correction, geometric correction, denoising, and optical diffraction degradation simulation; the expert model invocation step includes one or more of the following: a deep learning-based defect detection model, a target detection-based defect localization model, a grayscale rule-based visual algorithm model, a texture statistics-based detection model, a super-resolution-based image quality optimization model, and a physical mechanism-based degradation simulation model.

[0009] According to the automatic optical inspection system provided in this application, the large language model decision module is used to: obtain the candidate detection path set; comprehensively score each candidate detection path in the candidate path set based on multi-dimensional scoring indicators, the multi-dimensional scoring indicators including accuracy, measurement error, execution latency, cross-batch stability, explanatory sufficiency, resource consumption and risk prior matching degree; sort each candidate detection path according to the comprehensive scoring results, and sequentially determine the main detection path and the backup detection path.

[0010] According to the automatic optical inspection system provided in this application, the path execution module is used to: execute image processing and expert model invocation steps according to the main inspection path; when the execution status of the main inspection path meets the preset switching conditions, suspend the subsequent execution of the main inspection path, and continue to execute the unexecuted subsequent steps in the backup inspection path based on the intermediate processing results already completed by the main inspection path; wherein, the preset switching conditions include at least one of the following: any step of the main inspection path fails to execute; the execution time of any step of the main inspection path exceeds a preset timeout threshold; the confidence level output by any step of the main inspection path is lower than a preset confidence level threshold.

[0011] According to the automatic optical inspection system provided in this application, the result fusion module is used to: integrate the detection results of the main detection path and the detection results of the backup detection path, and output the final detection result of the current detection task.

[0012] According to the automatic optical inspection system provided in this application, the result fusion module is specifically used for: mapping the defect coordinates in the detection results of two paths to the original image coordinate system to obtain defect location information under unified coordinates; performing standardized mapping on the defect labels of the two paths to obtain standardized defect category information; fusing the measurement data of the two paths using a weighted average method to obtain fused measurement data, wherein the weight of the weighted average method is determined based on the measurement error index of the two paths; integrating the defect location information under unified coordinates, the standardized defect category information, and the fused measurement data to obtain a standardized fusion detection result, and outputting the standardized fusion detection result as the final detection result.

[0013] According to the automatic optical inspection system provided in this application, the early warning module is used to: acquire multi-dimensional correlation features of abnormal events in the current inspection scenario, the multi-dimensional correlation features including defect features, process operation features, equipment status features, and environmental correlation features; traverse the historical knowledge base to determine target historical cases that match the abnormal events, the target historical cases including abnormal event features, abnormal root cause information, and abnormal handling measures; based on the matching degree between the multi-dimensional correlation features and the abnormal event features, perform a comprehensive risk score on the root cause information of all target historical cases, and generate a first early warning handling suggestion based on the comprehensive risk score; wherein, the first early warning handling suggestion includes root cause information sorted by priority, as well as the corresponding root cause derivation basis and abnormal handling measures.

[0014] According to the automatic optical detection system provided in this application, the early warning module is further configured to: perform instantaneous over-limit judgment and temporal change trend judgment on the multi-dimensional monitoring indicators based on an initial early warning threshold to obtain a preliminary anomaly judgment result; based on the preliminary anomaly judgment result, match the corresponding historical maintenance cycle, similar fault characteristics and fault occurrence probability in the historical knowledge base, update the judgment threshold and trend judgment duration of the initial early warning threshold to obtain an optimized early warning threshold; perform anomaly judgment on the multi-dimensional monitoring indicators again based on the optimized early warning threshold, and output a second early warning processing suggestion, wherein the second early warning processing suggestion includes anomaly type, risk level and corresponding handling suggestions.

[0015] An automatic optical inspection system provided in this application also includes a historical knowledge base, which is used to store historical cases, process rules, equipment maintenance records, early warning thresholds and model routing strategies, and to perform incremental updates based on manual review results, path execution logs and actual handling data.

[0016] The technical solutions provided by the embodiments of this application may include the following beneficial effects: By acquiring image data of the object to be detected through the data access module, a stable and reliable data source can be provided for the entire detection process, effectively ensuring the consistency and integrity of subsequent processing; Since the candidate detection path set is automatically generated through the intelligent agent orchestration module, it no longer relies on manual configuration of fixed detection links, thus flexibly adapting to different defect types, production line cycles, and changes in working conditions, breaking through the limitations of traditional single detection processes; Since the main detection path and backup detection path are intelligently selected through the large language model decision module, autonomous optimization and reasonable allocation of detection paths can be achieved, improving efficiency. This system enhances the adaptability and stability of the detection process. By executing two detection paths (primary and backup) through the path execution module, it avoids detection interruptions due to single-path anomalies, significantly improving system reliability and fault tolerance. The result fusion module verifies and fuses multi-path results before outputting the final detection result, effectively reducing measurement deviations, improving detection accuracy, and making the detection process more interpretable and traceable. Furthermore, the early warning module, combined with a historical knowledge base for root cause attribution or based on changes in monitoring indicators, provides early warning suggestions, enabling the early identification of potential anomalies and the location of root causes, shifting from passive detection to proactive early warning. Thus, this system forms a complete closed loop from data input, path generation, intelligent decision-making, stable execution to result fusion and anomaly early warning, effectively solving problems such as fixed AOI detection links, reliance on human experience, poor detection adaptability, lack of early warning capabilities, and difficulty in auditing. It significantly improves detection accuracy, operational robustness, and overall intelligence, better meeting the needs of efficient, stable, and traceable automated detection in modern industrial mass production scenarios. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 This is a schematic diagram of the automatic optical inspection system provided in this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0021] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0022] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0023] This application describes some exemplary embodiments for illustrative purposes. It should be understood that this application may be implemented in other ways not specifically shown in the accompanying drawings.

[0024] like Figure 1 As shown in the figure, this application provides an automatic optical inspection system, which may include a data access module 101, an intelligent agent orchestration module 102, a large language model decision module 103, a path execution module 104, a result fusion module 105, an early warning module 106, and a historical knowledge base 107.

[0025] The data access module 101 is used to acquire image data of the object to be inspected. The data access module 101 can establish a communication connection with the production line camera and image acquisition unit to read line scan images, area scan images, region slice images, or offline image data in real time. Secondly, it performs integrity verification, resolution identification, data format conversion, and abnormal data filtering on the acquired image data, removing invalid frames and corrupted data. Furthermore, it assigns a unique identifier to each valid image frame and binds it with associated information such as acquisition timestamp, workstation number, and batch identifier. Finally, it pushes the standardized image data to subsequent modules for path generation and analysis by the intelligent agent orchestration module. Through the data access module 101, multi-source heterogeneous image data can be compatible, achieving stable image access and standardized preprocessing, reducing the impact of hardware differences on the inspection process, and improving system versatility and data reliability.

[0026] The intelligent agent orchestration module 102 is used to generate a set of candidate detection paths based on the image data.

[0027] The large language model decision module 103 is used to determine the main detection path and the backup detection path from the set of candidate detection paths.

[0028] The path execution module 104 is used to execute the main detection path and the backup detection path to obtain defect detection results.

[0029] The result fusion module 105 is used to verify and fuse the defect detection results generated by multiple detection paths and output the final detection result.

[0030] The early warning module 106 is used to perform root cause attribution based on the final detection result and the data in the historical knowledge base and output a first early warning processing suggestion, or to output a second early warning processing suggestion based on the monitoring indicators under the current detection scenario.

[0031] The historical knowledge base 107 is used to store historical cases, process rules, equipment maintenance records, early warning thresholds, and model routing strategies, and is incrementally updated based on manual review results, path execution logs, and actual handling data. First, the automated optical inspection system can classify, organize, structure, and index historical inspection cases, process rules, equipment maintenance records, early warning thresholds, and model routing strategies. Second, based on the current inspection task, anomaly characteristics, or decision-making needs, it quickly matches and outputs corresponding historical data and rules. After each inspection task, the knowledge base content is incrementally updated and optimized based on manual review results, path execution logs, actual handling data, and early warning feedback information. Finally, the updated data undergoes version management and conflict verification to ensure the accuracy and timeliness of the knowledge. In this way, unified management, efficient reuse, and continuous iteration of knowledge throughout the entire inspection process can be achieved, improving the accuracy of path decisions, the reliability of anomaly attribution, and the precision of early warnings, reducing reliance on human experience, and promoting continuous self-optimization of the system.

[0032] Optionally, the intelligent agent orchestration module 102 is configured to: acquire a structured detection task corresponding to the image data, the structured detection task including detection requirements, defect types, and execution constraints; select a target processing step that matches the current detection task from multiple image processing steps based on the image attributes of the image data and the detection requirements; match the detection requirements and defect types by calling a capability tag library to select a target calling step that matches the current detection task from multiple expert model calling steps, the capability tag library being a pre-built structured database used to characterize the adaptation characteristics of each expert model; combine and configure the target processing step and the target calling step to generate at least two candidate detection paths, and filter the at least two candidate detection paths according to the execution constraints to obtain a candidate detection path set.

[0033] Specifically, the intelligent agent orchestration module 102 can acquire the image data of the object to be detected output by the data access module and simultaneously receive the structured detection task input from the front end, clarifying information such as the target object, defect type, measurement index, accuracy requirements, and latency constraints for this detection. Secondly, it performs feature analysis on the image data, extracting image attributes such as resolution, contrast, texture complexity, field of view, and grayscale distribution. Based on the detection requirements, it selects target processing steps from a pre-set image processing step library that can improve image quality and highlight defect features. Thirdly, according to the defect type, accuracy requirements, and speed constraints, it calls a pre-built capability tag library for retrieval and matching, selecting target calling steps from various expert models that have a higher detection rate, lower measurement error, and meet resource constraints in the current scenario. Subsequently, it orderly combines the target processing steps and target calling steps according to execution logic, constructing multiple candidate detection paths with different strategies to ensure that each path can independently complete the detection task. Finally, it performs legality verification and cost evaluation on all candidate paths based on execution constraints, eliminating unreasonable paths such as those that exceed time limits, resource limits, or accuracy standards, forming the final usable set of candidate detection paths. In this way, the automatic generation and filtering of detection paths can be achieved, improving path adaptability and generation efficiency, and meeting the differentiated needs of different detection tasks.

[0034] It should be noted that the structured detection task is to transform natural language or fuzzy requirements into executable standardized task information; image attributes are inherent features of the image such as resolution, contrast, size, and texture; the capability tag library is structured data that characterizes the applicable defect types, accuracy, speed, and resource consumption of each expert model; and the candidate detection path is a complete detection process formed by combining image processing and model invocation.

[0035] Optionally, the candidate detection path includes at least one image processing step and at least one expert model invocation step; the image processing step includes one or more of the following: image cropping, image stitching, image alignment, downsampling texture enhancement, super-resolution enhancement, brightness equalization, contrast correction, geometric correction, denoising, and optical diffraction degradation simulation; the expert model invocation step includes one or more of the following: a deep learning-based defect detection model, a target detection-based defect localization model, a grayscale rule-based visual algorithm model, a texture statistics-based detection model, a super-resolution-based image quality optimization model, and a physical mechanism-based degradation simulation model.

[0036] Optionally, the large language model decision module 103 is used to: obtain the candidate detection path set; comprehensively score each candidate detection path in the candidate path set based on multi-dimensional scoring indicators, the multi-dimensional scoring indicators including accuracy, measurement error, execution latency, cross-batch stability, explanatory sufficiency, resource consumption and risk prior matching degree; sort each candidate detection path according to the comprehensive scoring results, and determine the main detection path and the backup detection path in sequence.

[0037] Specifically, firstly, the large language model decision module 103 obtains the filtered candidate detection path set from the intelligent agent orchestration module 102, and simultaneously reads auxiliary information such as production line metadata, historical cases, and process rules. Secondly, based on preset multi-dimensional scoring indicators, each candidate path is quantitatively scored item by item, including the path's historical accuracy in similar tasks, estimated measurement error, expected execution latency, cross-batch production stability, decision interpretability, hardware resource consumption, and risk matching degree with the current equipment status. Thirdly, the scores of each dimension are weighted and summed to obtain the comprehensive score of each path. The weights can be dynamically adjusted according to the task type, with higher weights for high-precision tasks and higher weights for high-speed production lines. Then, the candidate paths are sorted from high to low according to the comprehensive score, and the one with the highest score is selected as the optimal execution plan. Finally, the path ranked first is determined as the main detection path used under normal conditions, and the path ranked second is determined as the backup detection path used for switching under abnormal conditions, and path selection instructions and risk warning information are generated simultaneously. In this way, intelligent and objective optimization of detection paths can be achieved, improving the rationality and reliability of path selection, while providing interpretable decision-making basis for the system.

[0038] It should be noted that the comprehensive score is the overall merit score of the path calculated by weighting multiple dimensions of indicators. The main detection path is the optimal path that is executed first under normal working conditions, and the backup detection path is the alternative path that is switched to when the main path is abnormal.

[0039] Optionally, the path execution module 104 is configured to: execute the image processing stage and the expert model invocation stage according to the main detection path; when the execution status of the main detection path is detected to meet the preset switching conditions, pause the subsequent execution of the main detection path, and, based on the intermediate processing results already completed by the main detection path, continue to execute the subsequent stages not yet executed in the backup detection path; wherein, the preset switching conditions include at least one of the following: any stage of the main detection path fails to execute; the execution time of any stage of the main detection path exceeds a preset timeout threshold; the confidence level output by any stage of the main detection path is lower than a preset confidence threshold.

[0040] Specifically, firstly, the path execution module 104 loads the main detection path output by the large language model decision module 103, and executes image processing and expert model inference sequentially according to the preset order in the path, saving the corresponding intermediate processing results after each step is completed. Secondly, during execution, the running status of each step is monitored in real time, including whether the step ends normally, whether the actual execution time exceeds the preset threshold, and whether the confidence level of the output result meets the requirements. When any step is detected to have failed execution, exceeded the time limit, or had too low a confidence level, the path switching mechanism is immediately triggered. Subsequently, the subsequent steps of the main detection path are paused, and all completed intermediate images, feature data, and model outputs are retained to avoid resource waste and redundant calculations. Finally, the incomplete steps are directly resumed from the corresponding position in the backup detection path, without starting from the beginning, and a complete defect detection result is output, with the switching reason and execution log recorded. In this way, detection interruption due to a single path anomaly can be avoided, improving the fault tolerance and continuity of the detection process, and ensuring detection efficiency and result integrity.

[0041] It should be noted that the intermediate processing results are the stage data output after the main path has completed some steps, and the preset switching condition is the abnormal judgment standard that triggers the switching of the main and backup paths.

[0042] Optionally, the result fusion module 105 is used to: integrate the detection results of the main detection path and the detection results of the backup detection path, and output the final detection result of the current detection task. Specifically, it maps the defect coordinates in the detection results of the two paths to the original image coordinate system to obtain defect location information under unified coordinates; performs standardized mapping on the defect labels of the two paths to obtain standardized defect category information; fuses the measurement data of the two paths using a weighted average method to obtain fused measurement data, wherein the weights of the weighted average method are determined based on the measurement error indices of the two paths; integrates the defect location information under unified coordinates, the standardized defect category information, and the fused measurement data to obtain a standardized fused detection result, and outputs the standardized fused detection result as the final detection result.

[0043] Specifically, firstly, the result fusion module 105 reads the defect detection results output by the main detection path and the backup detection path, including information such as defect coordinates, category labels, area, length, and confidence level. Secondly, it performs a unified mapping on the defect coordinates, converting all local coordinates generated by different cropping scales, stitching methods, and ROI extractions into the global coordinate system of the original image to ensure consistent defect locations. Thirdly, it standardizes the defect category labels output by different models, unifying labels with different naming systems and classification granularities into the system's standard defect type, eliminating semantic differences. Then, it assigns weighted weights based on the historical measurement error indicators of the two paths, with higher weights for paths with smaller errors, and performs weighted averaging fusion on measurement data such as defect size and quantity to reduce the impact of random errors. Finally, it integrates the unified defect location, standardized category, and fused measurement data to form a standardized detection result with a complete structure, unified format, and higher reliability, which is then output as the final detection result. In this way, the differences between multi-path results can be eliminated, the accuracy of defect location, category, and measurement data can be improved, and a standardized, highly reliable final detection result can be output.

[0044] It should be noted that coordinate mapping is the operation of converting the defect locations of different scales and regions into a unified coordinate system of the original image, standardized labels are a unified definition of defect categories, and weighted average fusion is a measurement data integration method that allocates weights based on path accuracy.

[0045] Optionally, the early warning module 106 is configured to: acquire multi-dimensional correlation features of abnormal events in the current detection scenario, the multi-dimensional correlation features including defect features, process operation features, equipment status features, and environmental correlation features; traverse the historical knowledge base to determine target historical cases that match the abnormal events, the target historical cases including abnormal event features, abnormal root cause information, and abnormal handling measures; based on the matching degree between the multi-dimensional correlation features and the abnormal event features, perform a comprehensive risk score on the root cause information of all target historical cases, and generate a first early warning handling suggestion based on the comprehensive risk score; wherein, the first early warning handling suggestion includes root cause information sorted by priority, as well as the corresponding root cause derivation basis and abnormal handling measures.

[0046] Specifically, firstly, the early warning module 106 extracts multi-dimensional correlation features of abnormal events from the final detection results, including defect features such as morphology, size, distribution, and density; process operation features such as production line speed, formula parameters, and detection thresholds; equipment status features such as camera parameters, light source status, and runtime; and environmental correlation features such as temperature, humidity, and vibration. Secondly, it traverses the historical knowledge base to retrieve target historical cases with a similarity to the current features exceeding a preset threshold, obtaining the abnormal features, root cause types, handling measures, and treatment effects recorded in the cases. Thirdly, it performs a comprehensive risk score on each candidate root cause based on feature matching degree, case credibility, and historical occurrence probability; the higher the score, the greater the likelihood of the root cause. Then, it sorts the candidate root causes from high to low risk scores to form a priority list. Finally, combining the root cause derivation basis, historical similar cases, and standard handling procedures, it generates a first early warning handling suggestion that includes root cause ranking, derivation evidence, and handling steps. In this way, it can achieve rapid and intelligent location of abnormal root causes, provide traceable and executable early warning suggestions, and improve the efficiency and accuracy of abnormal handling.

[0047] It should be noted that the multi-dimensional correlation features are comprehensive characteristics of defects, processes, equipment, environment, etc. related to the abnormal event, the comprehensive risk score is a quantitative score that represents the likelihood of the root cause, and the first early warning handling suggestion is a root cause location and handling guide generated based on historical cases.

[0048] Optionally, the early warning module 106 is further configured to: perform instantaneous over-limit determination and time-series change trend determination on the multi-dimensional monitoring indicators based on the initial early warning threshold to obtain a preliminary anomaly determination result; based on the preliminary anomaly determination result, match the corresponding historical maintenance cycle, similar fault characteristics and fault occurrence probability in the historical knowledge base, update the determination threshold and trend determination duration of the initial early warning threshold to obtain an optimized early warning threshold; perform anomaly determination on the multi-dimensional monitoring indicators again based on the optimized early warning threshold, and output a second early warning processing suggestion, wherein the second early warning processing suggestion includes anomaly type, risk level and corresponding handling suggestion.

[0049] Specifically, firstly, the early warning module 106 can collect multi-dimensional monitoring indicators such as image brightness, contrast, sharpness, and signal-to-noise ratio in real time. These multi-dimensional monitoring indicators can reflect the equipment's operating status and image quality. Secondly, it compares the real-time values ​​of the indicators with the initial early warning threshold to complete the instantaneous over-limit judgment. At the same time, it analyzes the changing trend of the indicators within a continuous time window to determine whether there is continuous decay, continuous increase, or abnormal fluctuation, and obtains a preliminary anomaly judgment result. Thirdly, based on the type and trend characteristics of the preliminary anomaly, it matches the historical maintenance cycle, typical characteristic curves, and failure probability of similar faults in the historical knowledge base. Based on this, it adaptively adjusts the judgment threshold and trend judgment duration to obtain an optimized early warning threshold that better fits the current equipment status. Then, it uses the optimized early warning threshold to judge the monitoring indicators again to confirm whether the anomaly is real and eliminates false judgments caused by instantaneous interference. Finally, based on the confirmation result, it generates a second early warning processing suggestion containing the anomaly type, risk level, and recommended handling measures, and pushes it to the operation and maintenance personnel. In this way, it can identify progressively deteriorating anomalies, avoid false alarms and missed alarms caused by fixed thresholds, and improve the accuracy and advance warning of early warnings.

[0050] It should be noted that the multi-dimensional monitoring indicators are quantitative indicators that reflect the status of equipment and image quality; the instantaneous over-limit judgment is a single comparison between the real-time value of the indicator and the threshold; the time-series trend judgment is an analysis of the changing pattern of the indicator over a continuous period of time; and the optimized warning threshold is a warning judgment standard that is dynamically adjusted based on historical data.

[0051] In this embodiment, the image data of the object to be detected is acquired through the data access module, providing a stable and reliable data source for the entire detection process and effectively ensuring the consistency and integrity of subsequent processing. Since the candidate detection path set is automatically generated through the intelligent agent orchestration module, it no longer relies on manual configuration of fixed detection links, thus flexibly adapting to different defect types, production line cycles, and changes in operating conditions, breaking through the limitations of traditional single detection processes. The intelligent selection of the main and backup detection paths through the large language model decision module enables autonomous optimization and reasonable allocation of detection paths, improving the adaptability and stability of the detection process. The execution of both the main and backup detection paths through the path execution module avoids detection interruptions due to single-path anomalies, significantly improving the reliability and fault tolerance of system operation. The verification and fusion of multi-path results through the result fusion module, followed by the output of the final detection result, effectively reduces measurement deviations, improves detection accuracy, and makes the detection process more interpretable and traceable. The early warning module, combined with a historical knowledge base for root cause attribution or outputting early warning suggestions based on changes in monitoring indicators, can identify potential anomalies and locate the root cause of problems in advance, achieving a shift from passive detection to proactive early warning. Thus, this system forms a complete closed loop from data input, path generation, intelligent decision-making, stable execution to result fusion and anomaly warning, effectively solving the problems of fixed AOI detection links, reliance on human experience, poor detection adaptability, inability to provide early warnings, and difficulty in auditing existing systems. It significantly improves detection accuracy, operational robustness, and overall intelligence level, and can better meet the needs of efficient, stable, and traceable automated detection in modern industrial mass production scenarios.

[0052] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0053] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0054] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An automatic optical inspection system, characterized in that, include: The module includes a data access module, an agent orchestration module, a large language model decision module, a path execution module, a result fusion module, and an early warning module. The data access module is used to acquire image data of the object to be detected; The intelligent agent orchestration module is used to generate a set of candidate detection paths based on the image data; The large language model decision module is used to determine the main detection path and backup detection paths from the candidate detection path set; The path execution module is used to execute the main detection path and the backup detection path to obtain defect detection results; The result fusion module is used to verify and fuse the defect detection results generated by multiple detection paths and output the final detection result. The early warning module is used to perform root cause attribution based on the final detection result and the data in the historical knowledge base and output a first early warning processing suggestion, or to output a second early warning processing suggestion based on the monitoring indicators under the current detection scenario.

2. The automatic optical inspection system according to claim 1, characterized in that, The intelligent agent orchestration module is used for: Obtain the structured detection task corresponding to the image data, wherein the structured detection task includes detection requirements, defect types, and execution constraints; Based on the image attributes of the image data and the detection requirements, a target processing step that is suitable for the current detection task is selected from multiple image processing steps. The capability tag library is used to match the detection requirements and the defect type. The target calling link that is suitable for the current detection task is selected from multiple expert model calling links. The capability tag library is a pre-built structured database used to characterize the adaptation characteristics of each expert model. The target processing stage and the target invocation stage are combined and configured to generate at least two candidate detection paths. The at least two candidate detection paths are then filtered according to the execution constraints to obtain a set of candidate detection paths.

3. The automatic optical inspection system according to claim 2, characterized in that, The candidate detection path includes at least one image processing step and at least one expert model invocation step; The image processing steps include one or more of the following: image cropping, image stitching, image alignment, downsampling texture enhancement, super-resolution enhancement, brightness equalization, contrast correction, geometric correction, noise reduction, and optical diffraction degradation simulation. The expert model invocation process includes one or more of the following: a deep learning-based defect detection model, a target detection-based defect localization model, a grayscale rule-based visual algorithm model, a texture statistics-based detection model, a super-resolution-based image quality optimization model, and a physical mechanism-based degradation simulation model.

4. The automatic optical inspection system according to claim 2 or 3, characterized in that, The large language model decision module is used for: Obtain the candidate detection path set; Each candidate detection path in the candidate path set is comprehensively scored based on multi-dimensional scoring indicators, including accuracy, measurement error, execution latency, cross-batch stability, explanatory sufficiency, resource consumption, and risk prior matching degree. Based on the comprehensive scoring results, the candidate detection paths are sorted, and the main detection path and the backup detection paths are determined in sequence.

5. The automatic optical inspection system according to claim 4, characterized in that, The path execution module is used for: The image processing stage and the expert model invocation stage are executed according to the main detection path. When the execution status of the main detection path meets the preset switching conditions, the subsequent execution of the main detection path is paused, and the subsequent stages that have not been executed in the backup detection path are executed based on the intermediate processing results that have been completed in the main detection path. The preset switching conditions include at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; or the confidence level of the output of any step of the main detection path is lower than a preset confidence threshold.

6. The automatic optical inspection system according to claim 5, characterized in that, The result fusion module is used to integrate the detection results of the main detection path and the detection results of the backup detection path, and output the final detection result of the current detection task.

7. The automatic optical inspection system according to claim 6, characterized in that, The result fusion module is specifically used for: The defect coordinates in the detection results of the two paths are mapped to the original image coordinate system to obtain the defect location information under unified coordinates; Perform standardized mapping on the defect labels of the two paths to obtain standardized defect category information; The measurement data of the two paths are fused using a weighted average method to obtain fused measurement data. The weights of the weighted average method are determined based on the measurement error index of the two paths. The defect location information under unified coordinates, standardized defect category information, and fused measurement data are integrated to obtain standardized fusion detection results, which are then output as the final detection results.

8. The automatic optical inspection system according to claim 1, characterized in that, The early warning module is used for: Obtain multi-dimensional correlation features of abnormal events in the current detection scenario. The multi-dimensional correlation features include defect features, process operation features, equipment status features, and environmental correlation features. Traverse the historical knowledge base to determine target historical cases that match the abnormal event. The target historical cases include abnormal event characteristics, abnormal root cause information, and abnormal handling measures. Based on the matching degree between the multi-dimensional correlation features and the abnormal event features, a comprehensive risk score is performed on the root cause information of all target historical cases, and a first early warning processing suggestion is generated based on the comprehensive risk score. The first early warning processing suggestion includes root cause information sorted by priority, as well as the corresponding root cause derivation basis and anomaly handling measures.

9. The automatic optical inspection system according to claim 8, characterized in that, The early warning module is also used for: Based on the initial warning threshold, the multi-dimensional monitoring indicators are instantaneously over-limited and time-series change trend is determined to obtain preliminary anomaly judgment results. Based on the preliminary anomaly determination results, the judgment threshold and trend judgment duration of the initial warning threshold are updated by matching the corresponding historical maintenance cycle, similar fault characteristics and fault occurrence probability in the historical knowledge base to obtain the optimized warning threshold. Based on the optimized early warning threshold, the multi-dimensional monitoring indicators are re-evaluated for anomalies, and a second early warning handling suggestion is output. The second early warning handling suggestion includes the anomaly type, risk level, and corresponding handling suggestions.

10. The automatic optical inspection system according to claim 1, characterized in that, It also includes a historical knowledge base, which is used to store historical cases, process rules, equipment maintenance records, early warning thresholds and model routing strategies, and to perform incremental updates based on manual review results, path execution logs and actual handling data.

Images