Automatic optical inspection path execution method, device, apparatus, storage medium and program product

Abstract

This application discloses an automatic optical detection path execution method, apparatus, device, storage medium, and program product. The method includes: acquiring a main detection path and a backup detection path for the current detection task, each of the main and backup detection paths including at least one image processing step and at least one expert model invocation step; executing the image processing step and the expert model invocation step according to the main detection path; when the execution status of the main detection path meets a preset switching condition, pausing the subsequent execution of the main detection path, and based on the intermediate processing results already completed in the main detection path, continuing to execute the unexecuted subsequent steps in the backup detection path; integrating the detection results of the main detection path and the detection results of the backup detection path, and outputting the final detection result of the current detection task.

Description

Technical Field

[0001] This application relates to the field of industrial automated optical inspection technology, and more specifically, to an automated optical inspection path execution method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] In the field of industrial automated optical inspection (AOI), automated optical inspection systems have become core equipment for ensuring product quality, targeting defect detection and measurement tasks for products such as panels, wafers, and precision devices.

[0003] Existing automated optical inspection systems typically employ a single, fixed inspection path to perform defect identification and measurement tasks. That is, the system is pre-configured with a fixed inspection process, including fixed steps such as image acquisition, feature extraction, defect classification, and result output, and completes the entire production line inspection based on this inspection process.

[0004] However, existing solutions cannot adapt to the dynamic operating conditions of production lines in real time. When an abnormality occurs during the testing process, the lack of an automatic fault-tolerance mechanism can easily lead to the interruption of the testing process and production line shutdown, seriously affecting production efficiency and equipment utilization. Summary of the Invention

[0005] The main objective of this application is to provide an automatic optical inspection path execution method, apparatus, equipment, storage medium, and program product to solve the problems of existing AOI systems that are prone to interruption due to abnormalities and have poor adaptability when using a single fixed path. This application enables dynamic fault-tolerant switching, ensures continuous operation of the production line, and improves the robustness of the inspection system and the utilization rate of the equipment.

[0006] To achieve the above objectives, a first aspect of this application proposes an automatic optical detection path execution method, comprising: acquiring a main detection path and a backup detection path for a current detection task, wherein both the main detection path and the backup detection path include at least one image processing step and at least one expert model invocation step; executing the image processing step and the expert model invocation step according to the main detection path; when the execution status of the main detection path is detected to meet a preset switching condition, pausing the subsequent execution of the main detection path, and continuing to execute the unexecuted subsequent steps in the backup detection path based on the intermediate processing results already completed by the main detection path; integrating the detection results of the main detection path and the detection results of the backup detection path, and outputting the final detection result of the current detection task; wherein the preset switching condition includes at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; the confidence level output by any step of the main detection path is lower than a preset confidence threshold.

[0007] According to the automatic optical detection path execution method provided in this application, the step of continuing to execute the unexecuted subsequent steps in the backup detection path based on the intermediate processing results of the completed main detection path includes: recording the abnormal step index of the main detection path, filtering out the connecting steps that are functionally compatible with the abnormal steps from the backup detection path; using the intermediate processing results of the completed main detection path as the input of the connecting step, and sequentially executing the subsequent steps of the backup detection path starting from the connecting step.

[0008] According to the automatic optical inspection path execution method provided in this application, the step of integrating the detection results of the main inspection path and the detection results of the backup inspection path to output the final detection result of the current inspection task includes: mapping the defect coordinates in the detection results of the two paths to the original image coordinate system to obtain defect location information under unified coordinates; performing standardized mapping on the defect labels of the two paths to obtain standardized defect category information; fusing the measurement data of the two paths using a weighted average method to obtain fused measurement data, wherein the weight of the weighted average method is determined based on the measurement error index of the two paths; integrating the defect location information under unified coordinates, the standardized defect category information, and the fused measurement data to obtain a standardized fused detection result, and outputting the standardized fused detection result as the final detection result.

[0009] According to the automatic optical inspection path execution method provided in this application, after integrating the detection results of the main inspection path and the detection results of the backup inspection path and outputting the final detection result of the current inspection task, the method further includes: generating a consistency score based on the detection results of the main inspection path and the detection results of the backup inspection path, wherein the consistency score is used to measure the degree of consistency between different paths in terms of defect location and measurement value; if the consistency score is lower than a preset consistency threshold, the corresponding inspection sample is marked as a high-dispute sample and manual review is triggered.

[0010] According to the automatic optical detection path execution method provided in this application, the method further includes: recording the execution information of each step to generate a full-process execution log during the execution of the image processing step and the expert model invocation step; generating an evidence package based on the full-process execution log; wherein the evidence package includes at least one of the following: original image identifier, candidate path list, main detection path identifier, backup detection path identifier, execution parameters of each step, model version, execution time, detection result, fusion result, judgment reason, warning information and manual review information.

[0011] According to the automatic optical detection path execution method provided in this application, the method further includes: obtaining the verification path of the current detection task; selecting a target detection sample from the current detection task according to a preset sampling rule, executing the verification path on the target detection sample, and obtaining the verification result of the target detection sample; and comparing and verifying the verification result with the final detection result.

[0012] This application also provides an automatic optical detection path execution device, comprising the following modules: an acquisition module, a processing module, and an output module; the acquisition module is used to acquire the main detection path and the backup detection path of the current detection task, wherein both the main detection path and the backup detection path include at least one image processing step and at least one expert model invocation step; the processing module is used to execute the image processing step and the expert model invocation step according to the main detection path, and when the execution status of the main detection path is detected to meet the preset switching conditions, to pause the subsequent execution of the main detection path, and to continue executing the unexecuted subsequent steps in the backup detection path based on the intermediate processing results already completed by the main detection path; the output module is used to integrate the detection results of the main detection path and the detection results of the backup detection path, and output the final detection result of the current detection task; wherein the preset switching conditions include at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; the confidence level of the output of any step of the main detection path is lower than a preset confidence threshold.

[0013] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic optical detection path execution method as described above.

[0014] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automatic optical detection path execution method as described above.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the automatic optical detection path execution method as described above.

[0016] The technical solution provided by the embodiments of this application can include the following beneficial effects: Since the main detection path and backup detection path are obtained in advance, and the execution is carried out sequentially according to the main detection path while the execution status is monitored in real time, when an anomaly occurs in the main detection path, such as a failure in the execution of a step, a timeout exceeding a preset timeout threshold, or an output confidence level lower than a preset confidence threshold, the subsequent execution of the main detection path can be suspended in a timely manner, avoiding the problem of interruption of detection due to an anomaly in the existing single fixed path. Furthermore, since the switching is based on the intermediate processing results already completed in the main detection path, the subsequent steps not yet executed in the backup detection path are executed, eliminating the need to repeatedly process completed data. Therefore, the continuous execution of the detection process can be ensured, production line downtime can be avoided, and production efficiency and equipment utilization can be significantly improved. Simultaneously, by integrating the execution results of the two paths to output the final detection result, the integrity and reliability of the detection result are further guaranteed, effectively adapting to the detection needs of dynamic working conditions in industrial AOI scenarios. Thus, dynamic fault-tolerant switching can be achieved, effectively adapting to dynamic working conditions of the production line, thereby significantly improving the robustness of the detection system and equipment utilization. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 One of the flowcharts for the automatic optical detection path execution method provided in this application; Figure 2 The second flowchart illustrates the automatic optical detection path execution method provided in this application. Figure 3 The third flowchart illustrating the automatic optical detection path execution method provided in this application; Figure 4 This is a schematic diagram of the automatic optical detection path execution device provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0021] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0022] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0023] This application describes some exemplary embodiments for illustrative purposes. It should be understood that this application may be implemented in other ways not specifically shown in the accompanying drawings.

[0024] like Figure 1 As shown, this application provides an automatic optical detection path execution method, which can be applied to an automatic optical detection path execution device. The automatic optical detection path execution method may include steps S101-S103: S101, The automatic optical inspection path execution device obtains the main inspection path and backup inspection path of the current inspection task.

[0025] The main detection path and the backup detection path both include at least one image processing step and at least one expert model invocation step.

[0026] Specifically, the main detection path and the backup detection path are both output after being scored and sorted based on production line metadata, historical cases and process rules. The main detection path is the detection path with the best comprehensive score under the current detection task. It is more suitable for the current detection constraints and production line conditions in terms of defect detection rate, measurement accuracy, execution latency and resource consumption. The backup detection path is a suboptimal path that complements the main detection path. It can seamlessly take over when the main detection path fails, ensuring the continuity of the detection process. Both detection paths can independently complete the defect detection and measurement requirements of the current detection task.

[0027] Furthermore, each detection path includes at least one image processing step and at least one expert model invocation step. The image processing step can perform preprocessing and optimization of the detection image, while the expert model invocation step can realize defect detection and measurement. The image processing step and the expert model invocation step of the two paths can be flexibly combined according to the objectives of the detection task. The image processing step includes at least one of the following: image cropping, stitching, alignment, downsampling texture enhancement, super-resolution enhancement, brightness equalization, contrast correction, geometric correction, denoising, and optical diffraction degradation simulation. The expert model invocation step includes at least one of the following: a deep learning-based defect detection model, a target detection-based defect localization model, a grayscale rule-based visual algorithm model, a texture statistics-based detection model, a super-resolution-based image quality optimization model, and a physical mechanism-based degradation simulation model.

[0028] It should be noted that when the automatic optical inspection path execution device acquires these two inspection paths, it can also simultaneously acquire the corresponding path selection reasons, key constraint satisfaction status, potential risk items, and preset switching conditions. The path selection rationale clearly explains the selection logic and adaptation basis of the main detection path and backup detection path, making it easier for operators to understand the system decision-making process. It also provides a clear reference direction for subsequent detection strategy review and path optimization, and can serve as evidence of the rationality of the decision during quality audits and customer reviews. The key constraint satisfaction status can be used to determine the compliance status of the main detection path and backup detection path with core constraints such as detection task latency requirements, measurement accuracy, resource consumption, and false negative / false positive rate thresholds. This allows the device to confirm the matching of the path with the current detection task before executing the path, avoiding invalid detection results or production line interruption due to the path not meeting the constraints. Potential risk items can inform the device and operators in advance of potential problems such as insufficient hardware resources, unstable model inference, and sensitivity to image quality during the execution of the main detection path and backup detection path. This allows the device to prepare for resource allocation and anomaly monitoring in advance, and also provides a reference for operators to develop contingency plans. The preset switching conditions are clear quantitative and qualitative standards for determining whether the main detection path needs to be switched to the backup detection path.

[0029] S102. The automatic optical detection path execution device executes the image processing stage and the expert model calling stage according to the main detection path. When the execution status of the main detection path is detected to meet the preset switching conditions, the subsequent execution of the main detection path is paused, and the unexecuted subsequent stages in the backup detection path are executed based on the intermediate processing results that have been completed in the main detection path.

[0030] The preset switching conditions include at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; or the confidence level of the output of any step of the main detection path is lower than a preset confidence threshold.

[0031] Specifically, the automatic optical inspection path execution device can first call the corresponding image processing algorithms and expert models sequentially according to the order of the main inspection path steps to complete the execution operation. During the execution process, the automatic optical inspection path execution device can monitor the running status of each step in real time. The specific monitoring dimensions can cover whether the step execution is completed normally, whether the execution time is within the preset range, and whether the confidence level of the step output result meets the preset standard, ensuring that various operational anomalies of the main inspection path can be captured in a timely manner. When the monitoring results show that the execution status of the main inspection path meets any preset switching condition, the automatic optical inspection path execution device can first record the index position of the abnormal step, clarify the main inspection path execution interruption boundary, and then immediately suspend the scheduling and operation of the unexecuted steps in the main inspection path to avoid unnecessary computational power and time consumption. At the same time, the intermediate processing results generated by the completed steps of the main inspection path are cached and marked to ensure the integrity and callability of these results. Subsequently, based on the pre-acquired backup detection path configuration and the abnormal step index of the main detection path, the connecting steps in the backup detection path that are functionally compatible with the abnormal steps are selected as the starting nodes. The cached intermediate processing results are used as the input data for this connecting step, and the subsequent unexecuted image processing steps and expert model invocation steps are executed sequentially according to the step order of the backup detection path. The entire switching and continuation process requires no manual intervention and retains the execution record of the main detection path, the abnormal step index, and the cause of the abnormality throughout, enabling seamless connection of the detection process.

[0032] For example, in a wafer micro-defect detection task, the main detection path is "image cropping - super-resolution enhancement - deep learning defect detection model - measurement model". Each step corresponds to index 1, index 2, index 3, and index 4 respectively. When the automatic optical detection path execution device executes the super-resolution enhancement step of index 2, it detects that the execution time of this step exceeds the timeout threshold preset by the production line. It then records the abnormal step index as index 2, immediately suspends the execution of the defect detection model and measurement model of subsequent indexes 3 and 4, caches the cropped image data output by the image cropping step of index 1, and then calls the backup detection path "image cropping - edge enhancement - target detection defect model - measurement model". It selects the edge enhancement step that is compatible with the function of the super-resolution enhancement step of index 2 as the connection starting node, uses the cropped image data as the input of this step, and executes all subsequent steps in sequence starting from the edge enhancement step.

[0033] Optionally, the step of continuing to execute subsequent steps that have not been executed in the backup detection path based on the intermediate processing results that have been completed in the main detection path includes: recording the index of abnormal steps in the main detection path, filtering out connecting steps that are functionally compatible with the abnormal steps from the backup detection path; using the intermediate processing results that have been completed in the main detection path as the input of the connecting step, and sequentially executing the subsequent steps of the backup detection path starting from the connecting step.

[0034] Specifically, after the automatic optical inspection path execution device detects that the main inspection path meets the preset switching conditions and suspends its subsequent execution, it can first assign node numbers and status markers to all execution stages of the main inspection path. This allows for precise location and recording of the index position of the abnormal stage within the main inspection path, clarifying the boundary between the abnormal node and the normally completed stages, thus providing accurate node references for the subsequent connection of the backup inspection path. Subsequently, it performs functional analysis on all image processing stages and expert model invocation stages of the backup inspection path, extracting the functional attributes, input data requirements, and output data types of each stage. Based on the output functions of the completed stages of the main inspection path and the functional location of the abnormal stages, it selects connecting stages from the backup inspection path that are functionally compatible with the abnormal stages of the main inspection path and whose input data matches the intermediate processing results of the completed stages of the main inspection path. This connecting stage serves as the starting point for the continued execution of the backup inspection path, ensuring that the intermediate processing results of the main inspection path can be directly used as valid input for the backup inspection path. After completing the selection of the connecting links, the automatic optical detection path execution device can adapt and convert the intermediate processing results generated by the completed links of the main detection path, which have been cached and marked, according to the input data format requirements of the connecting links of the backup detection path. The adapted intermediate processing results are used as the input data of the connecting links. Finally, according to the inherent link order of the backup detection path, all subsequent unexecuted image processing links and expert model calling links are scheduled and executed sequentially from the connecting link until the execution of the entire backup detection path is completed.

[0035] For example, in the task of detecting contamination in precision components, the main detection path is "image denoising - brightness equalization - grayscale rule detection model - measurement model". The abnormal link is the grayscale rule detection model, with an index of 3. After recording the index of the abnormal link, the automatic optical detection path execution device can perform functional analysis on the backup detection path "image denoising - brightness equalization - target detection defect model - measurement model", select the target detection defect model that is functionally compatible with the grayscale rule detection model as the connecting link, and use the optimized image output from the image denoising and brightness equalization links already executed in the main detection path as the input of the target detection defect model. Starting from this link, the subsequent measurement models are executed sequentially.

[0036] The above process achieves a fine connection between the main detection path and the backup detection path through precise recording of abnormal link index and functional filtering of connection links. This avoids the resource consumption and time delay caused by the backup detection path executing from the beginning. At the same time, the format adaptation and conversion of intermediate processing results ensures the effectiveness and compatibility of data transmission between the main and backup paths.

[0037] S103. The automatic optical detection path execution device integrates the detection results of the main detection path and the detection results of the backup detection path, and outputs the final detection result of the current detection task.

[0038] Optionally, integrating the detection results of the main detection path and the detection results of the backup detection path to output the final detection result of the current detection task includes: mapping the defect coordinates in the detection results of the two paths to the original image coordinate system to obtain defect location information under unified coordinates; performing standardized mapping on the defect labels of the two paths to obtain standardized defect category information; fusing the measurement data of the two paths using a weighted average method to obtain fused measurement data, wherein the weight of the weighted average method is determined based on the measurement error index of the two paths; integrating the defect location information under unified coordinates, the standardized defect category information, and the fused measurement data to obtain a standardized fused detection result, and outputting the standardized fused detection result as the final detection result.

[0039] Specifically, the automatic optical inspection path execution device first performs spatial registration processing on the inspection results output by the main inspection path and the backup inspection path. Addressing the different defect coordinate systems resulting from cropping, stitching, and scaling operations in the image processing stage, a coordinate transformation algorithm is used to uniformly map all defect coordinates to the coordinate system of the original acquired image. This eliminates coordinate deviations between paths, accurately obtaining defect location information under unified coordinates, ensuring the uniqueness and consistency of the location of the same defect detected by different paths. After coordinate registration, the defect labels output by the two inspection paths are standardized. Based on a preset unified defect semantic label system, the defect category descriptions output by different expert models are normalized, eliminating differences in label definitions between models, obtaining standardized defect category information, and ensuring the uniformity of defect category determination results. Subsequently, a weighted average method is used to fuse the measurement data of defect area, length, width, and height output from the two detection paths. First, historical measurement error indices are extracted for the main and backup detection paths. These indices comprehensively reflect the measurement accuracy and stability of the paths in similar detection tasks. Then, corresponding weights are assigned to the two paths based on the measurement error indices; the smaller the measurement error, the higher the weight. Weighted calculations are then performed on the same measurement index based on the assigned weights to obtain the fused measurement data, improving the accuracy and reliability of the measurement results. Finally, the defect location information under unified coordinates, standardized defect category information, and the fused measurement data are structurally integrated and organized according to the output requirements of the detection task to form a standardized fused detection result. This result is then output as the final detection result of the current detection task, ensuring the completeness, standardization, and effectiveness of the final detection result.

[0040] It should be noted that the above processing steps, through spatial registration, label standardization, and weighted fusion of measurement data, solve the problems of inconsistent formats and data caused by different processing links in multi-path detection results, and achieve efficient fusion of multi-path detection results. At the same time, the weighting strategy based on measurement error index makes the fusion results more in line with actual detection needs, effectively improving the accuracy and reference value of the final detection results.

[0041] Optionally, after integrating the detection results of the main detection path and the detection results of the backup detection path to output the final detection result of the current detection task, the automatic optical detection path execution device can generate a consistency score based on the detection results of the main detection path and the detection results of the backup detection path. The consistency score is used to measure the degree of consistency between different paths in terms of defect location and measurement value. If the consistency score is lower than a preset consistency threshold, the corresponding detection sample is marked as a high-dispute sample and manual review is triggered.

[0042] Specifically, after the automatic optical inspection path execution device completes the fusion of the detection results of the main inspection path and the backup inspection path and outputs the final detection result of the current inspection task, it can also generate a consistency score based on the defect location information and standardized measurement data of the two inspection paths that have completed spatial registration. This consistency calculation algorithm can comprehensively consider the spatial deviation value of the defect location and the numerical deviation rate of the same measurement index. After quantifying and weighting the two types of deviations, a standardized consistency score is obtained. The higher the value, the higher the consistency between the two paths in defect location determination and measurement data output. The lower the value, the greater the deviation between the two detection results. This achieves accurate measurement of the matching degree of detection results of different paths.

[0043] Furthermore, after generating the consistency score, the automatic optical inspection path execution device can compare it with a preset consistency threshold. This preset consistency threshold is pre-set based on the accuracy requirements of the inspection task and the production line quality standards. If the consistency score is higher than or equal to the preset consistency threshold, it is determined that the matching degree of the two path inspection results meets the standard, and no additional intervention is required. If the consistency score is lower than the preset consistency threshold, it is determined that the deviation of the multi-path inspection results of the current inspection sample exceeds the reasonable range. The automatic optical inspection path execution device can immediately mark the inspection sample as a highly controversial sample and automatically trigger the manual review process. It pushes the relevant inspection information of the inspection sample to the manual review end, including the original image, the complete inspection results of the main and backup paths, the basis for the consistency score calculation, and the final inspection result after fusion, providing comprehensive and detailed reference data for manual review.

[0044] It should be noted that the above process achieves quantitative verification of the reliability of multi-path detection results through consistency scores. It can accurately identify highly controversial samples with questionable detection results, avoid detection errors caused by single-path deviation or mismatch of multi-path results. At the same time, the automatic triggering of the manual review process allows highly controversial samples to be professionally reviewed and confirmed by humans, which effectively improves the accuracy and credibility of the final detection results and reduces the probability of missed defects and misjudgments.

[0045] In this embodiment, since the main detection path and backup detection path are pre-acquired and executed sequentially according to the main detection path while the execution status is monitored in real time, when the main detection path experiences anomalies such as execution failure, time exceeding a preset timeout threshold, or output confidence level lower than a preset confidence threshold, the subsequent execution of the main detection path can be promptly paused, avoiding the problem of interrupting detection due to anomalies in existing single fixed paths. Furthermore, because the switching is based on the intermediate processing results already completed in the main detection path, the subsequent unexecuted steps in the backup detection path are executed, eliminating the need to reprocess completed data. This ensures continuous detection process, avoids production line downtime, and significantly improves production efficiency and equipment utilization. Simultaneously, by integrating the execution results of the two paths to output the final detection result, the integrity and reliability of the detection result are further guaranteed, effectively adapting to the dynamic detection requirements of industrial AOI scenarios. Thus, dynamic fault-tolerant switching can be achieved, effectively adapting to dynamic production line conditions, thereby significantly improving the robustness of the detection system and equipment utilization.

[0046] like Figure 2 As shown in the embodiments of this application, an automatic optical detection path execution method may further include S201-S202: S201. During the execution of the image processing stage and the expert model invocation stage, the automatic optical detection path execution device records the execution information of each stage to generate a full-process execution log.

[0047] Specifically, the automatic optical inspection path execution device can perform full-dimensional, uninterrupted real-time recording of the execution information of each stage during the sequential execution of the main inspection path and the expert model invocation stage of the backup inspection path. This forms a traceable, end-to-end execution log, and the recording operation can cover the entire lifecycle of path execution, from the start of the first stage of the main inspection path to the end of the last stage of the backup inspection path, ensuring the integrity of the log information. The recorded execution information can include the input identifier, output identifier, execution parameters, execution time, and status information of each stage. The input and output identifiers are used to trace the input and output data of each stage. The execution parameters are the algorithm parameters of each image processing stage and the inference parameters of the expert model invocation stage. The execution time is the actual running time of each stage from start to completion. The status information includes the running status such as normal completion, execution failure, timeout, and abnormal confidence. At the same time, the path switching nodes and switching reasons can also be recorded synchronously in the execution log, making each execution step verifiable and traceable.

[0048] S202, The automatic optical detection path execution device generates an evidence package based on the full-process execution log.

[0049] The evidence package includes at least one of the following: original image identifier, candidate path list, main detection path identifier, backup detection path identifier, execution parameters of each stage, model version, execution time, detection result, fusion result, judgment reasoning, warning information and manual review information.

[0050] Specifically, the automatic optical detection path execution device can use the full-process execution log as the core data source, and combine it with the full-cycle relevant data of the current detection task to perform structured integration and standardized encapsulation to generate an evidence package with full-link traceability capabilities. The content of the evidence package can cover the key information of the entire process of the detection task from path generation to result output and re-judgment, and at least include one or more of the following: original image identifier, candidate path list, main detection path identifier, backup detection path identifier, execution parameters of each stage, model version, execution time, detection result, fusion result, judgment reason, warning information and manual re-judgment information. Among them, the original images associated with the detection task can be used to trace the source of image data; the candidate path list and the primary and backup detection path identifiers can be used to clarify the path selection and the final selection result; the execution parameters, model version, and execution time of each link are extracted and integrated from the full-process execution log to restore the actual execution configuration and runtime of each link; the detection result is the original detection data output by each of the two detection paths, and the fusion result is the final detection result after spatial registration, label standardization, and measurement data fusion; the judgment reason is the basis for the selection of primary and backup paths output by the large language model judgment module; the warning information is the equipment, process, and quality-related warnings generated during the detection process; and the manual review information is the manual review conclusion, review time, review personnel, and other relevant records of highly controversial samples.

[0051] It should be noted that the automatic optical inspection path execution device stores all information in the evidence package in a structured manner and assigns a unique identifier to each evidence package, which can achieve precise association with the inspection task and the inspection sample.

[0052] It should be noted that the generation of the evidence package unifies and encapsulates key data from the entire testing process, which can be directly used for internal quality traceability, customer audit verification, and regulatory compliance audit. It also provides complete dataset support for subsequent model iteration analysis and testing strategy optimization, making the entire AOI testing process fully traceable and interpretable.

[0053] like Figure 3 As shown, the automatic optical detection path execution method provided in this application embodiment may further include S301-S303: S301, The automatic optical inspection path execution device obtains the verification path of the current inspection task.

[0054] Specifically, the automatic optical inspection path execution device can obtain the verification path corresponding to the current inspection task. This verification path is an independent inspection link from the main inspection path and the backup inspection path. It also consists of at least one image processing step and at least one expert model calling step. Compared with the main inspection path and the backup inspection path, the verification path focuses more on the accuracy and authority of the inspection results in the optimization strategy of the image processing step and the model selection of the expert model calling step. Therefore, the verification path adopts a more accurate model or a more detailed image processing process.

[0055] S302. The automatic optical inspection path execution device selects a target inspection sample from the current inspection task according to a preset sampling rule, executes the verification path on the target inspection sample, and obtains the verification result of the target inspection sample.

[0056] Specifically, after acquiring the verification path, the automatic optical inspection path execution device can select target inspection samples from all inspection samples of the current inspection task according to the preset sampling rules through a standardized screening process. The preset sampling rules are determined by the production line quality control requirements and the defect risk level of the inspection task, and may include fixed proportion sampling, key area sample sampling, random sampling, abnormal trend sample sampling, etc. For batches with high defect incidence and high precision requirements, the sampling ratio can be increased. For regular batches, fixed proportion or random sampling can be used to ensure that the selected target inspection samples are representative and can effectively reflect the overall accuracy of the inspection results of the current inspection task. After the target detection sample is selected, the automatic optical detection path execution device can sequentially execute the corresponding image processing and expert model invocation steps according to the step sequence of the verification path. During the execution, it follows the same execution information recording rules as the main detection path and backup detection path, and records the input and output, execution parameters, and running status of each step in real time. It operates independently throughout the process and is not affected by the execution process of the main detection path and backup detection path. Finally, it outputs complete detection information such as defect location, category, and measurement data of the target detection sample as the verification result of the sample. The output format of the verification result is standardized and consistent with the detection results of the main and backup paths to facilitate subsequent comparison and analysis.

[0057] S303, The automatic optical detection path execution device compares and verifies the verification result with the final detection result.

[0058] Specifically, after obtaining the verification result of the target inspection sample, the automatic optical inspection path execution device can compare the verification result with the final inspection result in all dimensions. The comparison dimensions cover defect location, defect type, defect quantity, and various measurement data. Among them, defect location comparison is based on spatial deviation quantification analysis of the original image coordinate system, defect type and quantity are verified for consistency matching, and measurement data comparison calculates the numerical deviation of indicators such as defect area, length, width, and height to ensure the comprehensiveness and accuracy of the comparison results. During the comparison process, the result of each comparison item is quantified and marked, and consistent items and deviation items are identified. For deviation items, the deviation value is calculated and compared with a preset deviation allowable threshold to determine whether the deviation is within a reasonable range. After completing the full-dimensional comparison, the automatic optical inspection path execution device can comprehensively judge the comparison results of the target inspection sample. If all the contents of the verification result and the final inspection result are consistent, or the deviation value of the deviation item is within the preset allowable threshold, the final inspection result of the sample is judged to be accurate and valid. If there are deviation items that exceed the allowable deviation threshold, or if there is a significant inconsistency in the defect category or quantity, the final inspection result of the sample is judged to be questionable. The automatic optical inspection path execution device can mark the sample with questionable final inspection result and trigger a secondary review process. At the same time, the comparison result is synchronously recorded to the full process execution log and included in the evidence package for storage.

[0059] Furthermore, the automatic optical inspection path execution device can also summarize and statistically analyze the comparison and verification results of all target inspection samples in the current inspection task, calculate the overall accuracy and pass rate of the inspection results, and provide data support for the overall quality assessment of the current inspection task and the optimization of the main and backup path strategies.

[0060] It should be noted that by comparing and verifying the final test results through the verification path, detection deviations, missed detections, and misjudgments can be accurately identified, effectively improving the accuracy and reliability of the test results and reducing quality risks.

[0061] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0062] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0063] The automatic optical detection path execution method provided in this application can be executed by an automatic optical detection path execution device or a control module for automatic optical detection path execution within that device. This application uses the execution of the automatic optical detection path execution method by the automatic optical detection path execution device as an example to illustrate the automatic optical detection path execution device provided in this application.

[0064] It should be noted that the embodiments of this application can divide the automatic optical detection path execution device into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. Optionally, the module division in the embodiments of this application is illustrative and is only a logical functional division; other division methods may be used in actual implementation.

[0065] like Figure 4 As shown, this application embodiment provides an automatic optical detection path execution device 400. The automatic optical detection path execution device 400 includes: an acquisition module 401, a processing module 402, and an output module 403. The acquisition module 401 is used to acquire the main detection path and the backup detection path of the current detection task. Both the main detection path and the backup detection path include at least one image processing step and at least one expert model invocation step. The processing module 402 is used to execute the image processing step and the expert model invocation step according to the main detection path. When the execution status of the main detection path meets the preset switching conditions, the subsequent execution of the main detection path is paused, and the unexecuted subsequent steps in the backup detection path are executed based on the intermediate processing results already completed in the main detection path. The output module 403 is used to integrate the detection results of the main detection path and the detection results of the backup detection path and output the final detection result of the current detection task. The preset switching conditions include at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; the confidence level of the output of any step of the main detection path is lower than a preset confidence threshold.

[0066] Optionally, the step of continuing to execute subsequent steps that have not been executed in the backup detection path based on the intermediate processing results that have been completed in the main detection path includes: recording the index of abnormal steps in the main detection path, filtering out connecting steps that are functionally compatible with the abnormal steps from the backup detection path; using the intermediate processing results that have been completed in the main detection path as the input of the connecting step, and sequentially executing the subsequent steps of the backup detection path starting from the connecting step.

[0067] Optionally, integrating the detection results of the main detection path and the detection results of the backup detection path to output the final detection result of the current detection task includes: mapping the defect coordinates in the detection results of the two paths to the original image coordinate system to obtain defect location information under unified coordinates; performing standardized mapping on the defect labels of the two paths to obtain standardized defect category information; fusing the measurement data of the two paths using a weighted average method to obtain fused measurement data, wherein the weight of the weighted average method is determined based on the measurement error index of the two paths; integrating the defect location information under unified coordinates, the standardized defect category information, and the fused measurement data to obtain a standardized fused detection result, and outputting the standardized fused detection result as the final detection result.

[0068] Optionally, after integrating the detection results of the main detection path and the detection results of the backup detection path and outputting the final detection result of the current detection task, the processing module 402 is used to generate a consistency score based on the detection results of the main detection path and the detection results of the backup detection path. The consistency score is used to measure the degree of consistency between different paths in terms of defect location and measurement value. If the consistency score is lower than a preset consistency threshold, the corresponding detection sample is marked as a high-dispute sample and manual review is triggered.

[0069] Optionally, the processing module 402 is used to record the execution information of each step during the execution of the image processing step and the expert model invocation step to generate a full-process execution log; and to generate an evidence package based on the full-process execution log; wherein the evidence package includes at least one of the following: original image identifier, candidate path list, main detection path identifier, backup detection path identifier, execution parameters of each step, model version, execution time, detection result, fusion result, judgment reason, warning information and manual review information.

[0070] Optionally, the acquisition module 401 is used to acquire the verification path of the current detection task; the processing module 402 is used to select a target detection sample from the current detection task according to a preset sampling rule, execute the verification path on the target detection sample, and obtain the verification result of the target detection sample; and compare and verify the verification result with the final detection result.

[0071] In this embodiment, since the main detection path and backup detection path are pre-acquired and executed sequentially according to the main detection path while the execution status is monitored in real time, when the main detection path experiences anomalies such as execution failure, time exceeding a preset timeout threshold, or output confidence level lower than a preset confidence threshold, the subsequent execution of the main detection path can be promptly paused, avoiding the problem of interrupting detection due to anomalies in existing single fixed paths. Furthermore, because the switching is based on the intermediate processing results already completed in the main detection path, the subsequent unexecuted steps in the backup detection path are executed, eliminating the need to reprocess completed data. This ensures continuous detection process, avoids production line downtime, and significantly improves production efficiency and equipment utilization. Simultaneously, by integrating the execution results of the two paths to output the final detection result, the integrity and reliability of the detection result are further guaranteed, effectively adapting to the dynamic detection requirements of industrial AOI scenarios. Thus, dynamic fault-tolerant switching can be achieved, effectively adapting to dynamic production line conditions, thereby significantly improving the robustness of the detection system and equipment utilization.

[0072] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logic instructions in the memory 530 to execute an automatic optical detection path execution method. This method includes: acquiring a main detection path and a backup detection path for the current detection task, each including at least one image processing step and at least one expert model invocation step; executing the image processing step and the expert model invocation step according to the main detection path; when the execution status of the main detection path meets a preset switching condition, pausing the subsequent execution of the main detection path, and continuing to execute the unexecuted subsequent steps in the backup detection path based on the intermediate processing results already completed in the main detection path; integrating the detection results of the main detection path and the detection results of the backup detection path, and outputting the final detection result of the current detection task; wherein the preset switching condition includes at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; the confidence level output by any step of the main detection path is lower than a preset confidence threshold.

[0073] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the automatic optical detection path execution method provided by the above methods. The method includes: obtaining a main detection path and a backup detection path for the current detection task, wherein both the main detection path and the backup detection path include at least one image processing step and at least one expert model invocation step; executing the image processing step and the expert model invocation step according to the main detection path; when the execution status of the main detection path is detected to meet a preset switching condition, pausing the subsequent execution of the main detection path, and continuing to execute the unexecuted subsequent steps in the backup detection path based on the intermediate processing results already completed by the main detection path; integrating the detection results of the main detection path and the detection results of the backup detection path, and outputting the final detection result of the current detection task; wherein the preset switching condition includes at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; the confidence level output by any step of the main detection path is lower than a preset confidence threshold.

[0075] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an automatic optical detection path execution method provided by the above methods. The method includes: obtaining a main detection path and a backup detection path for the current detection task, wherein both the main detection path and the backup detection path include at least one image processing step and at least one expert model invocation step; executing the image processing step and the expert model invocation step according to the main detection path; when the execution status of the main detection path is detected to meet a preset switching condition, pausing the subsequent execution of the main detection path, and continuing to execute the unexecuted subsequent steps in the backup detection path based on the intermediate processing results already completed by the main detection path; integrating the detection results of the main detection path and the detection results of the backup detection path, and outputting the final detection result of the current detection task; wherein the preset switching condition includes at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; the confidence level output by any step of the main detection path is lower than a preset confidence threshold.

[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0077] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0078] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An automatic optical detection path execution method, characterized in that, include: Obtain the main detection path and backup detection path for the current detection task. Both the main detection path and the backup detection path include at least one image processing step and at least one expert model invocation step. The image processing stage and the expert model invocation stage are executed according to the main detection path. When the execution status of the main detection path meets the preset switching conditions, the subsequent execution of the main detection path is paused, and the subsequent stages that have not been executed in the backup detection path are executed based on the intermediate processing results that have been completed in the main detection path. Integrate the detection results of the main detection path with the detection results of the backup detection path, and output the final detection result of the current detection task; The preset switching conditions include at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; or the confidence level of the output of any step of the main detection path is lower than a preset confidence threshold.

2. The automatic optical detection path execution method according to claim 1, characterized in that, The step of continuing to execute subsequent steps that have not yet been executed in the backup detection path based on the intermediate processing results already completed in the main detection path includes: Record the index of abnormal links in the main detection path, and select the connection links that are functionally compatible with the abnormal links from the backup detection paths; The intermediate processing results of the main detection path are used as the input of the connecting link, and the subsequent links of the backup detection path are executed sequentially starting from the connecting link.

3. The automatic optical detection path execution method according to claim 1, characterized in that, The process of integrating the detection results from the main detection path and the detection results from the backup detection path to output the final detection result of the current detection task includes: The defect coordinates in the detection results of the two paths are mapped to the original image coordinate system to obtain the defect location information under unified coordinates; Perform standardized mapping on the defect labels of the two paths to obtain standardized defect category information; The measurement data of the two paths are fused using a weighted average method to obtain fused measurement data. The weights of the weighted average method are determined based on the measurement error index of the two paths. The defect location information under unified coordinates, standardized defect category information, and fused measurement data are integrated to obtain standardized fusion detection results, which are then output as the final detection results.

4. The automatic optical detection path execution method according to claim 1, characterized in that, After integrating the detection results of the main detection path and the detection results of the backup detection path, and outputting the final detection result of the current detection task, the method further includes: A consistency score is generated based on the detection results of the main detection path and the detection results of the backup detection path. The consistency score is used to measure the degree of consistency between different paths in terms of defect location and measurement value. If the consistency score is lower than the preset consistency threshold, the corresponding test sample will be marked as a highly controversial sample and manual review will be triggered.

5. The automatic optical detection path execution method according to any one of claims 1-4, characterized in that, The method further includes: During the execution of the image processing stage and the expert model invocation stage, the execution information of each stage is recorded to generate a full-process execution log; An evidence package is generated based on the aforementioned full-process execution log; The evidence package includes at least one of the following: original image identifier, candidate path list, main detection path identifier, backup detection path identifier, execution parameters of each stage, model version, execution time, detection result, fusion result, judgment reasoning, warning information and manual review information.

6. The automatic optical detection path execution method according to any one of claims 1-4, characterized in that, The method further includes: Obtain the verification path for the current detection task; According to the preset sampling rules, target detection samples are selected from the current detection task, and the verification path is executed on the target detection samples to obtain the verification result of the target detection samples; The verification result is compared and verified with the final detection result.

7. An automatic optical detection path execution device, characterized in that, include: The module includes an acquisition module, a processing module, and an output module. The acquisition module is used to acquire the main detection path and the backup detection path of the current detection task. The main detection path and the backup detection path each include at least one image processing step and at least one expert model invocation step. The processing module is used to execute the image processing stage and the expert model calling stage according to the main detection path. When the execution status of the main detection path is detected to meet the preset switching conditions, the subsequent execution of the main detection path is paused, and the subsequent stages that have not been executed in the backup detection path are executed based on the intermediate processing results that have been completed in the main detection path. The output module is used to integrate the detection results of the main detection path and the detection results of the backup detection path, and output the final detection result of the current detection task. The preset switching conditions include at least one of the following: any step of the main detection path fails to execute; the execution time of any step of the main detection path exceeds a preset timeout threshold; or the confidence level of the output of any step of the main detection path is lower than a preset confidence threshold.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the automatic optical detection path execution method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the automatic optical detection path execution method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the automatic optical detection path execution method as described in any one of claims 1 to 6.

Images