Camera and light source imaging quantitative monitoring method, system and medium in AOI detection

Abstract

This application relates to the field of industrial vision inspection and equipment health management technology. A method for quantitative monitoring of camera and light source imaging in AOI inspection includes: acquiring multiple frames of reference images and constructing a benchmark profile based on these frames; acquiring multiple frames of images to be evaluated and extracting fixed and adaptive ROI regions from these frames to obtain a set of clean ROI regions; extracting multiple indicators from the set of clean ROI regions to obtain multiple indicator vectors; robustly aggregating these indicator vectors and comparing them with the benchmark profile to obtain a relative degradation rate and standardized difference index describing imaging degradation; using the relative degradation rate and standardized difference index to decompose imaging degradation into defocus / out-of-focus degradation indicators, motion blur degradation indicators, and optical anomaly degradation indicators to obtain the sub-item health and overall health of imaging degradation; and then obtaining an estimated remaining lifetime and a semantic diagnosis and chain of evidence.

Description

Technical Field

[0001] This application relates to the field of industrial vision inspection and equipment health management technology, and more specifically, to a method, system and medium for quantitative monitoring of camera and light source imaging in AOI inspection. Background Technology

[0002] In AOI inspection production lines related to panels and semiconductors, multiple cameras and multiple light sources need to operate continuously for a long time. Affected by factors such as camera focal plane drift, mechanism vibration, lens contamination, light source attenuation, and changes in illumination distribution, the image clarity and contrast will gradually decrease, directly causing fluctuations in defect detection rate and significantly increasing the risk of false alarms and missed detections, becoming an important factor restricting the stable operation of the production line.

[0003] Currently, the industry relies heavily on manual inspection or offline sampling to monitor the imaging status of AOI inspection equipment. These methods are not timely and make it difficult to provide quantifiable, traceable, and interpretable health assessments of the operating status of each camera and each light source within the production cycle. This makes it impossible to achieve refined and real-time control over the imaging equipment status.

[0004] Meanwhile, variations in actual production line conditions and differences in product patterns can severely impact the stability of monitoring indicators such as image clarity. Traditional, simple threshold-based judgment strategies are prone to generating numerous false alarms, increasing the ineffective work of maintenance personnel. Therefore, the industry urgently needs an imaging health and lifespan monitoring solution that can quantify image degradation based on the initial state, accurately pinpoint problem types, and provide professional handling recommendations. Summary of the Invention

[0005] The main purpose of this application is to provide solutions to the technical problems in the background art.

[0006] To achieve the above objectives, the first aspect of this application proposes a method for quantitative monitoring of camera and light source imaging in AOI detection, comprising: After the camera and light source are calibrated, multiple reference images are acquired, and a reference file is constructed based on the multiple reference images. The reference file includes the identifier ID that forms the multiple reference images and the reference parameters. During production operation, a camera is used to acquire multiple frames of images to be evaluated within a preset detection window. Fixed ROI regions and adaptive ROI regions are extracted from the multiple frames of images to be evaluated. Based on the AOI detection results, interfering regions in the fixed ROI regions and the adaptive ROI regions are removed to obtain a set of clean ROI regions. For the set of pure ROI regions, multiple indicators are extracted to obtain multiple indicator vectors, wherein the multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. Robust aggregation of multiple indicator vectors is performed to form a window indicator vector, which is then compared with the benchmark archive to obtain the relative decay rate and standardized difference index describing image degradation. Based on the relative degradation rate and standardized difference index, the imaging degradation is decomposed into defocusing degradation index, motion blur degradation index and optical anomaly degradation index, and the corresponding sub-item health and overall health of the imaging degradation are obtained. The sub-health scores and the total health score are combined over time or over a window sequence to form a health index curve. The records and semantic interpretations are then linked to obtain an estimated remaining lifespan and a semantic diagnosis and chain of evidence.

[0007] In some feasible methods, the step of acquiring multiple reference images after the camera and light source are calibrated, and constructing a reference archive based on the multiple reference images, includes: After the camera and light source are calibrated, multiple reference images are acquired, and the multiple reference images are associated with the identifier ID to determine the associated identifier information of the reference file. The identifier ID includes camera ID, light source ID, recipe ID and ROI identifier. Based on the association identification information of the benchmark archive, the corresponding ROI region is extracted for each frame of the benchmark image and the single value of multiple indicators is calculated. The single values ​​of the same type of multiple indicators of multiple frames of the benchmark image under the same association identification information are statistically analyzed to obtain the median and absolute median difference of each indicator as the benchmark statistics. The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. The ROI region is processed for illumination surface and a baseline illumination surface template is generated. The baseline statistics and the baseline illumination surface template are used together as the components of the baseline parameter index vector. The associated identification information is bound to the corresponding baseline parameter index vector to obtain the baseline file.

[0008] In some feasible methods, the step of acquiring multiple frames of images to be evaluated using a camera within a preset detection window during production operation, extracting fixed ROI regions and adaptive ROI regions from the multiple frames of images to be evaluated, and removing interfering regions from the fixed ROI regions and the adaptive ROI regions based on the AOI detection results to obtain a set of clean ROI regions includes: During production operation, a detection window is constructed for the same camera according to a preset number of detections, and multiple frames of images to be evaluated are collected within the detection window. For each frame of the image to be evaluated, a preset fixed ROI region and an adaptive ROI region selected based on gradient energy are extracted to form a set of ROI regions corresponding to each frame of the image. Based on the AOI detection results, the defect region mask and saturation region mask of the image to be evaluated are obtained, and the defect region and saturation region are identified as interference regions. The interfering regions are removed from the set of ROI regions corresponding to each frame of the image to be evaluated, the effective ROI regions of each frame of the image to be evaluated are extracted, and the effective ROI regions of multiple frames of the image to be evaluated are aggregated and summarized according to the detection window dimension to obtain the set of clean ROI regions.

[0009] In some feasible methods, the step of extracting multiple indicators from the set of pure ROI regions to obtain multiple indicator vectors includes: For each frame of the image to be evaluated in the set of pure ROI regions, a sharpness index based on gradient energy is formed by calculating the sum of squared gradient magnitudes, and a sharpness index based on second-order difference energy is formed by calculating the variance or mean energy of the Laplacian response. The sharpness index based on gradient energy and the sharpness index based on second-order difference energy are combined to obtain the sharpness index value of the effective ROI region of each frame of the image to be evaluated. For each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is calculated using the structure tensor or gradient direction energy distribution to obtain the directional index value of the effective ROI region of each frame of the image to be evaluated, which is used to characterize the directionality of motion blur. For each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is obtained by low-pass filtering or surface fitting and normalized. Then, the similarity, vignetting index and uniformity error are calculated with the reference illumination surface template to obtain the illumination surface index value of the effective ROI region of each frame of the image to be evaluated. For each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is obtained by calculating the histogram distance of the effective ROI region of each frame of the image to be evaluated through grayscale or brightness quantile drift, saturation pixel ratio change and chi-square distance or Earth movement distance. For each frame of the image to be evaluated in the set of pure ROI regions, the noise index value of the effective ROI region of each frame of the image to be evaluated is obtained by using at least one of high-frequency residual energy, row and column fringe intensity and fixed pattern noise index. The sharpness index value, the directionality index value, the illumination surface index value, the histogram drift index value, and the noise index value are aggregated to form a single-frame index vector corresponding to each frame of the image to be evaluated. Then, the single-frame index vectors of multiple frames of the image to be evaluated in the clean ROI region set are combined to obtain multiple index vectors.

[0010] In some feasible approaches, the step of robustly aggregating multiple indicator vectors to form a window indicator vector, and comparing it with the benchmark archive to obtain a relative degradation rate and a standardized difference index describing image degradation includes: For multiple indicator vectors within a preset detection window, the median or truncated mean is used to calculate the window aggregate value of each indicator according to the indicator type. The absolute median difference is then used to characterize the fluctuation of each indicator after aggregation. The window aggregate value of multiple indicators and the corresponding fluctuation are combined one by one according to the indicator type to obtain the window indicator vector. Based on the identifier ID associated with the window indicator vector, the benchmark statistics corresponding to each indicator in the matching benchmark file are retrieved. Then, the window indicator vector and the benchmark statistics are matched one by one by indicator type to obtain the one-to-one matching relationship between the multi-indicator window aggregate value and the benchmark statistics. In the one-to-one matching relationship between the multi-index window aggregate value and the benchmark statistic, the relative degradation rate of the sharpness index, directionality index, illumination surface index and histogram drift index is obtained by subtracting the ratio of the window aggregate value to the corresponding benchmark statistic from 1. The larger the values ​​of the sharpness index, directionality index, illumination surface index and histogram drift index, the better the imaging state. In the one-to-one matching relationship between the multi-index window aggregated value and the benchmark statistic, the relative decay rate of the noise index is obtained by subtracting 1 from the ratio of the window aggregated value to the corresponding benchmark statistic. The larger the index value of the noise index, the worse the imaging state. Based on the window aggregate value of multiple indicators, the corresponding benchmark statistics, and the absolute median difference in the benchmark statistics, the standardized difference value of multiple indicators is calculated separately according to the indicator type. Based on the one-to-one correspondence of the relative decay rates of sharpness, directionality, illumination, and histogram drift, the relative decay rate of noise, and the standardized difference values ​​of multiple indicators, the relative decay rate and standardized difference index describing image degradation are obtained.

[0011] In some feasible methods, the step of decomposing the imaging degradation into defocus / out-of-focus degradation indicators, motion blur degradation indicators, and optical anomaly degradation indicators based on the relative degradation rate and standardized difference index, and obtaining the corresponding sub-item health and overall health of the imaging degradation, includes: The relative degradation rate of the sharpness index is merged and quantified with the standardized difference value to obtain the defocusing degradation index, wherein the defocusing degradation index is used to characterize the degree of defocusing in the image. The relative degradation rate of the sharpness index and the directionality index is fused and quantized with the standardized difference value. When the sharpness index degrades and the directionality index increases, the representation weight of the fused quantization is increased to obtain the motion blur degradation index, wherein the motion blur degradation index is used to characterize the degree of degradation of imaging motion blur. The relative decay rate of the illumination surface index, the histogram drift index, and the noise index is fused and quantified with the standardized difference value. The illumination surface index, the histogram drift index, and the noise index respectively characterize the decay state of light source aging, abnormal light intensity distribution, and abnormal acquisition link sensor, to obtain the optical anomaly degradation index. The optical anomaly degradation index is used to characterize the overall decay degree of imaging optical anomalies. Based on preset quantitative evaluation rules, the defocusing degradation index, the motion blur degradation index, and the optical anomaly degradation index are weighted and calculated respectively to obtain the imaging sub-item health of the defocusing degradation index, the motion blur degradation index, and the optical anomaly degradation index. According to the preset health score weight allocation rules, the imaging health scores of the imaging degradation types are comprehensively weighted and fused to obtain the total health score. The total health score represents the overall imaging status of the camera and the light source. The imaging degradation types include out-of-focus, motion blur, and optical anomaly.

[0012] In some feasible methods, the step of forming a health index curve by combining the sub-items of health with the total health over time or over a window sequence, and binding records and semantic interpretations to obtain an estimated remaining lifespan and a semanticized diagnosis and chain of evidence, includes: The sub-item health scores and the total health scores are aligned according to a time series or detection window sequence. The cumulative operating conditions are then fused to fit and calculate the above health score data to obtain the imaging health index curve of the camera and the light source. The cumulative operating conditions include at least one of the following: cumulative number of shots, cumulative light source illumination duration, and cumulative exposure. The degradation trend analysis and feature extraction of the imaging health index curve are performed. Combined with the preset imaging maintenance threshold, the degradation degree of the imaging status of the camera and the light source is quantitatively evaluated, and the remaining lifespan of the imaging of the camera and the light source is estimated. The identifier ID, the features of the imaging health index curve, the remaining lifespan estimate, the sub-item health and the total health data, and the imaging degradation type are associated and integrated to generate a parameter result binding record that includes indicator evidence, degradation conclusion, and confidence level. The parameter result binding records are enhanced by searching historical cases, maintenance manuals, and production line process events. The enhanced information and the parameter result binding records are then input into a large language model to generate a semantic diagnostic report. The semantic diagnostic report includes at least the core cause of imaging degradation, an explanation of the indicator evidence chain, recommended actions, and verification steps.

[0013] Secondly, this application provides a camera and light source imaging quantization monitoring system for AOI inspection, applied to the aforementioned camera and light source imaging quantization monitoring method for AOI inspection. The system includes: The data acquisition module is used to acquire multiple frames of reference images after the camera and light source are calibrated, and to construct a reference file based on the multiple frames of reference images. The reference file includes an identifier ID that forms the multiple frames of reference images, and reference parameters. The ROI management module is used to acquire multiple frames of images to be evaluated within a preset detection window using a camera during production operation, extract fixed ROI regions and adaptive ROI regions from the multiple frames of images to be evaluated, and remove interfering regions from the fixed ROI regions and the adaptive ROI regions based on the AOI detection results to obtain a set of clean ROI regions. The indicator calculation module is used to extract multiple indicators from the set of pure ROI regions to obtain multiple indicator vectors. The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. The benchmark and comparison module is used to robustly aggregate multiple index vectors to form a window index vector, which is then compared with the benchmark file to obtain the relative decay rate and standardized difference index describing image degradation. The degradation decomposition and alarm module is used to decompose the imaging degradation into defocusing degradation index, motion blur degradation index and optical anomaly degradation index based on the relative degradation rate and standardized difference index, and obtain the sub-item health and total health of the imaging degradation accordingly. The lifespan assessment module and the binding and semantic interpretation module are used to form a health index curve by combining the sub-items of health and the total health over time or over a window sequence, and to bind the records and semantic interpretation to obtain an estimated remaining lifespan and a semantic diagnosis and chain of evidence.

[0014] Thirdly, this application provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned AOI detection camera and light source imaging quantization monitoring method.

[0015] Fourthly, this application provides a computer program that, when executed by a processor, implements the steps of the aforementioned AOI detection camera and light source imaging quantization monitoring method.

[0016] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This application discloses a quantitative monitoring method for camera and light source imaging in AOI inspection. It constructs a baseline profile based on the calibrated state of the camera and light source, replacing traditional manual inspection and offline sampling methods. Relying on an online image acquisition and processing flow with a preset detection window, it can monitor the imaging status of each camera and each light source within the production cycle, achieving refined and real-time control of the imaging equipment status. By extracting fixed and adaptive ROI regions and removing interfering regions, and combining multi-index extraction and robust aggregation to form a window index vector, which is then compared with the baseline profile, the method weakens the impact of production line operating condition changes and product pattern differences on the stability of monitoring indicators. This avoids the numerous false alarms associated with traditional simple threshold judgment strategies and reduces the ineffective work of maintenance personnel. This method precisely decomposes imaging degradation into three categories of degradation indicators: defocusing, motion blur, and optical anomalies. It outputs quantifiable and traceable sub-item health and overall health, enabling precise localization of imaging problems. Furthermore, it can generate a health index curve based on health over time or window sequences to estimate remaining lifespan. Combined with binding records and semantic interpretation, it outputs semantic diagnosis and evidence chain explanations, forming a complete monitoring solution that quantifies imaging degradation based on the initial state, accurately locates problems, and outputs professional handling basis. This effectively improves the problem of defect detection rate fluctuations caused by decreased imaging clarity and contrast, significantly reduces the risk of false alarms and missed alarms, solves the industry pain point of AOI inspection production line imaging equipment status monitoring, and improves the overall stable operation capability of the production line. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 This application provides a logic diagram for a camera and light source imaging quantization monitoring method in AOI detection.

[0018] Figure 2 This application provides a flowchart of a method for quantitative monitoring of camera and light source imaging in AOI detection. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0022] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0023] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0024] The following explanations of some terms used in this application are provided to aid in understanding the technical solution of this application: Gradient energy is a core quantitative indicator in the fields of digital image processing and machine vision. It is used to characterize the degree of change in pixel grayscale values ​​in a local area of ​​an image. Essentially, it reflects the richness and obviousness of features such as edges, textures, and details in the image. In AOI (Automated Optical Inspection) scenarios, it is a key basis for evaluating the sharpness of camera imaging and selecting effective analysis areas. The more drastic the grayscale change, the higher the gradient energy value, and the clearer the details / edge features of the image.

[0025] Second-order difference energy is a core quantitative indicator for characterizing image sharpness in digital image processing. It belongs to the image feature parameters of the second-order differential domain. In this AOI inspection camera and light source health monitoring scheme, it is the core calculation basis for the sharpness indicator based on second-order difference energy. It is used to characterize the rate of change of pixel gray value within the ROI region of the image (i.e., the degree of change of gray gradient). Essentially, it reflects the steepness and sharpness of image edges and details. The sharper the edges and the clearer the details, the higher the second-order difference energy value. It is an important sharpness evaluation dimension that complements the sharpness indicator based on gradient energy (first-order difference).

[0026] The Laplacian operator is a classic second-order differential operator in the field of digital image processing. It is also a core tool for calculating sharpness indicators based on second-order difference energy. Its core function is to capture the second-order change (change in the rate of gray-level change) of image pixel gray values. It is highly sensitive to the sharpness of image edges and the steepness of gray-level abrupt changes, and is a key technical means to evaluate image sharpness and judge camera image degradation.

[0027] The structure tensor method is a core method for calculating directional indices. It is a classic algorithm in the field of digital image processing for extracting gradient direction features and texture direction saliency of local image regions. Its core function is to accurately quantify the directional distribution of pixel gray-level gradients within the ROI region of an image, thereby depicting the directionality and intensity of motion blur and providing a quantitative basis for subsequent determination of the type of motion blur and image degradation.

[0028] Chi-square distance is one of the core quantitative methods for calculating histogram distance in histogram drift metrics. It is a distance metric in the field of statistics, specifically used to measure the similarity between two probability distributions / frequency distributions.

[0029] Robust aggregation is an anti-interference data fusion method in the fields of statistical analysis and industrial machine vision. In the scenario of AOI (Automated Optical Inspection) camera and light source imaging health monitoring, it specifically refers to the highly robust statistical calculation of similar imaging index values ​​of multiple frames of images within the detection window. The core is to select statistics that resist outliers, eliminate the influence of index outliers caused by non-equipment degradation factors such as occasional defects and operating condition disturbances, and obtain index results that can truly reflect the overall imaging status within the window.

[0030] like Figure 1 As shown, this application adopts the following technical solution. First, after the camera and light source are manually calibrated, a benchmark file is established. The benchmark file stores sharpness indicators, directionality indicators, illumination surface templates, histogram templates, and noise statistics according to camera ID, light source ID, formula ID, and ROI identifier. During production operation, an evaluation window is constructed according to a preset number of tests N. Fixed ROIs and adaptive ROIs are extracted from several frames of images within the window, and defective and saturated regions are removed by combining the AOI detection output. A multi-index vector is calculated for the removed ROIs, where the sharpness index reflects high-frequency detail, the directionality index reflects blur directionality, the illumination surface index reflects the spatial distribution of the light source and vignetting changes, the histogram drift index reflects brightness contrast and saturation changes, and the noise index reflects the acquisition link and sensor status. The indices within the window are robustly aggregated to form a window index vector, and the ratio is compared with the benchmark file to obtain the relative decay rate and standardized difference. Based on the multi-index fusion strategy, the decay is decomposed into three categories: defocusing, motion blur, and optical anomalies, and the sub-item health and overall health are output. The system performs trend detection on the health status sequence using exponentially weighted moving averages or cumulative sums, and generates early warnings or alarms by combining continuous window confirmation and recovery hysteresis mechanisms. Furthermore, it integrates the health status sequence with cumulative operating conditions to construct a health index curve and estimate remaining lifespan. The system binds parameters and results to form bound records, enhances retrieval by combining historical cases, maintenance manuals, and production line events, and uses a large language model to generate semantic diagnostic reports, including the type of decline, evidence chain, possible causes, suggested actions, and verification steps. The system also writes back user actions and review results to form a closed loop.

[0031] like Figure 2 As shown, in a first aspect, this application provides a method for quantitative monitoring of camera and light source imaging in AOI detection, comprising: S100: After the camera and light source are calibrated, multiple reference images are acquired, and a reference archive is constructed based on the multiple reference images.

[0032] This application can be applied to AOI inspection production lines related to panels / semiconductors to construct a benchmark profile for an industrial area array camera and its matching ring light source in the production line. The camera and light source have completed the entire process of calibration operations, including manual focusing, light source intensity calibration, and optical path debugging, to confirm that the imaging state is the clearest benchmark state without blur or optical anomalies.

[0033] Using calibrated cameras and light sources as the target, standard test samples without defects in the production line are selected as the subjects for imaging. Multiple reference images are acquired according to the preset benchmark of the production line with fixed parameters (including fixed exposure time, fixed gain, and fixed light source brightness). Through the process of "image and ID association → ROI area index calculation and statistics → illumination surface template generation → parameter and ID binding", a unique corresponding benchmark file is constructed. This benchmark file serves as a unified benchmark reference for the quantitative comparison of imaging status in subsequent production operations.

[0034] The reference file includes an identifier ID that forms multiple frames of the reference image, as well as reference parameters.

[0035] Specifically, constructing a benchmark archive based on multiple frames of the benchmark images may include the following steps: S101, after the camera and light source are calibrated, multiple frames of reference images are acquired, and the multiple frames of reference images are associated with the identifier ID to determine the associated identifier information of the reference file.

[0036] The identifier ID includes camera ID, light source ID, recipe ID, and ROI identifier.

[0037] Specifically, the baseline acquisition mode of the AOI inspection system (existing system) is activated, and K frames of baseline images are continuously acquired from the standard test sample according to the preset baseline acquisition fixed parameters (K can be 50 frames in this embodiment to meet the statistical validity requirements and adapt to the routine needs of production line baseline establishment). During the acquisition process, it is ensured that there is no positional shift of the camera, light source, and sample being photographed, and no external interference.

[0038] The AOI control system automatically attaches a unique identifier ID to the batch of 50 reference images. In this embodiment, the identifier ID specifically includes: camera ID, light source ID, formula ID (which is a unique identifier for a set of detection parameters preset to adapt to a specific detection object and detection scenario, and this set of parameters can be directly called "detection formula"), and ROI identifier. Each sub-item of the identifier ID corresponds one-to-one with the reference image.

[0039] The 50 reference images with the attached ID are stored frame by frame in the AOI system database. At the same time, a unique timestamp is assigned to each image. Finally, the camera ID + light source ID + formula ID + ROI identifier are determined as the unique associated identification information of the corresponding reference file, ensuring that the matching reference file can be accurately retrieved according to the identification information in subsequent production operations.

[0040] S102, based on the association identification information of the benchmark archive, extract the corresponding ROI region for each frame of benchmark image and calculate the single value of multiple indicators. Statistically analyze the single values ​​of the same type of multiple indicators for multiple frames of the benchmark image under the same association identification information to obtain the median and absolute median difference of each indicator, which are used as benchmark statistics.

[0041] The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator.

[0042] Specifically, this step is carried out based on the association identification information of S101. Each ROI region is processed independently, and finally the baseline statistics (median + absolute median difference) corresponding to multiple indicators under each ROI region are obtained. The baseline statistics are the core components of the baseline parameters.

[0043] ROI region extraction: Based on the ROI identifiers (such as ROI-01~ROI-03) in the associated identification information, three corresponding ROI regions are accurately extracted from each frame of the reference image according to the preset pixel coordinate range of each ROI. During the extraction process, the pixel size and position of the ROI regions are kept consistent with the preset recipe.

[0044] Multi-metric single-value calculation: For each ROI region of each frame of the baseline image, calculate the single-value of the sharpness metric, directionality metric, illumination surface metric, histogram drift metric, and noise metric. Each metric is calculated according to the following core principles: Sharpness index: The sharpness of an image is characterized by the degree of grayscale variation of edges and details in the ROI region. The more obvious the grayscale variation, the higher the sharpness index value. Directional index: The directionality of blur is characterized by capturing the main direction and intensity of pixel grayscale changes in the ROI region of the image. In the baseline state, there is no motion blur, and the value of this directional index is at a stable low level. Illumination surface index: By smoothing the image to eliminate the interference of details and noise, the true illumination distribution base of the ROI region is restored, and feature values ​​that characterize the uniformity of illumination are calculated accordingly. Histogram drift index: Characterizes the gray-level distribution baseline under the baseline state by statistically analyzing the gray-level distribution characteristic parameters of the ROI region; Noise index: The noise level is characterized by capturing the degree of irregular gray-scale fluctuations in the ROI region. The smaller the fluctuation, the lower the noise index value.

[0045] After the calculation was completed, 50 single values ​​of each of the five categories of indicators were obtained for each of the 50 reference images.

[0046] The baseline statistic is obtained by statistically analyzing all single values ​​of the same type of indicator within the same ROI region under the same associated identification information. First, the median M of this type of indicator is obtained. Then, the absolute median difference (MAD) is calculated based on the median. Both are used as the baseline statistic for this indicator. Median M: Reflects the central reference value of this type of indicator under the baseline condition, and characterizes the baseline level of the indicator; The absolute median deviation (MAD) reflects the natural volatility of this type of indicator under baseline conditions. The formula is: MAD = median(|xi|). In the formula M|), xi is the i-th single value of a certain type of indicator, M is the median of that type of indicator, and median() is the median operation. In this embodiment, the median and absolute median difference of the five types of indicators (sharpness, directionality, illumination surface, histogram drift, and noise) are finally obtained for each region from ROI-01 to ROI-03, forming a complete benchmark statistic.

[0047] S103, perform illumination surface processing on the ROI region and generate a reference illumination surface template. Use the reference statistics and the reference illumination surface template together as the components of the reference parameter index vector. Bind the associated identification information to the corresponding reference parameter index vector to obtain the reference file.

[0048] Specifically, this step supplements the core reference parameter, the reference illumination surface template, integrates all reference parameters and binds them with associated identification information to form a complete reference file that can be directly retrieved.

[0049] Reference illumination surface template generation: Illumination surface processing is performed on each ROI region of the reference image extracted in S102. The core principle of the processing is: low-pass smoothing is used to eliminate details and noise interference in the image to restore the real illumination distribution surface of the ROI region; then the illumination distribution surface is normalized to uniformly map the illumination intensity value to the 0~1 range, eliminate the numerical influence of absolute light intensity, and only retain the relative characteristics of illumination distribution.

[0050] For the same ROI region under the same associated identification information, the average value of the normalized illumination distribution surface of the same ROI region in 50 reference images is taken to generate the reference illumination surface template corresponding to the ROI region. The reference illumination surface template is a two-dimensional pixel matrix, which represents the standard illumination distribution characteristics of the ROI region under the reference state. It is the core reference for judging illumination abnormalities and light source aging in subsequent online monitoring.

[0051] Constructing benchmark parameter index vectors: Integrate the benchmark statistics (median of five types of indicators + absolute median difference) of each ROI region obtained in S102 with the benchmark illumination surface template of each ROI region generated in this step, and construct benchmark parameter index vectors according to ROI identifiers. That is, each ROI identifier corresponds to an independent benchmark parameter index vector. The vector contains the benchmark statistics of five types of indicators, namely sharpness, directionality, illumination surface, histogram drift, and noise, under that ROI region, as well as all pixel feature information of the benchmark illumination surface template of that ROI region, to ensure that the vector corresponds one-to-one with the ROI region and that no parameters are omitted.

[0052] Binding and generating benchmark files: The unique associated identification information (camera ID + light source ID + formula ID + ROI identifier) ​​determined in S101 is bound one-to-one with the benchmark parameter index vectors corresponding to each ROI region constructed in this step and stored in the benchmark file database of the AOI detection system to form a complete benchmark file.

[0053] This benchmark file has a unique retrieval identifier (i.e., associated identifier information). In subsequent production operations, the matching benchmark file can be accurately retrieved through the actual identifiers of production line cameras, light sources, and formulas, providing a unified reference for the quantitative comparison of imaging status.

[0054] S200, during production operation, uses a camera to acquire multiple frames of images to be evaluated within a preset detection window, and extracts fixed ROI regions and adaptive ROI regions from the multiple frames of images to be evaluated. Based on the AOI detection results, interfering regions in the fixed ROI regions and the adaptive ROI regions are removed to obtain a set of pure ROI regions.

[0055] During normal production on the production line, for the same camera with established benchmark files, a single inspection window is constructed according to a preset number of inspections. Multiple frames of images to be evaluated are collected within the window according to the production rhythm. For each frame image, a preset fixed ROI region and an adaptive ROI region selected based on gradient energy are extracted simultaneously to form a single-frame ROI region set. Then, based on the real-time detection results of the AOI inspection system, defects and saturation-type interference regions in the set are identified and removed, and the effective ROI region of the single frame is extracted. Finally, according to the inspection window dimension, the effective ROI regions of all frames within the window are classified and summarized to obtain a pure ROI region set exclusive to the inspection window. This set provides an interference-free image analysis area for subsequent multi-index extraction, ensuring that the index calculation can truly reflect the imaging state of the camera and light source, and eliminating interference from non-equipment degradation factors such as product overexposure / underexposure.

[0056] Specifically, obtaining a set of pure ROI regions may include the following steps: S201: During production operation, a detection window is constructed for the same camera according to a preset number of detections, and multiple frames of images to be evaluated are collected within the detection window.

[0057] Specifically, the detection window construction rules are as follows: Based on the production cycle and imaging status monitoring requirements of the AOI inspection production line, the number of detections N for a single detection window is preset for the camera, which can be N=20 frames (this value can be flexibly configured according to the production line cycle to meet the statistical validity of imaging indicators without affecting normal production). The construction of the detection window is only for the same camera, the same light source, and the same detection formula, to ensure that the imaging environment and detection object of the image to be evaluated within the window are consistent with the baseline file when it is constructed, and to ensure the validity of subsequent indicator comparisons.

[0058] Image acquisition for evaluation: During normal production on the production line, 20 frames of images to be evaluated within the detection window are acquired synchronously with the product flow rhythm. During the acquisition process, the installation position of the camera and light source, the optical path, and the imaging parameters are kept consistent with those in the baseline file construction stage to avoid deviations in imaging indicators caused by parameter changes.

[0059] Image metadata mounting: The AOI control system automatically mounts an identifier ID consistent with the baseline file for each frame of the image to be evaluated. At the same time, it assigns a unique timestamp and window frame number to each frame of the image to be evaluated. All images to be evaluated with mounted metadata are stored in the AOI system database according to the detection window, which facilitates the traceability and processing of subsequent steps.

[0060] S202, for each frame of the image to be evaluated, extract the preset fixed ROI region and the adaptive ROI region selected based on gradient energy to form a set of ROI regions corresponding to each frame of the image.

[0061] Specifically, this step processes each frame of the image to be evaluated independently within the detection window. The location and size of the extracted ROI region are consistent with the preset detection formula, ensuring that the ROI regions between different frames and different windows are comparable, and finally forming a set of ROI regions exclusive to each frame image.

[0062] Fixed ROI region extraction: Based on the core detection region pixel coordinate range preset by the recipe ID, three fixed ROI regions, such as ROI-01, ROI-02, and ROI-03, are accurately extracted from each frame of the image to be evaluated. During the extraction process, the pixel size and position of each fixed ROI are kept consistent with the baseline file construction stage. The fixed ROI region is the core area of ​​interest for AOI detection and is the basic analysis area for imaging status monitoring.

[0063] Adaptive ROI selection: Based on the gradient energy principle, adaptive ROI regions are selected from each frame of the image to be evaluated. Gradient energy reflects the degree of grayscale change in an image region. The more significant the grayscale change, the higher the gradient energy, and the richer the details of the corresponding image. This is a key supplementary region for AOI defect detection. The selection rules are as follows: For each frame of the image to be evaluated, the effective detection area other than the fixed ROI region is divided into sub-regions, and the pixel size of the sub-regions is consistent with that of the fixed ROI region. Calculate the gradient energy of each sub-region, and select the top two sub-regions with the highest gradient energy as the adaptive ROI regions of this frame (such as AD-01 and AD-02) to ensure that the adaptive ROI regions are the regions with the richest details in the image, thus supplementing the analysis limitations of fixed ROI regions. Formation of a single-frame ROI region set: The three fixed ROI regions and two adaptive ROI regions extracted from each frame of the image to be evaluated are integrated to form a unique ROI region set for that frame, such as a single-frame ROI region set = {ROI-01, ROI-02, ROI-03, AD-01, AD-02}. Each region is an independent image analysis unit, and interference regions will be removed from each unit separately.

[0064] S203, based on the AOI detection results, obtain the defect region mask and saturation region mask of the image to be evaluated, and determine the defect region and saturation region as interference regions.

[0065] Specifically, this step relies on the real-time defect detection and image quality analysis results of the AOI inspection system to generate a binarized region mask, identify the interfering regions that need to be removed, and ensure that subsequent index calculations only target normal image regions without interference, excluding the influence of non-equipment degradation factors such as product defects and imaging saturation. The mask is a binarized pixel matrix in industrial vision inspection, where regions with a pixel value of 1 represent target regions and regions with a pixel value of 0 represent normal regions.

[0066] Defect region mask acquisition: Each frame of the image to be evaluated is input into the AOI detection system for real-time defect detection. The AOI detection system identifies the pixel areas in the image that contain product defects according to the preset defect judgment rules and generates a defect region mask. The area with a pixel value of 1 in the mask is the product defect area. The grayscale change in this area is caused by the product itself and is unrelated to the imaging state of the camera and light source. It needs to be removed as an interference area.

[0067] Saturation region mask acquisition: The AOI detection system simultaneously analyzes the pixel grayscale values ​​of each frame of the image to be evaluated, identifies the pixel regions in the image where the grayscale value reaches the extreme value (including the upper limit saturation of grayscale value caused by overexposure and the lower limit saturation of grayscale value caused by underexposure), and generates a saturation region mask. The region with a pixel value of 1 in this mask is the imaging saturation region. The imaging abnormality in this region is caused by temporary changes in illumination, product reflection, etc., and is not a long-term degradation problem of camera or light source. It needs to be removed as an interference region.

[0068] Interference region identification: The defect region mask and the saturation region mask of each frame of the image to be evaluated are merged. All pixel regions after merging are the interference regions of the image of that frame. The identification and removal of interference regions only applies to the ROI region set and does not involve other non-analyzed regions of the image.

[0069] S204, remove the interfering regions from the ROI region set corresponding to each frame of the image to be evaluated, extract the effective ROI regions of each frame of the image to be evaluated, and aggregate and summarize the effective ROI regions of multiple frames of the image to be evaluated according to the detection window dimension to obtain the pure ROI region set.

[0070] Specifically, this step first performs pixel-level interference region removal on the single-frame ROI region set to extract the effective ROI region of the single frame, and then classifies and aggregates them according to the detection window dimension and ROI type to obtain the pure ROI region set of the detection window. This ensures that subsequent multi-index extraction can be uniformly calculated according to ROI type, which is in line with the overall evaluation logic of the detection window.

[0071] Single-frame effective ROI region extraction: For each independent ROI region (such as ROI-01~03, AD-01~02) in the ROI region set of the image to be evaluated in each frame, pixel-level matching is performed with the defect region mask and saturation region mask of that frame. The part of each ROI region that overlaps with the interference region with a pixel value of 1 in the mask is removed, and the normal image part with a pixel value of 0 is retained. The retained part is the effective ROI region of the corresponding ROI region in that frame. If the proportion of interference region in a certain ROI region exceeds 50%, the ROI region is directly discarded to ensure the effectiveness of the analysis of effective ROI regions.

[0072] Detection window dimension aggregation: For the 20 frames of images to be evaluated within this detection window, all single-frame effective ROI regions are classified and aggregated according to ROI type. For example, the effective parts of ROI-01, ROI-02, ROI-03, AD-01, and AD-02 in the 20 frames are aggregated respectively to form 5 independent groups of effective ROI regions. Pure ROI region set generation: The five groups of valid ROI regions under the detection window are integrated to obtain a pure ROI region set specific to the detection window, such as pure ROI region set = {ROI-01 valid group, ROI-02 valid group, ROI-03 valid group, AD-01 valid group, AD-02 valid group}. Each group contains the corresponding valid ROI regions of multiple frames of images within the detection window. This pure ROI region set is the analysis object for subsequent multi-index extraction, and all index calculations are based on this pure ROI region set.

[0073] S300, perform multi-index extraction on the set of pure ROI regions to obtain multiple index vectors.

[0074] The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator.

[0075] Based on the established set of clean ROI regions specific to the detection window, for each effective ROI region in the 20 frames of images to be evaluated within the detection window, the values ​​of five core indicators—sharpness, directionality, illumination surface, histogram drift, and noise—are calculated. First, the values ​​of similar indicators for all effective ROI regions in a single frame are normalized and integrated to form a single-frame indicator vector for each frame of the image to be evaluated. Finally, all single-frame indicator vectors from the 20 frames within the detection window are combined to obtain multiple indicator vectors for that detection window. These extracted indicator vectors provide core data support for subsequent robust aggregation and benchmark comparison. All indicator calculations are based on interference-free effective ROI regions, ensuring that the values ​​accurately reflect the actual imaging state of the camera and light source.

[0076] Specifically, obtaining multiple index vectors may include the following steps: S301, for the effective ROI region corresponding to each frame of the image to be evaluated in the pure ROI region set, a sharpness index based on gradient energy is formed by calculating the sum of squared gradient magnitudes, and a sharpness index based on second-order difference energy is formed by calculating the variance or mean energy of the Laplacian response. The sharpness index based on gradient energy and the sharpness index based on second-order difference energy are combined to obtain the sharpness index value of the effective ROI region of each frame of the image to be evaluated.

[0077] Specifically, the sharpness index is used to characterize the richness of edges and details in the image. The higher the index value, the sharper the image. This step calculates the sharpness index using two methods: gradient energy and second-order difference energy. The two methods are then combined to obtain the final sharpness index value. The two calculation methods complement each other, improving the index's sensitivity to image blur. Furthermore, the index is calculated independently for each effective ROI region of each frame of the image to be evaluated.

[0078] The sharpness index is calculated based on gradient energy. The core principle is that gradient energy reflects the degree of change in pixel grayscale within the effective ROI region. The more significant the grayscale change (richer edges and details), the greater the gradient energy, and the higher the corresponding sharpness index value. This index is calculated using the sum of squared gradient magnitudes, as shown in the formula: ; In the formula: The clarity index value based on gradient energy for the effective ROI region. This is the current valid ROI region to be calculated; The coordinates of the pixels within the effective ROI region; For pixels The horizontal gradient magnitude, For pixels The vertical gradient magnitude.

[0079] The sharpness index is calculated based on second-order difference energy. Second-order difference energy reflects the detailed texture changes in the effective ROI region of an image through the Laplacian operator. The richer the texture, the greater the fluctuation of the Laplacian response, and the higher the mean energy, the higher the corresponding sharpness index value. This embodiment uses the variance of the Laplacian response to calculate this index. The core principle is to perform a second-order difference operation on the region using the Laplacian operator, and then calculate the variance of the operation result. The variance value is the sharpness index value L based on the second-order difference energy.

[0080] The final sharpness index value is obtained by merging the two types of sharpness indices. Since the two types of sharpness indices have the same dimensions, the arithmetic mean of the sharpness index value G based on gradient energy and the sharpness index value L based on second-order difference energy is calculated to obtain the final sharpness index value C for each effective ROI region in each frame of the image to be evaluated. The formula is as follows: .

[0081] After the calculation is completed, the sharpness index value corresponding to each effective ROI region in the 20 frames of images is obtained. The sharpness index value decreases as the image defocusing and blurriness increase.

[0082] S302, for each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is calculated using the structure tensor or gradient direction energy distribution to obtain the directional index value of the effective ROI region of each frame of the image to be evaluated, which is used to characterize the directionality of motion blur.

[0083] Specifically, the directionality index is used to characterize the directionality and intensity of motion blur in the image. The higher the index value, the more significant the main direction of motion blur and the higher the degree of blur. When there is no motion blur, the index value is at a stable low level. In this embodiment, the structural tensor method is used to calculate the index, and it is calculated independently for each effective ROI region of each frame of the image to be evaluated.

[0084] The core calculation principle in this embodiment is to use the structure tensor method to calculate the directionality index. The core principle is to capture the main direction and intensity of grayscale changes in the region by calculating the gradient matrix of pixels within the effective ROI region. When motion blur exists, the pixel grayscale changes will be concentrated in a specific direction, and the structure tensor will output the directional intensity value of the main direction after calculation. When there is no motion blur, the grayscale changes have no obvious main direction, and the directional intensity value is close to 0.

[0085] The final directional index value is determined by using the directional intensity value calculated from the structure tensor as the directional index value D for the effective ROI region. The core judgment rule of this step is: when the sharpness index value C of an effective ROI region shows a decreasing trend, and the directional index value D of the effective ROI region shows an increasing trend, it is determined that the probability of motion blur in the image corresponding to the effective ROI region has increased.

[0086] S303, for the effective ROI region corresponding to each frame of the image to be evaluated in the pure ROI region set, the illumination surface is obtained by low-pass filtering or surface fitting and normalized, and then the similarity, vignetting index and uniformity error are calculated with the reference illumination surface template to obtain the illumination surface index value of the effective ROI region of each frame of the image to be evaluated.

[0087] Specifically, the illumination surface index is used to characterize the illumination distribution state of the image and is a core indicator for determining light source aging and optical anomalies. The lower the index value, the more the illumination distribution deviates from the baseline state, and the higher the risk of optical anomalies. This step first restores and normalizes the true illumination surface of the effective ROI region, then calculates three types of feature parameters with the corresponding baseline illumination surface template in the baseline file, and finally fuses them to obtain the illumination surface index value. This is calculated independently for each effective ROI region of each frame of the image to be evaluated, and the ROI identifier is matched one-to-one with the baseline illumination surface template (e.g., the effective group of ROI-01 matches the baseline illumination surface template of ROI-01 in the baseline file).

[0088] For example, the illumination surface of the effective ROI region is extracted and normalized, and then low-pass filtering is performed on the effective ROI region (to remove image details and noise interference while retaining the illumination distribution base) to restore the true illumination distribution surface of the region, i.e., the original illumination surface. Then, the original illumination surface is normalized to uniformly map the illumination intensity value to the range of 0 to 1, eliminating the numerical influence of absolute light intensity and retaining only the relative characteristics of the illumination distribution to obtain the normalized illumination surface.

[0089] The normalized illumination surface is compared pixel-level with the reference illumination surface template to calculate three types of feature parameters: similarity, vignetting index, and uniformity error. Similarity: Characterizes the degree of fit between the current illuminated surface and the reference illuminated surface. The value ranges from 0 to 1. The closer the value is to 1, the closer the illumination distribution is to the reference. Vignetting index: Characterizes the degree of light intensity attenuation at the edge of the current illuminated surface. The larger the value, the more severe the vignetting at the image edge, and the more obvious the light source path or lens contamination. Uniformity error: Characterizes the uniformity of light intensity distribution on the current illuminated surface. The larger the value, the more uneven the light distribution, and the more significant the aging of the light source or the abnormal light intensity distribution.

[0090] The final illumination surface index value is obtained by fusion. After normalizing and calibrating the three types of feature parameters, a weighted fusion is performed with a similarity weight of 0.5, a vignetting index weight of 0.25, and a uniformity error weight of 0.25 to obtain the final illumination surface index value I. The fusion rule is: the higher the similarity and the smaller the vignetting index and uniformity error, the higher the illumination surface index value I.

[0091] S304, for the effective ROI region corresponding to each frame of the image to be evaluated in the pure ROI region set, the histogram drift index value of the effective ROI region of each frame of the image to be evaluated is obtained by calculating the histogram distance based on grayscale or brightness quantile drift, saturation pixel ratio change and chi-square distance or Earth movement distance.

[0092] Specifically, the histogram drift index is used to characterize the degree of deviation of the gray-level distribution of the image from the baseline state. The higher the index value, the more serious the gray-level distribution drift, which is prone to problems such as decreased image contrast, overexposure / underexposure. This step obtains the index value by fusing three types of parameters: gray-level quantile drift, saturation pixel ratio change, and histogram distance. The index is calculated independently for each effective ROI region of each frame of the image to be evaluated.

[0093] Calculate the grayscale quantile drift and statistically analyze the grayscale distribution characteristics of the effective ROI region. Three key quantiles, namely 10%, 50%, and 90% grayscale values, can be selected. Calculate the sum of the deviations between the current quantile and the grayscale benchmark quantile of the same ROI in the benchmark file. The smaller the deviation, the closer the grayscale distribution is to the benchmark.

[0094] Calculate the change in saturated pixel ratio, count the percentage of saturated pixels in the effective ROI region that have reached extreme gray values, and calculate the absolute difference between the current saturated pixel ratio and the baseline saturated pixel ratio. The smaller the difference, the more stable the saturation state of the image.

[0095] Calculate the histogram distance, which measures the overall difference between the current grayscale histogram and the reference grayscale histogram. This embodiment uses chi-square distance for calculation, and the formula is as follows: ; In the formula: The chi-square distance value. Number of gray levels The th in the grayscale histogram of the current effective ROI region Number of pixels in each grayscale level The first gray level in the baseline grayscale histogram The number of pixels for each gray level; the smaller the chi-square distance value, the more consistent the current gray level distribution is with the baseline.

[0096] After normalizing the three types of parameters mentioned above, the final histogram drift index value H can be obtained by fusing them with equal weights (e.g., 1 / 3 each). The fusion rule can be as follows: after normalizing the gray-level quantile deviation value, the absolute difference of the saturated pixel ratio, and the chi-square distance value, they can be fused with equal weights. The smaller the gray-level quantile deviation value, the absolute difference of the saturated pixel ratio, and the chi-square distance value, the higher the fused histogram drift index value H.

[0097] S305, for the effective ROI region corresponding to each frame of the image to be evaluated in the pure ROI region set, the noise index value of the effective ROI region of each frame of the image to be evaluated is obtained by using at least one of high-frequency residual energy, row and column fringe intensity and fixed pattern noise index.

[0098] Specifically, noise metrics are used to characterize the noise level of the image and are a core indicator for identifying camera sensor aging and acquisition link anomalies. Higher noise metric values ​​indicate greater noise and poorer image quality. In this embodiment, three types of metrics—high-frequency residual energy, row and column fringe intensity, and fixed-pattern noise—are fused together for calculation, and the calculation is performed independently for each effective ROI region of each frame of the image to be evaluated.

[0099] Calculate the high-frequency residual energy, perform low-frequency filtering on the effective ROI region, and extract the filtered high-frequency residual signal. This high-frequency residual signal reflects the irregular gray-scale fluctuations (i.e., random noise) within the region. Calculate the energy value of the high-frequency residual signal. The larger the energy value, the more obvious the random noise.

[0100] Calculate the intensity of the row and column stripes, count the horizontal and vertical stripe grayscale fluctuations in the effective ROI region, calculate the brightness contrast and distribution density of the stripes, and the product of the two is the intensity value of the row and column stripes. The larger the value, the more serious the row and column stripe noise in the image and the more obvious the circuit interference of the acquisition link.

[0101] Calculate fixed-pattern noise, which is the inherent noise of the camera sensor. It manifests as recurring grayscale spots in the image. Calculate the pixel percentage and brightness deviation of such spots in the effective ROI region. The product of the two is the fixed-pattern noise value. The larger the value, the more severe the sensor aging.

[0102] The final noise index value is obtained by fusion. After normalizing the three types of noise parameters, the average value can be taken by equal weight (e.g., 1 / 3 each) to obtain the final noise index value N. The fusion rule is: the smaller the high-frequency residual energy, row and column fringe intensity and fixed pattern noise value, the higher the noise index value N.

[0103] S306, the sharpness index value, the directionality index value, the illumination surface index value, the histogram drift index value and the noise index value are aggregated to form a single-frame index vector corresponding to each frame of the image to be evaluated. Then, the single-frame index vectors of multiple frames of the image to be evaluated in the clean ROI region set are combined to obtain multiple index vectors.

[0104] Specifically, this step normalizes and integrates the five types of index values ​​obtained from the previous calculation, first forming a single-frame index vector for each frame of the image to be evaluated, and then combining them to obtain multiple index vectors for the detection window, ensuring that the vector dimensions are consistent and the data can be used for subsequent robust aggregation, thus solving the problem of ambiguous detailed descriptions.

[0105] The index values ​​of a single frame of the image to be evaluated are normalized and integrated. Each frame of the image to be evaluated contains five valid ROI regions, namely ROI-01~03 and AD-01~02, and the five index values ​​of sharpness, directionality, illumination surface, histogram drift and noise have been calculated for each region. The arithmetic mean of the index values ​​of the same type for each frame of the image to be evaluated is calculated to obtain the global five index values ​​of the frame of the image to be evaluated, namely the average sharpness, average directionality, average illumination surface, average histogram drift and average noise of the single frame, to ensure that only one set of unified five index values ​​is output for each frame of the image to be evaluated.

[0106] Following a fixed order of sharpness, directionality, illumination, histogram drift, and noise, the global values ​​of the five categories of indicators for each frame of the image to be evaluated are aggregated to form a single-frame indicator vector corresponding to that frame of the image to be evaluated. The vector is a 5-dimensional numerical vector.

[0107] Multiple index vectors are obtained by combining the 20 single-frame index vectors corresponding to the 20 frames of images to be evaluated within the detection window. These 20 index vectors are combined in order from frame number 1 to 20 to obtain multiple index vectors for the detection window. These multiple index vectors are the only operation objects for robust aggregation in subsequent windows. The values ​​of the same type of all vectors will be statistically aggregated according to index type.

[0108] S400, robustly aggregate multiple indicator vectors to form a window indicator vector, compare it with the benchmark file to obtain the relative decay rate and standardized difference index describing image degradation.

[0109] This process involves robustly aggregating multiple indicator vectors within a detection window, categorized into five types: sharpness, directionality, illumination, histogram drift, and noise. This yields the windowed aggregation value and fluctuation of each indicator, which are then combined into a windowed indicator vector. Based on the associated identifier ID of the windowed indicator vector, matching baseline statistics for each indicator are retrieved from the baseline database, completing a one-to-one type matching between the windowed indicator vector and the baseline statistics. Relative degradation rates are calculated for both imaging performance and imaging degradation indicators, and standardized differences are calculated for each indicator based on the fluctuation characteristics of the baseline statistics. Finally, the relative degradation rates and standardized differences of each indicator are combined according to their type to obtain the relative degradation rate and standardized difference indicators describing imaging degradation. The indicators output in this step provide core data support for subsequent imaging degradation type decomposition and health calculation. The robust aggregation method effectively reduces the impact of occasional production line disturbances and minor image noise on the evaluation results, ensuring the accuracy and stability of the comparison results.

[0110] Specifically, obtaining the relative decay rate and standardized difference index describing image degradation may include the following steps: S401, for multiple indicator vectors within the preset detection window, the median or truncated mean is used to calculate the window aggregate value of each indicator according to the indicator type. Then, the absolute median difference is used to characterize the fluctuation of each indicator after aggregation. The window aggregate value of multiple indicators and the corresponding fluctuation are combined one by one according to the indicator type to obtain the window indicator vector.

[0111] Specifically, in this step, the median (a common choice for robust aggregation in industrial AOI inspection) is used to calculate the window aggregation value of each indicator. The absolute median difference is used to characterize the fluctuation of each indicator. After combining them one by one according to the indicator type, a window indicator vector is formed. A unified aggregation rule is applied to the five types of indicators to ensure the consistency of the results.

[0112] Index type splitting: The 20 single-frame index vectors in the detection window are split into five index types: sharpness index, directionality index, illumination index, histogram drift index, and noise index. This results in 20 single-frame index values ​​for each of the five index types, namely, sharpness index value set, directionality index value set, illumination index value set, histogram drift index value set, and noise index value set. Each value set contains 20 valid data points.

[0113] Calculate the window aggregation value for each indicator: For the numerical sets of the above five types of indicators, calculate the median for each indicator type, and use this median as the window aggregation value for the corresponding indicator, denoted as X. The method of calculating the median can effectively shield outliers in the dataset and reduce the impact of occasional production line disturbances on the indicator aggregation results.

[0114] Calculate the aggregated volatility of each indicator: For the numerical sets of the above five types of indicators, calculate the absolute median difference for each indicator type, and use this absolute median difference as the aggregated volatility of the corresponding indicator, denoted as . The calculation formula is: ; In the formula: For the first in a certain type of index value set Each frame contains a single-frame metric value, where X is the window aggregation value for that type of metric. This is for median calculation. The absolute median difference is used to quantify the natural fluctuation of a certain type of indicator within the detection window. The smaller the fluctuation, the more stable the imaging state of that indicator within the window.

[0115] A window index vector is formed by combining the window aggregate values ​​of five types of indicators with their corresponding fluctuations, according to the indicator type: sharpness indicator (window aggregate value + fluctuation), directionality indicator (window aggregate value + fluctuation), illumination area indicator (window aggregate value + fluctuation), histogram drift indicator (window aggregate value + fluctuation), and noise indicator (window aggregate value + fluctuation). These are then integrated in a fixed order: sharpness, directionality, illumination area, histogram drift, and noise, resulting in the window index vector for the detection window. This window index vector is a composite vector; each indicator dimension contains two core data points: aggregate value and fluctuation, which can comprehensively characterize the overall imaging indicator features within the detection window.

[0116] S402, based on the identifier ID associated with the window indicator vector, retrieve the benchmark statistics corresponding to each indicator in the matching benchmark file, and then perform a one-to-one matching comparison of the indicator types between the window indicator vector and the benchmark statistics to obtain the one-to-one matching relationship between the multi-indicator window aggregate value and the benchmark statistics.

[0117] Specifically, this step fills in the ambiguities in the details of the basis for retrieving benchmark statistics and the matching rules, clarifies the core components and matching logic of benchmark statistics, and ensures that the comparison between window indicators and benchmark indicators is unique and effective.

[0118] Retrieve matching baseline statistics: Based on the unique identifier ID associated with the window indicator vector (camera ID, light source ID, recipe ID, ROI identifier: ROI-01~03+AD-01~02), accurately retrieve the baseline statistics bound to the same identifier ID from the baseline archive of the AOI detection system. The retrieved baseline statistics include: median and absolute median deviation of the sharpness indicator; median and absolute median deviation of the directionality indicator; median and absolute median deviation of the illumination area indicator; median and absolute median deviation of the histogram drift indicator; and median and absolute median deviation of the noise indicator.

[0119] Complete the one-to-one matching and comparison of indicator types: The aggregated values ​​of each indicator window in the indicator vector are matched one-to-one with the median of the same type of indicator in the retrieved benchmark statistics, according to indicator type. This yields a one-to-one matching relationship between the aggregated values ​​of multiple indicator windows and the benchmark statistics, namely: sharpness window aggregated value and sharpness benchmark median, directionality window aggregated value and directionality benchmark median, illumination window aggregated value and illumination benchmark median, histogram drift window aggregated value and histogram drift benchmark median, and noise window aggregated value and noise benchmark median. This matching relationship provides a unique comparison basis for the subsequent calculation of relative degradation rate and standardized difference value.

[0120] S403, in the one-to-one matching relationship between the multi-index window aggregate value and the benchmark statistic, the relative decay rate of the sharpness index, directionality index, illumination surface index and histogram drift index is obtained by subtracting the ratio of the window aggregate value to the corresponding benchmark statistic from 1.

[0121] Among them, the larger the values ​​of the sharpness index, directionality index, illumination surface index, and histogram drift index, the better the imaging state.

[0122] Specifically, in this step, the sharpness index, directionality index, illumination surface index, and histogram drift index are imaging quality indices. That is, the larger the index value, the better the imaging state. The relative decay rate is calculated by subtracting the ratio of the window aggregation value to the baseline median from 1, and a unified calculation rule is applied to the four types of indices.

[0123] Core calculation principle and formula: The relative decay rate is used to quantify the degree of decay of the current imaging parameters relative to the baseline state. The calculation formula is as follows: ; In the formula: The relative decay rate of a certain type of imaging orientation index. This is the window aggregate value of the indicator. This is the baseline median of the indicator.

[0124] relative decline rate The range of values ​​for is ( ∞,1], where This indicates that the current indicator is consistent with the baseline state, and there is no decline. This indicates that the current indicator is below the baseline, suggesting a decline, and The higher the value, the more severe the decline; This indicates that the current indicators are better than the baseline, with no degradation and superior imaging quality.

[0125] Using the above formula, the relative decay rates of the sharpness index, directionality index, illumination surface index, and histogram drift index are calculated respectively, resulting in a set of relative decay rate results for the four types of imaging orientation indices. S404: In the one-to-one matching relationship between the aggregated value of a multi-indicator window and the benchmark statistic, the relative decay rate of the noise index is obtained by subtracting 1 from the ratio of the aggregated value of the window to the corresponding benchmark statistic.

[0126] The higher the value of the noise index, the worse the imaging condition.

[0127] Specifically, in this step, the noise index is an imaging degradation index, that is, the larger the index value, the worse the imaging state. The relative degradation rate is calculated by "the ratio of the window aggregation value to the baseline median minus 1". This calculation method can accurately quantify the degree of degradation of the noise index relative to the baseline state.

[0128] Core calculation principle and formula: The relative degradation rate of noise indicators is essentially the degradation rate of the indicator, and the calculation formula is as follows: ; In the formula: This represents the relative degradation rate (deterioration rate) of the noise index. This is the windowed aggregate value of the noise index. This represents the baseline median of the noise index.

[0129] Relative degradation rate of noise index The range of values ​​for is ( 1, +∞), where This indicates that the current noise performance is consistent with the baseline state, with no degradation. This indicates that the current noise level is higher than the baseline, indicating degradation, and The higher the value, the higher the noise level and the worse the imaging quality; This indicates that the current noise level is lower than the baseline, with no degradation and lower imaging noise.

[0130] S405, based on the window aggregate value of multiple indicators, the corresponding benchmark statistics, and the absolute median difference in the benchmark statistics, calculates the standardized difference value of multiple indicators separately according to the indicator type.

[0131] Specifically, in this step, the standardized difference value is used to normalize the deviation between the window aggregate value and the benchmark median to the fluctuation level of the benchmark state, eliminate the dimensional differences of different indicators, realize the quantitative comparison of the deviation of each indicator, and the calculation results provide standardized data for subsequent multi-indicator fusion decomposition imaging decay type.

[0132] Based on the window aggregate value of multiple indicators, the median and absolute median difference of the corresponding benchmark statistics, the standardized difference value is calculated separately according to the indicator type. The calculation formula is as follows: ; In the formula: For a certain type of indicator, the standardized difference value, This is the window aggregate value of the indicator. This is the baseline median of the indicator. This is the baseline absolute median deviation of the indicator.

[0133] Standardized difference value It is a dimensionless statistic with no fixed range of values, among which This indicates that the current window's aggregated value is exactly the same as the benchmark median, with no deviation. The larger the value, the more significant the deviation of the current indicator from the baseline state. The deviation has been calibrated in conjunction with the natural fluctuations of the baseline state, and the deviation can be compared across indicators.

[0134] Using the formulas described above, the standardized difference values ​​for the sharpness index, directionality index, illumination index, histogram drift index, and noise index are calculated respectively, resulting in a set of standardized difference values ​​for the five categories of indices. Each category of indices corresponds to a unique standardized difference value.

[0135] S406, based on the one-to-one correspondence of the relative decay rates of the sharpness index, directionality index, illumination surface index and histogram drift index, the relative decay rate of the noise index and the standardized difference values ​​of multiple indices, the relative decay rate and standardized difference index describing the image degradation are obtained.

[0136] Specifically, following the core principle of "one-to-one correspondence between indicator types," the relative degradation rate and standardized difference value are integrated into the final indicator describing imaging degradation, ensuring that the indicator structure is clear and the dimensions are consistent, and can be directly used for subsequent decomposition of imaging degradation types.

[0137] Decline-difference pairs for single indicators: The relative decline rates of the five types of indicators are bound to their corresponding standardized difference values ​​according to indicator type, forming five sets of decline-difference feature pairs for single indicators. Each feature pair can fully characterize the degree of decline and deviation of a certain type of indicator relative to the benchmark state.

[0138] The final index is formed by integrating the five sets of single indexes in a fixed order: sharpness index, directionality index, illumination surface index, histogram drift index, and noise index. The decay and difference characteristics of these five single indexes are then integrated to obtain the relative decay rate and standardized difference index of the imaging decay of the detection window.

[0139] This indicator is a composite feature set, containing 10 core data points including the relative degradation rate and standardized difference value of five types of indicators. It is the only data basis for subsequent multi-indicator fusion decomposition of three types of imaging degradation: defocusing, motion blur, and optical anomalies. All data have undergone robust aggregation and standardized calibration, and can accurately and objectively reflect the actual imaging degradation status of the camera and light source.

[0140] S500, based on the relative degradation rate and standardized difference index, decomposes the imaging degradation into defocusing degradation index, motion blur degradation index and optical anomaly degradation index, and obtains the sub-item health and total health of the imaging degradation.

[0141] Based on the relative degradation rates and standardized difference indicators of each indicator obtained within the detection window, a multi-indicator fusion strategy involving single-indicator merging, dual-indicator fusion, and triple-indicator fusion is used to decompose and obtain defocus, motion blur, and optical anomaly degradation indicators. All three types of degradation indicators are quantitative values ​​in the range of 0 to 1, representing the actual severity of the corresponding imaging degradation type. Then, based on the production line's preset quantitative evaluation rules, the three types of degradation indicators are converted into imaging sub-items of health within the same range, corresponding one-to-one with the three imaging degradation types: defocus, motion blur, and optical anomaly. Finally, according to the preset health weight allocation rules, the three types of imaging sub-item health are comprehensively weighted and fused to obtain the total health score in the range of 0 to 1. The sub-item health scores and total health scores output in this step provide core data support for subsequent lifespan quantification, trend detection, and alarms.

[0142] Specifically, obtaining the sub-item health and overall health of the imaging degradation may include the following steps: S501, the relative degradation rate of the sharpness index and the standardized difference value are merged and quantified to obtain the out-of-focus degradation index.

[0143] The defocusing degradation index is used to characterize the degree of defocusing in imaging.

[0144] Specifically, this step merges and quantifies the relative degradation rate and standardized difference value of the sharpness index to obtain the defocusing and out-of-focus degradation index.

[0145] The core principle of merged quantization is that the relative degradation rate of the sharpness index directly reflects the actual degree of degradation of the sharpness index relative to the baseline state, while the standardized difference value reflects the degree of deviation of the sharpness index relative to the baseline state, and this deviation has been calibrated in combination with the natural fluctuations of the baseline state. The core of merged quantization is to integrate the numerical characteristics of the two dimensions to avoid the one-sidedness of a single numerical evaluation, while accurately representing the true degradation state of out-of-focus.

[0146] The specific implementation method of merge quantization is as follows: The relative decay rate and standardized difference value of the sharpness index are normalized to the range of 0 to 1. This maps two indices with different dimensions and different numerical ranges to the same interval of 0 to 1, eliminating the differences in numerical characteristics and ensuring the rationality of the merging. The normalized sharpness index relative degradation rate and standardized difference value are fused with equal weights to obtain the out-of-focus degradation index, which has a value range of 0 to 1.

[0147] The higher the value of the defocusing degradation index, the more severe the defocusing degradation of the image, and the more obvious the loss of high-frequency features such as the edges and details of the image; when the value is 0, it means that the sharpness index is completely consistent with the baseline state, with no defocusing degradation.

[0148] S502, the relative decay rate of the sharpness index and the directionality index is fused and quantified with the standardized difference value, and the representation weight of the fused quantization is increased when the sharpness index decays and the directionality index is enhanced, so as to obtain the motion blur class degradation index.

[0149] The motion blur degradation index is used to characterize the degree of decay of imaging motion blur.

[0150] Specifically, in this step, the sharpness index decreases while the directionality index increases. The core function of the directionality index is to characterize the directionality of motion blur. Therefore, the relative decay rate and standardized difference value of the sharpness index and the directionality index are fused and quantified. When the above core judgment conditions are met, the representation weight of the fusion quantization is increased to obtain a motion blur-type degradation index. This motion blur-type degradation index characterizes the degree of decay and probability of occurrence of imaging motion blur.

[0151] Sharpness index degradation: The relative degradation rate of the sharpness index is greater than 0, that is, the window aggregation value of the sharpness index is lower than the baseline median of the baseline state. Enhanced directional indicators: The window aggregation value of the directional indicator is higher than the baseline median of the baseline state, that is, the relative decay rate of the directional indicator is less than 0, which means that the main directional features of grayscale changes in the image are more significant.

[0152] The core principle of fusion quantization is to simultaneously integrate the relative degradation rate and standardized difference value of two types of indicators, sharpness and directionality, to strengthen the feature contribution of the core judgment condition of motion blur, so that the quantification results of motion blur degradation indicators are more in line with the actual imaging state.

[0153] Specific implementation methods of fusion quantification: The four values ​​of the sharpness index (relative decay rate, standardized difference value) and the directionality index (relative decay rate, standardized difference value) are normalized to a range of 0 to 1 to eliminate the difference between the dimensions and the numerical range. The four normalized values ​​are initially merged according to basic equal weights to obtain the basic merged value; If the core judgment condition of "deterioration of clarity index and enhancement of directional index" is met, the representation weight of the two types of index values ​​in the fusion is increased, and the contribution of motion blur features is strengthened; if the judgment condition is not met, the basic equal weight fusion is maintained, and the motion blur degradation index is finally obtained, which has a value range of 0 to 1.

[0154] The higher the value of the motion blur degradation index, the more severe the motion blur degradation of the image and the more obvious the directional blur characteristics of the image; when the value is 0, it means that there are no motion blur-related features in the image and the image has no motion blur degradation.

[0155] S503, the relative decay rate of the illumination surface index, the histogram drift index, and the noise index is fused and quantified with the standardized difference value, and the illumination surface index, the histogram drift index, and the noise index respectively correspond to the decay state of light source aging, abnormal light intensity distribution, and abnormal acquisition link sensor, to obtain the optical anomaly type degradation index.

[0156] The optical anomaly degradation index is used to characterize the overall degree of degradation of imaging optical anomalies.

[0157] Specifically, this step fuses and quantifies the relative decay rate and standardized difference value of the three types of indicators: the illumination surface index, the histogram drift index, and the noise index, to obtain an optical anomaly degradation index, which comprehensively characterizes the overall degree of degradation of imaging optical anomalies.

[0158] The core principle of fusion quantification is that the three indicators of illumination surface, histogram drift, and noise reflect the optical anomalies of the AOI detection system from three dimensions: the operating status of the light source, the distribution of imaging light intensity, and the status of the acquisition link and sensor. By fusing the relative decay rate and standardized difference value of the three indicators, it is possible to achieve comprehensive quantification of optical anomalies from all dimensions, avoiding the limitations of evaluating a single optical indicator.

[0159] Specific implementation methods of fusion quantification: The relative decay rate and standardized difference value of each of the six values ​​of the illumination surface index, histogram drift index, and noise index are normalized from 0 to 1 to eliminate the differences in the dimensions and numerical ranges between different indices and ensure the rationality of the fusion. The six normalized values ​​are fused with equal weights to obtain an optical anomaly degradation index, which has a value range of 0 to 1. During the fusion process, the independent characteristic contributions of the three types of indicators are maintained, and the abnormal characterization of any optical dimension is not weakened, so as to ensure that the quantification results can fully reflect the overall state of the optical system.

[0160] The higher the value of the optical anomaly degradation index, the more severe the overall degradation of the optical anomaly in the AOI detection system, and the more prominent the characteristics of problems such as light source aging, abnormal light intensity distribution, and abnormal sensor in the acquisition link. When the value is 0, it means that all indicators of the optical system are completely consistent with the baseline state, and there are no optical anomalies.

[0161] S504, based on preset quantitative evaluation rules, the defocusing degradation index, the motion blur degradation index, and the optical anomaly degradation index are weighted and calculated respectively to obtain the imaging sub-item health of the defocusing degradation index, the motion blur degradation index, and the optical anomaly degradation index.

[0162] Specifically, the preset quantitative assessment rules are as follows: Based on the general standards for assessing the health of industrial equipment, the preset quantitative assessment rules show that the degradation index and the sub-item health are negatively correlated. That is, the degradation index based on the 0~1 interval is directly converted into the imaging sub-item health of the same interval. The conversion process retains the quantitative characteristics of the degradation index and only completes the semantic conversion from "degree of degradation" to "health status".

[0163] The specific calculation of the sub-item health score is as follows: According to the above preset rules, the defocus, motion blur, and optical anomaly degradation indicators are converted into corresponding imaging sub-item health scores, namely, defocus sub-item health score, motion blur sub-item health score, and optical anomaly sub-item health score. The value range of the three sub-item health scores is 0~1.

[0164] Explanation of health status: The closer the health status value of each imaging sub-item is to 1, the better the state of the corresponding imaging degradation type and the higher the degree of fit with the baseline state; the closer the value is to 0, the more severe the degradation of the corresponding imaging degradation type and the greater the deviation from the baseline state; when the value is 1, it means that there is no degradation in the corresponding dimension and it is in the optimal state of the baseline.

[0165] S505 calculates the total health score by comprehensively weighting and fusing the health scores of the imaging sub-items of imaging degradation type according to the preset health score weighting rules.

[0166] The overall health rating represents the overall imaging status of the camera and light source, and the imaging degradation types include defocusing, motion blur, and optical anomalies.

[0167] Specifically, the preset health score weighting rules are designed around the comprehensive weighted integration of the three categories of health scores, and the weighting configuration is in line with the actual needs of panel / semiconductor AOI inspection production lines.

[0168] The preset health score weight allocation rules are clear: the preset health score weight allocation rules can be flexibly configured according to the actual testing needs of the production line. That is, based on the sensitivity of the AOI testing production line to the three types of degradation such as defocus, motion blur and optical abnormalities, different fusion weights are configured for the three types of imaging sub-items of health score; if the production line has no special sensitive needs, equal weights can be used for fusion to adapt to conventional testing scenarios.

[0169] For example, panel / semiconductor AOI inspection is more sensitive to optical anomalies. The health score of optical anomalies can be configured with a weight of 0.4, and the health scores of defocusing, out-of-focus, and motion blur can each be configured with a weight of 0.3, so as to achieve quantitative evaluation of the bonding production line.

[0170] The overall health score is calculated as follows: According to the above-preset health score weighting rules, the health scores of the out-of-focus, motion blur, and optical anomalies are comprehensively weighted and fused to obtain the overall health score. The value of this index ranges from 0 to 1.

[0171] Explanation of Overall Health: Overall health is the unique quantitative representation of the overall imaging status of the camera and light source. The closer the value is to 1, the better the overall imaging status of the camera and light source, the higher the degree of conformity between various indicators and the baseline status, and the no obvious imaging degradation. The closer the value is to 0, the worse the overall imaging status of the camera and light source, and the more severe the comprehensive degree of imaging degradation. When the value is 1, it means that the imaging status of the camera and light source is completely consistent with the baseline optimal state, and there is no degradation problem.

[0172] S600, the sub-item health scores and the total health score are combined over time or over a window sequence to form a health index curve, and the records and semantic interpretations are bound together to obtain an estimated remaining lifespan and a semantic diagnosis and chain of evidence.

[0173] Specifically, obtaining an estimated remaining lifespan and a semantically interpreted diagnostic and evidence chain description may include the following steps: S601, the sub-item health and the total health are aligned according to the time series or detection window sequence, and the cumulative operating conditions are fused to fit and calculate the above health data to obtain the imaging health index curve of the camera and the light source.

[0174] Based on the imaging sub-item health and overall health obtained from the previous steps for each detection window, the health data is precisely aligned according to the detection window sequence. The cumulative operating conditions of the camera and light source are then integrated to fit the health data, resulting in an imaging health index curve that intuitively reflects the degradation pattern. By analyzing the degradation trend and core characteristics of the health index curve, and combining it with the production line's preset imaging maintenance threshold, the degree of imaging degradation is quantitatively assessed, and a remaining lifespan estimate is obtained. The unique identifier ID, curve characteristics, remaining lifespan, health data, and imaging degradation type are linked and integrated to generate a parameter result binding record containing indicator evidence, degradation conclusions, and confidence levels. Finally, this binding record is enhanced with retrieval of historical cases, maintenance manuals, and production line process events. The retrieved information and the binding record are input into a large language model to generate a semantic diagnostic report for engineers. This step achieves a closed loop from health measurement to lifespan prediction and semantic interpretation, providing a quantifiable, traceable, and operable basis for predictive maintenance of production line cameras and light sources.

[0175] The cumulative operating parameters include at least one of the following: cumulative number of shots, cumulative light source illumination duration, and cumulative exposure.

[0176] Specifically, this step ensures dimensional consistency through data alignment, strengthens the real decline trend by integrating cumulative operating conditions, and finally constructs a multi-dimensional imaging health index curve, which can accurately reflect the natural decline of health during operation.

[0177] Health data is aligned according to the detection window sequence. For example, the production line uses a CAM-06+LIG-06-02 detection combination with preset detection windows based on the production cycle. The detection windows are uniquely numbered according to the generation order (window 1, window 2, ..., window n). To ensure data continuity and comparability, the detection window sequence is used as the unified horizontal axis and the health value is used as the vertical axis. The defocusing, motion blur, optical anomaly, and total health data calculated for each detection window are matched and bound one-to-one with the corresponding detection window number. At the same time, the cumulative operating condition quantity at the end of each detection window is associated with the health data of that detection window, realizing the dimensional alignment between health data and cumulative operating condition quantity, and ensuring that the subsequently fitted curve can reflect the correlation between health and actual operating losses.

[0178] Statistics and definition of cumulative operating conditions For the CAM-06+LIG-06-02 detection combination, the cumulative operating condition statistics should be calculated from the start of the combination's operation, including at least three categories: Total number of shots: The total number of frames captured by the camera for the formula, which is a core characteristic of the camera's imaging module; Cumulative light source illumination time: The actual total illumination time of the light source, which is the core loss characterization of the light source module; Cumulative exposure: The sum of the exposure times for each shot taken by the camera, which is a core characteristic of the camera sensor's wear and tear.

[0179] All cumulative operating conditions are automatically counted by the AOI inspection system and updated synchronously with the inspection window.

[0180] The imaging health index curve is fitted by integrating operating conditions. The core principle of the fitting is that the cumulative operating conditions are a direct representation of the actual operating losses of the camera and light source. The natural decline of health is strongly correlated with the increase of operating conditions. During the fitting process, the small fluctuations in health without changes in operating conditions (such as small changes in health caused by temporary disturbances in the production line) are weakened, while the trend of continuous decline in health with the increase of operating conditions is strengthened, so as to ensure that the curve can reflect the true change law of imaging status.

[0181] Based on the aligned health data and cumulative operating conditions, four imaging health index curves are obtained: the defocusing and out-of-focus sub-health index curve, the motion blur sub-health index curve, the optical anomaly sub-health index curve, and the overall health index curve. All curves are plotted with the detection window sequence on the horizontal axis and the health value (0~1) on the vertical axis, which can intuitively show the decay rate, fluctuation and trend of various health values ​​as the window / operating conditions increase.

[0182] S602, perform degradation trend analysis and feature extraction on the imaging health index curve, and combine it with the preset imaging maintenance threshold to achieve a quantitative assessment of the degradation degree of the imaging state of the camera and light source, and obtain the estimated remaining lifespan of the imaging of the camera and light source.

[0183] Specifically, the analysis and feature extraction of the decline trend of the imaging health index curve takes the overall health index curve as the core analysis object (with sub-curves as auxiliary references). Starting from the actual needs of production line equipment health management, the analysis extracts three core decline features of the curve. All features are direct quantitative representations of the degree of imaging decline: Degradation slope: Reflects the rate at which the overall health decreases as the detection window / cumulative operating conditions increase. The larger the absolute value of the slope, the faster the imaging state of the camera and light source degrades, and the more significant the actual operating losses. Fluctuation level: Reflects the range of numerical fluctuations in overall health during the decline process. The smaller the fluctuation level, the more stable the change in health, which can more realistically reflect the natural decline law of cameras and light sources, and is less affected by temporary operating conditions of the production line. Decline inflection point: The detection window / cumulative operating condition node that reflects the transition of overall health from slow decline to rapid decline. This decline inflection point is a key change point in the imaging status, and the imaging status needs to be closely monitored after the decline inflection point appears.

[0184] At the same time, by combining the three types of sub-health index curves, we can analyze the individual trend characteristics of each type of decline and identify the main type of decline (for example, if the absolute value of the decline slope of the optical anomaly sub-health index curve is the largest, it means that the current main type of decline is optical anomaly).

[0185] The preset imaging maintenance threshold is defined in conjunction with the imaging quality requirements of the panel / semiconductor AOI inspection production line. A flexibly configurable imaging maintenance threshold is set for the total health index curve (in this embodiment, the preset threshold is 0.5, and the production line can adjust it according to factors such as defect detection rate requirements and maintenance costs). This threshold is the critical value at which the camera and light source need to be manually maintained: when the total health drops to this threshold, the degradation of the imaging state has affected the normal inspection of the production line, and the camera / light source needs to be adjusted, cleaned or replaced in a timely manner.

[0186] The calculation and characterization of remaining lifetime estimates: The core principle of remaining lifetime estimation is as follows: Based on the degradation characteristics of the extracted total health index curve, combined with the total health value of the current window and the cumulative operating conditions, predict the number of subsequent detection windows or the number of new cumulative operating conditions required when the total health index curve drops to the preset imaging maintenance threshold, and convert it into actual operating indicators that can be directly interpreted by the production line, which is the remaining lifetime estimate.

[0187] In this embodiment, the remaining life estimate can be represented in three ways that fit the production line, and can be selected as needed: 1. Number of remaining inspection windows (e.g., 20 remaining inspection windows); 2. Remaining operating time (e.g., 80 hours of remaining production line operating time); 3. Remaining number of shots (e.g., 50,000 remaining shots). All estimates are based on the actual degradation trend, providing a quantitative basis for predictive maintenance and spare parts planning of the production line.

[0188] S603, the identifier ID, the features of the imaging health index curve, the remaining lifespan estimate, the sub-item health and the total health data, and the imaging degradation type are associated and integrated to generate a parameter result binding record containing indicator evidence, degradation conclusion, and confidence level.

[0189] Specifically, the rules for associating and integrating core information, for example, using the unique identifier ID of CAM-06+LIG-06-02+FOR-04 (camera ID: CAM-06, light source ID: LIG-06-02, formula ID: FOR-04, ROI identifier: ROI-01~03+AD-01~02) as the core index, integrate all the core evaluation information of this detection combination, which are all key data for lifetime estimation and semantic diagnosis.

[0190] The core component of the parameter result binding record is the generation of a unique parameter result binding record for the current detection window. This record includes a window number, a generation timestamp, and at least the following: Basic identification information: Camera ID, Light Source ID, Recipe ID, ROI identifier, Detection window number, Generation timestamp; Curve feature information: The core decline characteristics of the imaging health index curve (decline slope, degree of fluctuation, decline inflection point); Lifetime estimation information: Remaining lifetime estimate (multi-form representation); Raw health data: the health scores of the current window and the last 10 windows, as well as the total health score. Image degradation information: primary degradation type, secondary degradation type (e.g., primary: optical anomaly, secondary: slight defocusing). Indicator Evidence and Conclusions: Specific indicator changes supporting the determination of the degradation type (such as the optical anomaly sub-item health rate decreasing from 0.95 to 0.72 for five consecutive windows, and the relative degradation rate of the noise index reaching 0.25), and the conclusion of imaging status degradation. Confidence level: The confidence level between the decline conclusion and the remaining life estimate (90% in this example, and the value range is 0~100% depending on the stability of the health data and the significance of the decline trend).

[0191] The parameter result binding records have storage attributes. All parameter result binding records are automatically stored in a dedicated database by the AOI detection system. They are classified and retrieved by identifier ID + window number. The records are in an immutable structured format, supporting subsequent historical tracing, case retrieval, and data review. They can also be directly connected to subsequent search enhancement modules and large language models to achieve seamless data flow.

[0192] S604, perform retrieval enhancement on the parameter result binding record using historical cases, maintenance manuals, and production line process events, input the retrieval enhancement information and the parameter result binding record into the large language model, and generate a semantic diagnostic report.

[0193] The semantic diagnostic report includes at least the core cause of imaging degradation, an explanation of the evidence chain of indicators, recommended actions, and verification steps.

[0194] Specifically, the retrieval of parameter result binding records is enhanced. The core principle of retrieval enhancement is to combine the degradation characteristics and conclusions of the current detection combination, retrieve relevant reference information from the database preset by the production line, provide historical experience and production line background support for semantic diagnosis, make the diagnostic conclusions more in line with the actual production line, and make the treatment suggestions more operable.

[0195] Search scope: It can include three databases, all of which are pre-built and updated in real time by the AOI detection system; Historical case library: Past imaging degradation cases with the same camera / light source / formula, including degradation type, index changes, remedial actions, and remedial effects; Maintenance Manual Library: Official maintenance specifications for AOI inspection equipment cameras and light sources, including standard handling procedures, key points of operation, and tool requirements for various imaging problems; Production Line Process Event Database: Recent process adjustments, equipment debugging, environmental changes, etc. (such as light source cleaning, camera focusing, and workshop temperature and humidity adjustment) on the production line, including event time, operation content, and equipment involved.

[0196] Search and filtering rules: The AOI detection system performs precise searches based on the core information (identifier ID, main decline type, indicator evidence) in the parameter result binding records.

[0197] The system calls upon a large language model (existing large language models are not improved upon in this application) to generate a semantic diagnostic report. Structured retrieval enhancement auxiliary information and complete parameter result binding records are input into the large language model. The model generates a semantic diagnostic report according to the production line's preset report specifications. The report is designed for engineering personnel and covers at least four parts: Core causes of image degradation: By combining indicator evidence, retrieved historical cases and production line process events, the specific causes of image degradation are identified (e.g., "the LIG-06-02 ring light source shows an aging trend, and the CAM-06 camera sensor has slight wear, which are the core causes of optical abnormalities; there is a slight drift in the camera's focal plane, resulting in slight defocusing"). Explanation of the indicator evidence chain: Corresponding the causes of degradation to specific indicator changes one by one to form a traceable evidence chain (e.g., "the health of the optical anomaly sub-item decreased by 23% for 5 consecutive windows, corresponding to a relative degradation rate of 0.25 for the noise index, indicating slight wear and tear on the camera sensor; the similarity of the illumination surface index decreased to 0.78, indicating aging of the light source"). Recommended actions: Based on the maintenance manual and historical cases, provide specific and actionable actions, prioritized (e.g., "Level 1 action: Perform light intensity calibration on light source LIG-06-02; Level 2 action: Refocus the focal plane of camera CAM-06; Level 3 action: Clean the camera lens and the light-emitting surface of the light source"). Verification steps: Clarify how to verify whether the imaging status has improved after the treatment is completed (e.g., "After the treatment is completed, collect images from 3 detection windows. If the health score of the optical anomaly sub-item rises to above 0.85 and the health score of the defocus sub-item rises to above 0.90, it indicates that the treatment is effective; if it does not meet the standard, further testing of the camera sensor and light source is required").

[0198] In subsequent applications of the report, after the semantic diagnostic report is generated, it is stored in association with the parameter result binding record and pushed to the production line operation and maintenance terminal. After the engineering personnel perform the disposal action, they write back the review results (such as whether the disposal is effective and the actual adjustment content) to the parameter result binding record. This record will be stored in the historical case library as a new historical case, providing a reference for the diagnosis of similar degradation problems in the future, forming a closed-loop traceable management of detection-diagnosis-disposal-review.

[0199] Secondly, this application provides a camera and light source imaging quantization monitoring system for AOI inspection, applied to the aforementioned camera and light source imaging quantization monitoring method for AOI inspection. The system includes: The data acquisition module is used to acquire multiple frames of reference images after the camera and light source are calibrated, and to construct a reference file based on the multiple frames of reference images. The reference file includes an identifier ID that forms the multiple frames of reference images, and reference parameters. The ROI management module is used to acquire multiple frames of images to be evaluated within a preset detection window using a camera during production operation, extract fixed ROI regions and adaptive ROI regions from the multiple frames of images to be evaluated, and remove interfering regions from the fixed ROI regions and the adaptive ROI regions based on the AOI detection results to obtain a set of clean ROI regions. The indicator calculation module is used to extract multiple indicators from the set of pure ROI regions to obtain multiple indicator vectors. The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. The benchmark and comparison module is used to robustly aggregate multiple index vectors to form a window index vector, which is then compared with the benchmark file to obtain the relative decay rate and standardized difference index describing image degradation. The degradation decomposition and alarm module is used to decompose the imaging degradation into defocusing degradation index, motion blur degradation index and optical anomaly degradation index based on the relative degradation rate and standardized difference index, and obtain the sub-item health and total health of the imaging degradation accordingly. The lifespan assessment module and the binding and semantic interpretation module are used to form a health index curve by combining the sub-items of health and the total health over time or over a window sequence, and to bind the records and semantic interpretation to obtain an estimated remaining lifespan and a semantic diagnosis and chain of evidence.

[0200] Thirdly, this application provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned AOI detection camera and light source imaging quantization monitoring method.

[0201] Fourthly, this application provides a computer program that, when executed by a processor, implements the steps of the aforementioned AOI detection camera and light source imaging quantization monitoring method.

[0202] Example like Figure 1 As shown, this application includes modules such as data acquisition, ROI management, indicator calculation, benchmark comparison, degradation decomposition and alarm, lifetime assessment, binding and semantic interpretation.

[0203] The data acquisition module collects camera images and metadata from the AOI. The metadata includes at least the camera ID, light source ID, recipe ID, exposure, gain, trigger parameters, and beat information, and assigns a timestamp to each frame.

[0204] The ROI management module pre-configures fixed ROIs and selects adaptive ROIs based on gradient energy at runtime. It also uses the defect mask output by AOI detection and the saturation detection mask to eliminate interference areas.

[0205] The index calculation module computes multi-index vectors on the ROI. Sharpness indices include gradient energy and Laplacian energy indices. Directionality indices are obtained from the structure tensor, outputting directional intensity and principal direction. Illumination surface indices are obtained by low-pass filtering or quadratic surface fitting of the ROI, normalizing the illumination surface, and then calculating the similarity to the reference illumination surface template, as well as the vignetting index and uniformity error. Histogram drift indices are composed of quantile drift saturation ratio changes and histogram distances. Noise indices are composed of high-frequency residual energy row, column, and fringe intensity or fixed-pattern noise indices.

[0206] During the baseline profile establishment phase, after manual calibration, K baseline images are acquired for each camera and each formula. The aforementioned indicators are calculated for each ROI, and the median and absolute median difference are saved as baseline statistics. Simultaneously, baseline illumination surface templates and baseline histogram templates are saved. In the online monitoring phase, a window is constructed according to a preset number of detections N. M frames are selected within the window for evaluation, and indicator vectors are extracted for each frame. For the same indicator within the window, median aggregation is used to obtain the window indicator value, and its ratio is calculated with the baseline indicator value. The relative degradation rate is defined as one minus the ratio of the current value to the baseline. For indicators with larger and worse values, the degradation rate is calculated as the ratio of the current value to the baseline minus one. The degradation rates of each indicator are merged into defocus, blur, and optical anomaly sub-items, and a weighted fusion is used to obtain the overall health score. The alarm strategy adopts a threshold-configurable rule, with the threshold set to a ten-percentage-point degradation threshold. Continuous window confirmation and recovery hysteresis are used to avoid jitter.

[0207] During the lifespan assessment phase, the overall health and optical anomaly sub-items are combined with the window sequence to form a health index curve. The trend slope and fluctuation of the health index are calculated, and the remaining lifespan is estimated by combining the cumulative number of shots and the duration of light source illumination. When the health index continues to decline and approaches the preset maintenance threshold, the system generates a maintenance warning.

[0208] During the binding and semantic interpretation phase, the system generates parameter result binding records for each window. The binding records include camera ID, light source ID, formula parameters, environmental parameters, key indicator values, relative decay rate, conclusion confidence level, and alarm level.

[0209] The system enhances the retrieval of historical case databases, maintenance records, and process events. The retrieval results, along with the bound records, are input into a large language model, which outputs semantic diagnostic text. This text includes the main degradation types, evidence chain explanations, a list of possible causes, suggested actions, and verification steps. After the user executes the suggested actions, the system collects a review window and writes the review results back to the bound records for subsequent similar case retrieval and strategy optimization.

[0210] like Figure 1 As shown, Figure 1 For a detailed explanation of each process step, please refer to the preceding content: 1. Image and metadata acquisition: Camera ID, Light source ID, Recipe ID, Timestamp.

[0211] The system acquires camera images from AOI inspections, associates them with unique device identifiers (camera ID, light source ID) and inspection process parameters (recipe ID), and timestamps each image frame to achieve precise binding of images with equipment, process, and time, providing basic metadata for subsequent traceability and analysis.

[0212] 2. ROI Management: Fixed ROI and Adaptive ROI.

[0213] The acquired images are managed by region of interest (ROI). On the one hand, the fixed ROI (core detection region) preset in the detection formula is extracted. On the other hand, the adaptive ROI (region with rich image details) is automatically selected based on gradient energy, which takes into account both the fixedness and flexibility of detection and focuses on the effective analysis area.

[0214] 3. Defects and saturation removal.

[0215] By combining the AOI detection results, product defect areas and imaging saturation areas (overexposure / underexposure) in the image ROI are removed to eliminate interference from non-equipment imaging degradation factors and ensure that subsequent index calculations are only for interference-free normal image areas.

[0216] 4. Multi-indicator extraction: sharpness, directionality, illumination surface, histogram, noise.

[0217] Within the ROI region after removing interference, five core indicators characterizing the imaging state are calculated, which respectively describe the actual imaging quality of the camera and light source from five dimensions: image sharpness, motion blur characteristics, illumination distribution, grayscale distribution variation, and imaging noise level.

[0218] 5. Comparison with Golden + Window Robust Aggregation: Relative Decline Rate, Standardized Differences.

[0219] "Golden" refers to the baseline archive. First, the index values ​​of multiple frames within the preset detection window are robustly aggregated (to reduce the impact of operating condition disturbances) to obtain the window index vector. Then, it is compared with the index baseline values ​​in the baseline archive to calculate the decay rate and standardized difference value of each index relative to the baseline state, thereby quantifying the image decay.

[0220] 6. Degradation and trend detection: out-of-focus, motion blur, optical anomalies, alarm classification.

[0221] Based on the quantified relative degradation rate and standardized difference value, the imaging degradation is accurately decomposed into three specific types: defocusing, motion blur, and optical anomalies through multi-indicator fusion. At the same time, the health sequence is trend detected, and different alarm levels are set according to the degree of degradation to achieve graded early warning of imaging anomalies.

[0222] 7. Parameter result binding + search enhancement + large language model: semantic interpretation, evidence chain, suggested actions, and review loop.

[0223] All parameters, including equipment identification, indicator data, degradation conclusions, and alarm levels, are bound to the results to form an immutable record. The retrieval is enhanced by combining historical cases and maintenance manuals, and then input into a large language model to generate a simple semantic diagnostic report, which includes the indicator evidence chain and maintenance suggestions. Finally, the review results after maintenance are written back to the record, forming a closed-loop traceable management of "detection-diagnosis-treatment-review".

[0224] In summary, this application has the following beneficial effects: 1. Using the baseline archive as a unified reference, random sampling by window achieves stable quantification of image degradation, avoiding excessive alarms for extremely small fluctuations.

[0225] 2. By fusing multiple indicators, fading is decomposed into three categories: defocusing, out-of-focus, motion blur, and optical anomalies, thereby improving the interpretability of problem localization and maintenance efficiency.

[0226] 3. By introducing optical anomaly indicators such as illumination surface templates and histogram drift, it is possible to identify light source aging, lens contamination, and changes in illumination structure, pointing to specific light source channels.

[0227] 4. A trend detection continuous window confirmation and recovery hysteresis mechanism is adopted to reduce false alarms and adapt to fluctuations in production line conditions.

[0228] 5. Map the health sequence to a lifespan health index and estimate the remaining lifespan to enable predictive maintenance and spare parts planning.

[0229] 6. By binding parameter results to records and interpreting semantics using a large language model, an evidence chain and verification steps are generated for engineers, achieving a traceable closed loop.

[0230] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0231] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0232] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for camera and light source imaging quantification monitoring in AOI inspection, characterized in that, include: After the camera and light source are calibrated, multiple reference images are acquired, and a reference file is constructed based on the multiple reference images. The reference file includes the identifier ID that forms the multiple reference images and the reference parameters. During production operation, a camera is used to acquire multiple frames of images to be evaluated within a preset detection window. Fixed ROI regions and adaptive ROI regions are extracted from the multiple frames of images to be evaluated. Based on the AOI detection results, interfering regions in the fixed ROI regions and the adaptive ROI regions are removed to obtain a set of clean ROI regions. For the set of pure ROI regions, multiple indicators are extracted to obtain multiple indicator vectors, wherein the multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. Robust aggregation of multiple indicator vectors is performed to form a window indicator vector, which is then compared with the benchmark archive to obtain the relative decay rate and standardized difference index describing image degradation. Based on the relative degradation rate and standardized difference index, the imaging degradation is decomposed into defocusing degradation index, motion blur degradation index and optical anomaly degradation index, and the corresponding sub-item health and overall health of the imaging degradation are obtained. The sub-health scores and the total health score are combined over time or over a window sequence to form a health index curve. The records and semantic interpretations are then linked to obtain an estimated remaining lifespan and a semantic diagnosis and chain of evidence.

2. The method of claim 1, wherein the method further comprises: The step of acquiring multiple reference images after the camera and light source are calibrated, and constructing a reference file based on the multiple reference images, includes: After the camera and light source are calibrated, multiple reference images are acquired, and the multiple reference images are associated with the identifier ID to determine the associated identifier information of the reference file. The identifier ID includes camera ID, light source ID, recipe ID and ROI identifier. Based on the association identification information of the benchmark archive, the corresponding ROI region is extracted for each frame of the benchmark image and the single value of multiple indicators is calculated. The single values ​​of the same type of multiple indicators of multiple frames of the benchmark image under the same association identification information are statistically analyzed to obtain the median and absolute median difference of each indicator as the benchmark statistics. The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. The ROI region is processed for illumination surface and a baseline illumination surface template is generated. The baseline statistics and the baseline illumination surface template are used together as the components of the baseline parameter index vector. The associated identification information is bound to the corresponding baseline parameter index vector to obtain the baseline file.

3. The method of claim 1, wherein the method further comprises: The steps of acquiring multiple frames of images to be evaluated using a camera within a preset detection window during production operation, extracting fixed ROI regions and adaptive ROI regions from these multiple frames, and removing interfering regions from the fixed ROI regions and adaptive ROI regions based on the AOI detection results to obtain a set of clean ROI regions include: During production operation, a detection window is constructed for the same camera according to a preset number of detections, and multiple frames of images to be evaluated are collected within the detection window. For each frame of the image to be evaluated, a preset fixed ROI region and an adaptive ROI region selected based on gradient energy are extracted to form a set of ROI regions corresponding to each frame of the image. Based on the AOI detection results, the defect region mask and saturation region mask of the image to be evaluated are obtained, and the defect region and saturation region are identified as interference regions. The interfering regions are removed from the set of ROI regions corresponding to each frame of the image to be evaluated, the effective ROI regions of each frame of the image to be evaluated are extracted, and the effective ROI regions of multiple frames of the image to be evaluated are aggregated and summarized according to the detection window dimension to obtain the set of clean ROI regions.

4. The method of claim 1, wherein the method further comprises: The step of extracting multiple indicators from the set of pure ROI regions to obtain multiple indicator vectors includes: For each frame of the image to be evaluated in the set of pure ROI regions, a sharpness index based on gradient energy is formed by calculating the sum of squared gradient magnitudes, and a sharpness index based on second-order difference energy is formed by calculating the variance or mean energy of the Laplacian response. The sharpness index based on gradient energy and the sharpness index based on second-order difference energy are combined to obtain the sharpness index value of the effective ROI region of each frame of the image to be evaluated. For each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is calculated using the structure tensor or gradient direction energy distribution to obtain the directional index value of the effective ROI region of each frame of the image to be evaluated, which is used to characterize the directionality of motion blur. For each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is obtained by low-pass filtering or surface fitting and normalized. Then, the similarity, vignetting index and uniformity error are calculated with the reference illumination surface template to obtain the illumination surface index value of the effective ROI region of each frame of the image to be evaluated. For each frame of the image to be evaluated in the pure ROI region set, the effective ROI region is obtained by calculating the histogram distance of the effective ROI region of each frame of the image to be evaluated through grayscale or brightness quantile drift, saturation pixel ratio change and chi-square distance or Earth movement distance. For each frame of the image to be evaluated in the set of pure ROI regions, the noise index value of the effective ROI region of each frame of the image to be evaluated is obtained by using at least one of high-frequency residual energy, row and column fringe intensity and fixed pattern noise index. The sharpness index value, the directionality index value, the illumination surface index value, the histogram drift index value, and the noise index value are aggregated to form a single-frame index vector corresponding to each frame of the image to be evaluated. Then, the single-frame index vectors of multiple frames of the image to be evaluated in the clean ROI region set are combined to obtain multiple index vectors.

5. The method of claim 2, wherein the method further comprises: The step of robustly aggregating multiple indicator vectors to form a window indicator vector, and comparing it with the benchmark archive to obtain the relative degradation rate and standardized difference index describing image degradation includes: For multiple indicator vectors within a preset detection window, the median or truncated mean is used to calculate the window aggregate value of each indicator according to the indicator type. The absolute median difference is then used to characterize the fluctuation of each indicator after aggregation. The window aggregate value of multiple indicators and the corresponding fluctuation are combined one by one according to the indicator type to obtain the window indicator vector. Based on the identifier ID associated with the window indicator vector, the benchmark statistics corresponding to each indicator in the matching benchmark file are retrieved. Then, the window indicator vector and the benchmark statistics are matched one by one by indicator type to obtain the one-to-one matching relationship between the multi-indicator window aggregate value and the benchmark statistics. In the one-to-one matching relationship between the multi-index window aggregate value and the benchmark statistic, the relative degradation rate of the sharpness index, directionality index, illumination surface index and histogram drift index is obtained by subtracting the ratio of the window aggregate value to the corresponding benchmark statistic from 1. The larger the values ​​of the sharpness index, directionality index, illumination surface index and histogram drift index, the better the imaging state. In the one-to-one matching relationship between the multi-index window aggregated value and the benchmark statistic, the relative decay rate of the noise index is obtained by subtracting 1 from the ratio of the window aggregated value to the corresponding benchmark statistic. The larger the index value of the noise index, the worse the imaging state. Based on the window aggregate value of multiple indicators, the corresponding benchmark statistics, and the absolute median difference in the benchmark statistics, the standardized difference value of multiple indicators is calculated separately according to the indicator type. Based on the one-to-one correspondence of the relative decay rates of sharpness, directionality, illumination, and histogram drift, the relative decay rate of noise, and the standardized difference values ​​of multiple indicators, the relative decay rate and standardized difference index describing image degradation are obtained.

6. The method of claim 1, wherein the method further comprises: The step of decomposing the imaging degradation into defocusing degradation, motion blur degradation, and optical anomaly degradation based on the relative degradation rate and standardized difference index, and obtaining the corresponding sub-item health and overall health of the imaging degradation, includes: The relative degradation rate of the sharpness index is merged and quantified with the standardized difference value to obtain the defocusing degradation index, wherein the defocusing degradation index is used to characterize the degree of defocusing in the image. The relative degradation rate of the sharpness index and the directionality index is fused and quantized with the standardized difference value. When the sharpness index degrades and the directionality index increases, the representation weight of the fused quantization is increased to obtain the motion blur degradation index, wherein the motion blur degradation index is used to characterize the degree of degradation of imaging motion blur. The relative decay rate of the illumination surface index, the histogram drift index, and the noise index is fused and quantified with the standardized difference value. The illumination surface index, the histogram drift index, and the noise index respectively characterize the decay state of light source aging, abnormal light intensity distribution, and abnormal acquisition link sensor, to obtain the optical anomaly degradation index. The optical anomaly degradation index is used to characterize the overall decay degree of imaging optical anomalies. Based on preset quantitative evaluation rules, the defocusing degradation index, the motion blur degradation index, and the optical anomaly degradation index are weighted and calculated respectively to obtain the imaging sub-item health of the defocusing degradation index, the motion blur degradation index, and the optical anomaly degradation index. According to the preset health score weight allocation rules, the imaging health scores of the imaging degradation types are comprehensively weighted and fused to obtain the total health score. The total health score represents the overall imaging status of the camera and the light source. The imaging degradation types include out-of-focus, motion blur, and optical anomaly.

7. The method for quantitative monitoring of camera and light source imaging in AOI detection as described in claim 6, characterized in that, The step of forming a health index curve by combining the sub-items of health and the total health over time or over a window sequence, and binding records and semantic interpretations to obtain an estimated remaining lifespan and a semanticized diagnosis and chain of evidence, includes: The sub-item health scores and the total health scores are aligned according to a time series or detection window sequence. The cumulative operating conditions are then fused to fit and calculate the above health score data to obtain the imaging health index curve of the camera and the light source. The cumulative operating conditions include at least one of the following: cumulative number of shots, cumulative light source illumination duration, and cumulative exposure. The degradation trend analysis and feature extraction of the imaging health index curve are performed. Combined with the preset imaging maintenance threshold, the degradation degree of the imaging status of the camera and the light source is quantitatively evaluated, and the remaining lifespan of the imaging of the camera and the light source is estimated. The identifier ID, the features of the imaging health index curve, the remaining lifespan estimate, the sub-item health and the total health data, and the imaging degradation type are associated and integrated to generate a parameter result binding record that includes indicator evidence, degradation conclusion, and confidence level. The parameter result binding records are enhanced by searching historical cases, maintenance manuals, and production line process events. The enhanced information and the parameter result binding records are then input into a large language model to generate a semantic diagnostic report. The semantic diagnostic report includes at least the core cause of imaging degradation, an explanation of the indicator evidence chain, recommended actions, and verification steps.

8. A camera and light source imaging quantization monitoring system for AOI inspection, characterized in that, The system, which is applied to the camera and light source imaging quantization monitoring method in AOI inspection according to any one of claims 1-7, comprises: The data acquisition module is used to acquire multiple frames of reference images after the camera and light source are calibrated, and to construct a reference file based on the multiple frames of reference images. The reference file includes an identifier ID that forms the multiple frames of reference images, and reference parameters. The ROI management module is used to acquire multiple frames of images to be evaluated within a preset detection window using a camera during production operation, extract fixed ROI regions and adaptive ROI regions from the multiple frames of images to be evaluated, and remove interfering regions from the fixed ROI regions and the adaptive ROI regions based on the AOI detection results to obtain a set of clean ROI regions. The indicator calculation module is used to extract multiple indicators from the set of pure ROI regions to obtain multiple indicator vectors. The multiple indicators include at least the sharpness indicator, the directionality indicator, the illumination surface indicator, the histogram drift indicator, and the noise indicator. The benchmark and comparison module is used to robustly aggregate multiple index vectors to form a window index vector, which is then compared with the benchmark file to obtain the relative decay rate and standardized difference index describing image degradation. The degradation decomposition and alarm module is used to decompose the imaging degradation into defocusing degradation index, motion blur degradation index and optical anomaly degradation index based on the relative degradation rate and standardized difference index, and obtain the sub-item health and total health of the imaging degradation accordingly. The lifespan assessment module and the binding and semantic interpretation module are used to form a health index curve by combining the sub-items of health and the total health over time or over a window sequence, and to bind the records and semantic interpretation to obtain an estimated remaining lifespan and a semantic diagnosis and chain of evidence.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the camera and light source imaging quantization monitoring method in AOI detection as described in any one of claims 1 to 7.

10. A computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the camera and light source imaging quantization monitoring method in AOI detection as described in any one of claims 1 to 7.

Images