Abnormality detection method, device and equipment based on multi-band multi-illumination mode

Abstract

The application discloses an abnormality detection method, device and equipment based on a multi-band multi-illumination mode, and relates to the technical field of machine vision defect detection. The method is applied to a detection device and comprises the following steps: acquiring image data of an optical film to be detected; extracting abnormality-related features in the image data, wherein the abnormality-related features are used to represent the difference between an abnormal area and a background in the image data; generating a plurality of candidate illumination strategies according to the abnormality-related features; calculating an appearance score of each candidate illumination strategy according to a preset appearance score function, and taking the candidate illumination strategy corresponding to the maximum appearance score as a first illumination strategy to control the detection device to perform abnormality detection on the optical film to be detected.

Description

Technical Field

[0001] This application relates to the field of machine vision defect detection technology, and more specifically, to anomaly detection methods, apparatus and equipment based on multi-band multi-illumination modes. Background Technology

[0002] With the large-scale application of transparent film materials such as optical thin films and functional films in fields such as displays, photovoltaics, and optical devices, various abnormal defects such as scratches, particulate impurities, uneven film thickness, internal hidden damage, and surface reflective defects are easily generated during the product manufacturing process. If defects are missed or misdetected and flow into downstream processes, they will seriously affect the performance of finished products and product yield. Therefore, efficient and accurate online detection of surface and internal defects of film materials has become an industry necessity.

[0003] Currently, existing detection equipment typically uses fixed light source combinations or simple rotation methods for illumination acquisition. However, when facing highly reflective film surfaces, problems such as overexposure of the image, difficulty in highlighting weak low-contrast hidden damage, and unclear imaging characteristics of particulate defects are prone to occur. Existing illumination modes are difficult to take into account the imaging effects of different types of defects, which in turn leads to a low accuracy rate in subsequent image analysis and recognition. Summary of the Invention

[0004] The main objective of this application is to provide an anomaly detection method based on multi-band and multi-illumination modes, so as to solve the problem in the prior art that it is difficult to take into account the imaging effects of different types of defects.

[0005] To achieve the above objectives, the first aspect of this application proposes an anomaly detection method based on multi-band, multi-illumination modes, applied to detection equipment, comprising: Acquire image data of the optical film to be tested; Extracting anomaly-related features from the image data, wherein the anomaly-related features are used to characterize the difference in appearance between anomaly regions and the background in the image data; Based on the aforementioned anomaly-related features, multiple candidate lighting strategies are generated; The display score of each candidate illumination strategy is calculated according to a preset display score function. The candidate illumination strategy corresponding to the maximum display score is taken as the first illumination strategy to control the detection device to perform anomaly detection on the optical film to be detected.

[0006] Secondly, an anomaly detection device based on multi-band, multi-illumination modes is proposed and applied to detection equipment, including: The acquisition module is used to acquire image data of the optical film to be tested; An extraction module is used to extract anomaly-related features from the image data, wherein the anomaly-related features are used to characterize the difference in appearance between anomaly regions and the background in the image data; The generation module is used to generate multiple candidate lighting strategies based on the anomaly-related features; The selection module is used to calculate the display score of each candidate illumination strategy according to a preset display score function, and select the candidate illumination strategy corresponding to the maximum display score as the first illumination strategy to control the detection device to perform anomaly detection on the optical film to be detected.

[0007] Thirdly, an electronic device is proposed, comprising: At least one processor; and The memory is communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0008] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods in the embodiments of this disclosure.

[0009] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of the present disclosure.

[0010] The technical solutions provided by the embodiments of this application may include the following beneficial effects: In this application, by extracting the display difference features between the abnormal region and the background in the image of the optical film to be detected, generating multiple candidate illumination strategies, and then using a preset display score function to automatically select the illumination strategy with the highest display score for anomaly detection, the ability to display various types of anomalies can be effectively improved, the imaging quality can be effectively improved, and the accuracy and robustness of anomaly detection can be enhanced. Attached Figure Description

[0011] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A flowchart illustrating an anomaly detection method based on multi-band multi-illumination modes provided in this application; Figure 2 A schematic diagram of the structure of an anomaly detection device based on multi-band multi-illumination mode provided in this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0013] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0015] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0016] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0017] Figure 1A flowchart illustrating an anomaly detection method based on multi-band, multi-illumination modes, provided in this application, is applied to a detection device. The method includes: S110. Acquire image data of the optical film to be tested.

[0018] In this embodiment, an image is captured on the surface of the optical film using a visual imaging device to obtain image data. The acquisition process is completed using an initial general illumination method to ensure complete coverage of the area to be detected on the optical film.

[0019] In practice, a pre-scan acquisition can be performed on the optical film to be tested to obtain a pre-scan image as image data. For example, a pre-scan image can be obtained with fewer wavelengths, fewer illumination modes, and lower shooting costs. It can include several modes among white light orthogonal illumination, oblique side illumination, dark field, near infrared, or short-wave blue light. Alternatively, an intermediate detection image can be obtained as image data after performing acquisition of at least one candidate illumination strategy.

[0020] S120. Extract anomaly-related features from the image data, wherein the anomaly-related features are used to characterize the difference in appearance between the abnormal region and the background in the image data.

[0021] In this embodiment of the application, the abnormality-related features can be various quantitative feature information that can intuitively reflect the imaging difference between abnormal areas and normal background areas in an image, and can directly reflect the degree of difference between the two in visual presentation.

[0022] Feature extraction processing is performed on the acquired image data of the optical film to be tested. Candidate abnormal regions are located and screened within the image range. Various quantitative features that can reflect visual imaging differences are then extracted from these regions and integrated to form abnormality-related features.

[0023] S130. Based on the aforementioned anomaly-related features, generate multiple candidate lighting strategies.

[0024] In this embodiment of the application, after extracting the anomaly-related features, multiple candidate lighting strategies are generated based on the anomaly-related features. Each candidate lighting strategy corresponds to a set of actual executable lighting parameters (such as band, lighting mode, illumination angle, polarization state, exposure time, etc.) and can be used for subsequent anomaly detection and acquisition.

[0025] S140. Calculate the display score of each candidate illumination strategy according to the preset display score function, and take the candidate illumination strategy corresponding to the maximum display score as the first illumination strategy to control the detection device to perform anomaly detection on the optical film to be detected.

[0026] In this embodiment, the preset display score function can be used to predict the display effect of the abnormal region after the candidate lighting strategy is implemented, based on the parameters of the candidate lighting strategy itself and the anomaly-related features extracted from the image data. The display score is used to quantitatively represent the expected quality of a candidate lighting strategy in displaying the current abnormal region; the higher the score, the more significantly the strategy can make the anomaly different from the background, and the more beneficial it is to subsequent anomaly detection.

[0027] A preset display score function is invoked to output the display score corresponding to each candidate illumination strategy through quantitative calculation. The display scores of all candidate illumination strategies are compared and sorted, and the candidate illumination strategy with the highest display score is selected as the first illumination strategy. According to the determined first illumination strategy, the corresponding illumination components are adjusted to enable preset illumination parameters to perform accurate image acquisition and anomaly detection on the optical film to be inspected, effectively solving the problem that different types of anomalies are difficult to clearly display and improving the recognition accuracy of anomaly areas.

[0028] According to the embodiments of this application, by extracting the display difference features between the abnormal area and the background in the image of the optical film to be detected, generating multiple candidate illumination strategies, and then using a preset display score function to automatically select the illumination strategy with the highest display score for anomaly detection, the ability to display various types of anomalies can be effectively improved, the imaging quality can be effectively improved, and the accuracy and robustness of anomaly detection can be enhanced.

[0029] In one implementation, extracting anomaly-related features from the image data may include: The image data is used to locate abnormal regions and identify candidate abnormal regions. General imaging features, anomaly features, and difference features are extracted from the candidate anomaly regions, respectively. The general imaging features are used to characterize the overall image quality and interference level; the anomaly features are used to characterize the morphological and structural properties of the anomaly region; and the difference features are used to characterize the enhancement of anomaly visibility under different illumination modes. The general imaging features, the anomaly features, and the difference features are fused to obtain the anomaly-related features.

[0030] In this embodiment, when extracting features within a candidate anomalous region, the input data may include: image data of the optical film to be detected, the candidate anomalous region (ROI), the background region surrounding the ROI, and acquired metadata. The image data of the optical film to be detected includes a pre-scan image, an intermediate detection image, or an image acquired under multi-band / multi-illumination modes. The candidate anomalous region (ROI) includes ROI coordinates, a bounding box, a candidate mask, or a candidate center point. The background region surrounding the ROI can be obtained by expanding the ROI bounding box outward by a predetermined number of pixels (e.g., 10 to 30 pixels) to form a local window, and then subtracting the ROI mask body; alternatively, a non-anomalous region within the same local window can be used as the background. The acquired metadata includes band, illumination configuration mode, incident angle, polarization state, exposure time, gain, camera number, and acquisition time. In practice, a reference illumination image or historical standard image is also included to calculate the display differences between different illumination modes.

[0031] Based on the original image and candidate anomaly localization results, the Region of Interest (ROI) is determined according to candidate bounding boxes or candidate masks, and a local background ring is generated around the ROI. For example, a local window is formed by expanding outward by 10 to 30 pixels from the ROI's bounding box, and then the ROI mask area is excluded, with the remaining area being used as the background region. The output consists of the ROI mask, the background mask, and the ROI's bounding box.

[0032] After acquiring the ROI image patch, background image patch, and acquired metadata, flat field correction, dark current subtraction, grayscale normalization, denoising, exposure normalization, and necessary image registration are performed to reduce the impact of light source brightness fluctuations, exposure differences, and camera noise. The output consists of normalized ROI image patches and background image patches.

[0033] Based on the standardized ROI image patches and background image patches, indicators such as gray-level mean, gray-level variance, RMS contrast ratio, gradient energy, sharpness, saturation ratio, reflectivity ratio, noise level, and texture entropy are calculated to form a general imaging feature vector. This general imaging feature vector characterizes the overall imaging quality and interference level of the current image. The brightness mean is the average gray-level value of pixels within the ROI; the RMS contrast ratio is the root mean square change of the relative gray-level mean within the ROI; the saturation ratio is the number of oversaturated or undersaturated pixels divided by the total number of pixels in the ROI; and the reflectivity ratio is the number of specularly highlighted pixels divided by the total number of pixels in the ROI.

[0034] Using ROI masks, ROI image patches, and edge response maps, anomaly area, aspect ratio, principal direction, edge sharpness, edge closure, internal texture stability, halo radius, foreground / background separation, and morphological compactness are calculated to form anomaly feature vectors. These anomaly features describe the morphological and structural attributes of the anomalous target. The area is calculated by multiplying the number of pixels in the ROI mask by the pixel area; the aspect ratio is calculated by dividing the longer side of the bounding box by the shorter side; edge sharpness can be represented by the mean gradient near the boundary; and foreground / background separation can be represented by dividing the difference between the mean of the anomaly area and the mean of the background area by the sum of their standard deviations.

[0035] Image sets of the same ROI under different bands, illumination angles, polarization states, or exposure conditions are acquired. Indicators such as contrast difference, edge visibility difference, saliency difference, reflection suppression difference, saturation suppression difference, and segmentation stability difference are calculated to form a difference feature vector. These difference features characterize the improvement effect of different illumination modes on anomaly visibility. For example, using the initial illumination mode as a reference, the improvement in ROI-background separation, edge energy, reflection ratio, and saturation ratio are calculated under candidate illumination modes.

[0036] The general imaging feature vector, anomaly feature vector, and differential feature vector are taken as inputs and standardized using z-score based on the mean and standard deviation of the calibrated sample set, or normalized using min-max based on the minimum and maximum values. Then, vector concatenation, weighted concatenation, or lightweight model fusion are used to generate the final anomaly-related feature vector. This vector will be used for subsequent illumination strategy generation and display score calculation.

[0037] Specifically, if the candidate anomaly region is R, the background region is B, the current lighting mode is m, and the reference lighting mode is r, I m This represents an image acquired under illumination mode m. The formula for calculating the eigenvalues ​​of the relevant features is as follows: Characteristic values ​​of brightness difference feature: The larger this value, the more obvious the brightness difference between the abnormal area and the background area.

[0038] Feature values ​​of foreground / background separation: ,in , These are the average gray values ​​of the abnormal region and the background region, respectively. , These are the corresponding standard deviations. It is a very small constant to prevent the denominator from being zero. The larger the value, the easier it is to distinguish the abnormality.

[0039] Eigenvalues ​​of edge strength: That is, at the abnormal boundary Calculate the mean of the gradient magnitudes in the vicinity. The larger this value, the clearer the anomaly edge.

[0040] Characteristic values ​​of reflectivity: ,in This represents the number of pixels within the candidate anomalous region that were identified as specular highlights. This represents the total number of pixels in the area. The larger this value, the stronger the reflection interference.

[0041] Characteristic values ​​of saturation ratio: ,in This represents the number of oversaturated or undersaturated pixels within the candidate anomalous region. The larger this value, the less effective imaging information there is.

[0042] The eigenvalues ​​of the difference feature: These are defined by comparing the current lighting mode m with the reference lighting mode r. , , Among them, the improvement value of separation and edge intensity (positive number) indicates that the current lighting mode is better than the reference lighting mode, and the decrease value of reflectivity (positive number) indicates that the current mode effectively suppresses specular reflection.

[0043] After extracting the above features, the general imaging features are... Abnormal characteristics and differences Perform normalization (or standardization) to ensure that the values ​​fall within the range of [0,1], and then fuse them using one of the following methods: Vector concatenation: ; Weighted fusion: Where norm represents normalization or standardization, and W1, W2, and W3 are all weight coefficients that can be determined based on empirical rules, historical sample statistics, or lightweight model training.

[0044] According to the embodiments of this application, by extracting and fusing multiple types of features within the candidate abnormal region, the feature extraction range is narrowed and the computational cost is reduced. At the same time, effective information can be comprehensively collected from multiple dimensions such as imaging environment, defect ontology, and illumination adaptation. The obtained abnormality-related features can truly and comprehensively reflect the actual visual differences between the abnormal region and the background region, providing sufficient, reliable, and multi-dimensional data support for the subsequent accurate generation of adaptive lighting strategies.

[0045] In one implementation, generating multiple candidate lighting strategies based on the anomaly-related features may include: The abnormality-related features are quantitatively analyzed to determine the abnormality type and imaging interference type corresponding to the candidate abnormality region; Based on the anomaly type and the imaging interference type, the corresponding lighting parameters are matched using a preset anomaly feature and lighting parameter matching rule library. The lighting parameters include band parameters, lighting layout mode, light incident angle, polarization adjustment parameters, and imaging exposure parameters. Several lighting parameters are combined to generate multiple candidate lighting strategies.

[0046] In this embodiment, anomaly types may include surface contamination, sub-film foreign objects, scratches, particles, uneven coating, and specular artifacts. Imaging interference types may include high reflectivity interference, saturation interference, low contrast interference, complex texture interference, and shadow interference. The anomaly feature and illumination parameter matching rule base can be a pre-stored and established correspondence database, containing various illumination adjustment schemes and parameter standards adapted to different anomaly types and different imaging interference scenarios, providing a unified judgment basis for intelligent matching of illumination parameters.

[0047] Illumination parameters are adjustable variables that control the output characteristics of the light source, used to change lighting conditions to highlight different anomaly types. Band parameters refer to the emission spectrum frequency band of the light source used during image acquisition, adapted to the light transmission and reflection imaging requirements of different membrane materials. Illumination layout mode refers to the overall arrangement of the light sources, the combination of illumination points, and the operating mode, including one or more of coaxial light, oblique light, dark field light, backlight, and polarized light. Light incident angle refers to the tilt angle and illumination direction of the light source rays incident on the surface of the membrane material to be tested. Polarization adjustment parameters are used to adjust the polarization state of the light, reducing specular reflection and stray light reflection on the membrane surface. Imaging exposure parameters are shooting adjustment parameters that control the image acquisition exposure time and photosensitivity.

[0048] In this embodiment, the anomaly-related feature vectors are comprehensively analyzed to determine the anomaly type and imaging interference type of the current candidate anomaly region. For example: if the edge energy is high and the edge closure is good, and the texture entropy is low, it is likely to be identified as a scratch; if the gray-level distribution inside the region is uniform but the overall contrast is low, and the spectral response difference shows enhanced transmission under near-infrared light, it is likely to be identified as a foreign object under the film; if the saturation ratio is >10% and the specular reflection ratio is >5%, it is determined that there is high reflectivity interference; if the RMS contrast is <20 and the edge energy is low, it is determined that there is low contrast interference. The judgment method can use a threshold decision tree, a Bayesian classifier, or a lightweight classification model.

[0049] By taking the anomaly type and imaging interference type as input, the system queries the preset "anomaly feature and lighting parameter matching rule library" to obtain the corresponding recommended values ​​or constraints for lighting parameters.

[0050] The obtained lighting parameters are combined and matched, using either a single parameter independently or multiple parameters in combination to generate multiple candidate lighting strategies. For example, a Cartesian product is used to combine all possible values ​​of each parameter to generate a tuple of all possible lighting parameters.

[0051] According to the embodiments of this application, by quantitatively analyzing the abnormality-related features to identify the abnormality type and imaging interference type, and by using a rule base to match and flexibly combine suitable lighting parameters, a highly targeted candidate lighting strategy can be generated, thereby improving the matching efficiency and accuracy of the lighting scheme with the current abnormality display requirements.

[0052] In practical applications, the anomaly type and imaging interference type can be input into the lighting decision model trained based on deep learning algorithms (such as convolutional neural networks and decision tree models). The lighting parameter set output by the lighting decision model is then analyzed one by one to clarify the specific configuration corresponding to each set of lighting parameters (such as the specific values ​​of band parameters, lighting layout mode, and light incident angle). If it is a scenario where multiple parameters are used in conjunction, the execution order of each set of parameters also needs to be determined. The lighting parameter configuration and the corresponding execution order are integrated into a complete lighting implementation plan, which is a candidate lighting strategy.

[0053] In one embodiment, the preset display score function includes a single-mode score function and a combined-mode score function, and the candidate lighting strategy includes a single-mode lighting strategy and a combined-mode lighting strategy composed of multiple single modes. The step of calculating the display score of each candidate lighting strategy according to a preset display score function, and taking the candidate lighting strategy corresponding to the maximum display score as the first lighting strategy, may include: The single-mode display score for each single-mode lighting strategy is calculated using the single-mode scoring function; the combined-mode display score for each combined-mode lighting strategy is calculated using the combined-mode scoring function. Under preset constraints, a preset search algorithm is used to select the candidate lighting strategy corresponding to the maximum value from all the single-mode display scores and the combined-mode display scores as the first lighting strategy. The constraints include one or more of total acquisition time, total number of exposures and device switching cost. The preset search algorithm includes one or more of exhaustive search, greedy search, bundle search, dynamic programming, genetic algorithm, Bayesian optimization and reinforcement learning.

[0054] In this embodiment, a single-mode lighting strategy can be a lighting scheme that uses only one lighting mode and a single parameter combination, suitable for detecting a single type of anomaly. A combined-mode lighting strategy can be composed of multiple single-mode lighting strategies, working collaboratively through multiple sets of lighting parameters and multiple lighting modes, suitable for detecting complex anomalies and multiple types of anomalies. Preset constraints can be limitations set to ensure detection efficiency and control detection costs, including total acquisition time, total number of exposures, and equipment switching costs. Preset search algorithms can be algorithms used to screen the optimal candidate lighting strategy, including exhaustive search, greedy search, bundle search, dynamic programming, genetic algorithms, Bayesian optimization, reinforcement learning, etc.

[0055] For single-mode lighting strategies (i.e., lighting schemes using only one lighting mode and a single parameter combination), a single-mode score function is used for calculation. The formula for the single-mode score function is as follows: ,in, The single-mode display score represents the single-mode illumination strategy d. The higher the score, the better the adaptability and imaging effect of the single-mode illumination strategy. , , , , , , All are preset weighting coefficients; The score for abnormal appearance under single-mode lighting strategy d; The separation score between the abnormal region and the background region under the single-mode lighting strategy d; The edge visibility score under single-mode lighting strategy d; The texture rendering score under the single-mode lighting strategy d; For reflection penalty under single-mode lighting strategy d; This is the saturation penalty term under the single-mode lighting strategy d; The cost of mode switching and acquisition under the single-mode lighting strategy d.

[0056] In practical applications, anomaly display scores in single-mode scoring functions Separation score Edge visibility score and texture display score The corresponding feature values ​​extracted from the pre-scanned image can be used. Specifically: the separation score is the foreground / background separation of the candidate anomaly region in the pre-scanned image; the edge visibility score is the edge intensity of the candidate anomaly region in the pre-scanned image; the texture visibility score is the texture entropy of the candidate anomaly region in the pre-scanned image; and the anomaly visibility score is the mean of the anomaly saliency map of the candidate anomaly region in the pre-scanned image. Since all candidate strategies have the same sub-scores, the differences in visibility scores between different strategies mainly come from the reflection penalty term. Saturation penalty item and cost items Among them, the reflection penalty: if the candidate illumination strategy includes polarized light or a dark field mode, then Set to a smaller value (e.g., 0.1); otherwise set to a larger value (e.g., 0.5). Saturation penalty: If the candidate lighting strategy uses a low-exposure or dark mode, then... Set to a smaller value (e.g., 0.1); otherwise set to a larger value (e.g., 0.5). Cost item: Calculated according to preset weights based on the acquisition time, exposure times, and light source switching times required by the strategy.

[0057] For combined mode lighting strategies (i.e., schemes composed of multiple single-mode lighting strategies that work together through multiple sets of lighting parameters), a combined mode score function is used for calculation. The formula for calculating the combined mode score function is as follows: ,in, The combined mode illumination strategy C is represented by the combined mode display score. The higher the score, the better the overall adaptability, imaging effect and economy of the combined mode illumination strategy. The sum of the single-mode display scores of all single-mode lighting strategies m_i contained in the combined mode lighting strategy C; , , All are preset weight coefficients; Comp(C) is the combination complementarity score of the combined mode lighting strategy C, such as the information gain complementarity of different modes in the anomaly display dimension; Red(C) is the combination redundancy of the combined mode lighting strategy C, such as two modes providing almost the same information to the target region of interest; Switch(C) is the mode switching cost of the combined mode lighting strategy C, including time cost and control cost. Specifically, the complementarity score Comp(C) and the redundancy score Red(C) can be quantified as follows: The complementarity score is calculated by averaging the cosine distance between the feature vectors of every two single-mode lighting strategies in the combination C (e.g., vectors composed of expected separation degree, edge intensity, etc. corresponding to each strategy). The larger the distance, the stronger the information complementarity of the two strategies in the anomaly display dimension. The redundancy score is calculated by averaging the intersection-union ratio (IoU) of the anomaly region masks in the images acquired by every two strategies. The larger the IoU, the more similar the anomaly masks produced by the two strategies, and the higher the redundancy.

[0058] By clearly defining predefined constraints and considering the actual needs of industrial testing, the optimal strategy selected is ensured to not only achieve good testing results but also meet the efficiency and cost requirements of actual production. A predefined search algorithm is invoked to comprehensively analyze and filter all single-mode and combined-mode display scores. Through algorithmic calculations, the strategy with the highest display score that meets the predefined constraints is selected from all candidate lighting strategies and designated as the first lighting strategy, i.e., the optimal lighting strategy for the current testing scenario.

[0059] According to the embodiments of this application, by clearly classifying the scoring function and calculating the display scores of various lighting strategies, and combining preset constraints and search algorithms to screen the optimal strategy, it is possible to accurately select a lighting scheme that is suitable for the current detection scenario and balances detection accuracy and production efficiency. This effectively avoids the blind selection of lighting strategies, reduces detection errors caused by improper lighting, and controls detection costs and time consumption through constraints, ensuring that the detection process is efficient and economical. This provides a scientific and reliable strategy screening method for optical film anomaly detection.

[0060] In one embodiment, after selecting the candidate lighting strategy corresponding to the maximum display score as the first lighting strategy, the method may further include: The image of the optical film to be tested is acquired according to the first illumination strategy, and the acquisition result is obtained. Based on the collected results, calculate the abnormal manifestation gain of the first lighting strategy; Determine whether the abnormal display gain is greater than or equal to a preset gain threshold; If the abnormal display gain is greater than or equal to a preset gain threshold, then the first lighting strategy is used as the target lighting strategy. If the abnormality manifestation gain is less than a preset gain threshold, a second illumination strategy is selected from the multiple candidate illumination strategies to acquire an image of the optical film to be tested, and the abnormality manifestation gain is recalculated.

[0061] In this embodiment, the anomaly display gain can be used to quantify the actual improvement in the highlighting effect of the current illumination strategy relative to the reference illumination strategy in the anomaly region. The higher the value, the stronger the highlighting ability of the strategy and the more significant the improvement in imaging effect. The preset gain threshold can be a pre-set critical value for determining whether the anomaly display gain meets the standard, and can be flexibly adjusted according to the material of the optical film to be tested, the type of anomaly, the detection accuracy requirements, and the needs of industrial production.

[0062] Based on the first illumination strategy determined by the screening, the corresponding illumination components and imaging equipment are automatically adjusted, and the candidate abnormal areas of the optical film to be tested are subjected to secondary precise image acquisition according to the preset illumination parameters (such as waveband, incident angle, polarization state, etc.) and the adjustment execution sequence.

[0063] When calculating the anomaly detection gain, the required input data may include: a reference image, the current strategy image (i.e., the image acquired using the first illumination strategy), the ROI mask or coordinates of the same candidate anomaly region, and image acquisition metadata. The reference image can be the initial general illumination image, a pre-scan image, or an image from the previous round of known illumination strategies; the current strategy image is the image re-acquired according to the first illumination strategy or the current candidate illumination strategy; the ROI mask or coordinates of the same candidate anomaly region, and the corresponding background reference region; and the image acquisition metadata, including band, incident angle, polarization state, exposure time, gain, and acquisition time. In practice, multiple sampled images are also included to calculate segmentation stability, consistency, and repeatability.

[0064] The specific calculation process is as follows: Image registration and brightness normalization: for reference image I ref and the current policy image I s Perform ROI alignment, exposure normalization, and brightness normalization to eliminate positional shifts, overall brightness fluctuations, and local distortions caused by lighting changes, ensuring that subsequent comparisons accurately reflect differences in lighting appearance. The output consists of an aligned reference ROI image and the current strategy ROI image.

[0065] Calculate the anomaly / background separation gain: Calculate the separation degree between the anomaly region and the background region in both the reference image and the current policy image. If the separation degree under the current policy is significantly higher than that in the reference image, it indicates that the anomaly is more easily distinguished from the background. Output the separation gain G. sep .

[0066] Calculate edge visibility gain: Calculate gradient strength and edge energy (such as Sobel or Canny response) near the ROI boundary. A stronger edge response under the current policy indicates a clearer anomaly contour, which is beneficial for subsequent localization and segmentation. Output edge visibility gain G. edge .

[0067] Calculate reflection and saturation suppression gain: Statistically compare the proportions of reflected and saturated pixels within the ROI in the reference image and the current strategy image. If the current strategy reduces specular reflection and overexposure / underexposure, it indicates that the strategy reduces imaging interference. Output reflection suppression gain G. reflect and saturation suppression gain G sat .

[0068] Calculate segmentation stability or saliency gain: Perform lightweight segmentation or saliency detection on the reference image and the current policy image respectively, and compare the integrity, confidence, and consistency of the anomaly mask across multiple samplings. If the anomaly mask is more complete, has higher confidence, and is more stable under the current policy, then the actual detection reliability is higher. Output segmentation stability gain G. mask .

[0069] Weighted fusion yields the anomaly detection gain: The above sub-gains are normalized, then weighted and summed, with costs such as acquisition time, mode switching costs, and exposure counts deducted to obtain the final anomaly detection gain. The output is a scalar gain value and detailed scores for each sub-item.

[0070] If Sep(Is), Edge(Is), Reflect(Is), Sat(Is), Conf seg (Is) represent the separation degree of the image Is acquired under the current lighting strategy in the candidate anomaly region. Edge strength Reflectivity saturation ratio And the split confidence, Sep(I) ref Edge (I) ref ), Reflect(I ref ), Sat(I ref ), Conf seg (I ref (I) represents the image I under the reference illumination mode. ref Separation in candidate anomaly regions Edge strength Reflectivity saturation ratio And segmentation confidence. The segmentation confidence can be calculated based on the cross-union ratio between the segmentation result and the average mask from multiple samplings, or based on the score of the continuity of the segmentation mask edges. The sub-gains are defined as follows: Separation gain: A value greater than 0 indicates that the current strategy makes the separation of the exception from the background more obvious.

[0071] Edge visibility gain: A value greater than 0 indicates a stronger abnormal edge.

[0072] Reflection suppression gain: A value greater than 0 indicates that the current strategy reduces reflective interference.

[0073] Saturation suppression gain: A value greater than 0 indicates an increase in the effective pixel ratio.

[0074] Segmentation stability gain: Alternatively, the cross-union ratio of multiple sampling masks can be increased; a higher value indicates a more stable detection result.

[0075] Overall cost penalty: T acq N represents the data acquisition time. exp N represents the number of exposures. switch The number of mode switching times is represented by a, b, and c, which are preset weighting coefficients.

[0076] In one implementation, the anomaly manifestation gain is calculated using the following formula: , where N(·) represents the normalization function, and w1 to w6 are weighting coefficients, which can be determined based on the anomaly type, test criteria, or historical sample statistics. For example, for scratch-type anomalies, the weights of edge visibility gain and separation gain can be increased; for highly reflective film surfaces, the weights of reflection suppression gain can be increased; and for foreign objects under low-contrast film, the weights of separation gain and segmentation stability gain can be increased.

[0077] Alternatively, one can first define the display quality score of an image, and then calculate the abnormal display gain. The formula for calculating the display quality score V(I) is: The abnormal manifestation gain can then be expressed as the absolute increase. Or relative increase factor .

[0078] The calculated abnormality display gain value is compared with a preset gain threshold. If the abnormality display gain is greater than or equal to the preset gain threshold, it indicates that the actual imaging effect of the first illumination strategy has met expectations, effectively highlighting abnormal areas, weakening imaging interference, and meeting the accuracy requirements of abnormality detection. This first illumination strategy is then designated as the target illumination strategy, and subsequent abnormality detection of the optical film under test will be performed according to this target illumination strategy. If the abnormality display gain is less than the preset gain threshold, it indicates that the actual imaging effect of the first illumination strategy has not met expectations, and there may be problems such as unclear highlighting of abnormal areas or ineffective suppression of imaging interference. In this case, the strategy with the second-highest display score among all previously generated candidate illumination strategies is selected as the second illumination strategy. Image acquisition of the optical film under test is performed again according to the second illumination strategy, and the image acquisition, abnormality detection, and abnormality display gain calculation are repeated until the abnormality display gain meets the preset gain threshold or reaches the preset switching cost limit.

[0079] In practice, the abnormality display gain is calculated by combining one or more of the following factors: abnormal region contrast enhancement, edge integrity enhancement, artifact suppression degree, and segmentation stability.

[0080] According to the embodiments of this application, by actually collecting and verifying the first illumination strategy, calculating the abnormality display gain and comparing it with the preset gain threshold, a secondary screening and effect verification of the illumination strategy is realized. This effectively avoids the possible bias of evaluating solely through a scoring function, ensuring that the final determined target illumination strategy can truly adapt to the actual detection scenario and meet the anomaly detection accuracy requirements. At the same time, when the first illumination strategy fails to meet expectations, a second illumination strategy is selected for secondary verification, ensuring the continuity of the detection work, further reducing the probability of missed detections and false detections, and improving the reliability and stability of optical film anomaly detection.

[0081] In one embodiment, the anomaly-related features include one or more of the following: brightness features, contrast features, edge intensity features, texture stability features, saturation ratio features, and reflectivity ratio features.

[0082] In this embodiment, brightness features are the most basic feature type among anomaly-related features. During the extraction process, the pixel brightness values ​​of candidate anomaly regions and corresponding background regions in the image data are statistically analyzed across the entire domain. The average, maximum, minimum, and variance of pixel brightness within the region are calculated. By comparing the brightness statistics of the anomaly region and the background region, the brightness difference parameter between the two is obtained, which is the brightness feature.

[0083] Contrast features are used to characterize the degree of brightness contrast between candidate anomaly regions and background regions. During extraction, based on brightness features, a preset algorithm is used to calculate the brightness difference and brightness ratio between the anomaly and background regions, quantifying the magnitude of their contrast to obtain the contrast features. Specifically, the RMS contrast ratio calculation method can be used to quantify the overall contrast level; the specific calculation formula is as follows: ,in, RMS contrast of the target region of interest. The total number of pixels within the target region of interest. Let be the grayscale value of the i-th pixel within the region. This represents the average grayscale value of all pixels within the target region of interest; this metric directly reflects the overall contrast between the abnormal region and the background region. The higher the value, the more obvious the contrast between the anomaly and the background, and the higher the discrimination. Furthermore, the foreground / background separation index can be used to further quantify the separation effect between the anomaly and the background; the specific calculation formula is as follows: ,in, Foreground / background separation index and These represent the average intensity (average grayscale value) of the abnormal candidate foreground region (candidate abnormal region) and the background region, respectively. and These represent the standard deviations of the grayscale values ​​of the abnormal candidate foreground region and the background region, respectively. This is a local constant (used to avoid a denominator of 0 and ensure the validity of the calculation). The larger the value, the better the separation effect between the abnormal candidate foreground and the background, and the easier it is to identify the abnormal region.

[0084] Edge intensity features are used to characterize the clarity and sharpness of the edge contours of candidate anomaly regions. During extraction, edge detection algorithms (such as Sobel and Canny algorithms) are used to process the image data, identify the edge contours of the anomaly regions, calculate the gray-level gradient values ​​of the edge pixels, and quantify the clarity and sharpness of the edges to obtain the edge intensity features. Simultaneously, the highlighting effect of the current lighting mode on the anomaly edges can be further quantified by edge enhancement gain. The specific calculation formula is as follows: ,in, For edge visibility gain, This represents the edge energy of abnormal areas under the current lighting mode (the sum or mean of the grayscale gradient values ​​of edge pixels, used to quantify the edge sharpness). The edge energy of the anomalous region under the reference lighting mode (such as the initial acquisition lighting mode), A value greater than 1 indicates that the current lighting mode is better at highlighting abnormal edges than the reference lighting mode. The larger the value, the more obvious the edge enhancement.

[0085] Texture stability features are used to characterize the difference and stability between the texture structure within candidate anomaly regions and the texture structure of background regions. During the extraction process, statistical analysis is performed on the texture density, texture direction, and texture uniformity of the anomaly and background regions to calculate the texture difference parameters and the stability of the texture in the anomaly region, thus obtaining the texture stability features. Simultaneously, an anomaly visibility effectiveness quantification index can be used to help characterize the highlighting effect of the current lighting mode on the anomalies. The specific calculation formula is as follows: ,in, Used to quantify the effectiveness of the current lighting pattern in revealing anomalies. This is an anomaly saliency map or intermediate segmentation response map (used to visually represent the saliency of anomaly regions). The mean of the anomaly saliency map or the intermediate segmentation response map (the larger the mean, the more significant the anomaly region). For multiple sampling or multiple illumination modes, it is a consistency measure of anomaly saliency maps or intermediate segmentation response maps (the higher the consistency, the better the stability of anomaly manifestation in the current mode). The higher the value, the better the current lighting mode is at revealing anomalies and the stronger its stability.

[0086] The saturation ratio feature is used to characterize the pixel saturation degree of candidate abnormal regions and background regions in image data. During the extraction process, the proportion of oversaturated and undersaturated pixels in the abnormal regions and background regions is counted, and the saturation ratio parameter is calculated to obtain the saturation ratio feature.

[0087] The reflectance ratio feature is used to characterize the reflectance of candidate abnormal regions and background regions in image data. During extraction, image grayscale thresholding and reflectance region recognition algorithms are used to statistically analyze the proportion of reflective pixels within abnormal and background regions, calculate the reflectance ratio parameter, and obtain the reflectance ratio feature. The specific calculation methods for the saturation ratio feature and the reflectance ratio feature are as follows: ,in, For saturation ratio, This represents the total number of oversaturated and undersaturated pixels within the target region of interest. For the mirror reflection ratio, The number of specular reflective pixels within the target region of interest. The total number of pixels within the target region of interest; and Both should be included as negative constraints in the illumination selection model. The larger the values ​​of both, the more severe the saturation problem and reflection interference in the imaging process, which will negatively affect the accuracy of anomaly detection.

[0088] In practical applications, the lighting strategy and feature calculation parameters are dynamically updated. By dynamically adjusting the lighting strategy and feature calculation parameters, the problems of "variable anomaly types and complex imaging environments" in industrial inspection can be solved, ensuring that the detection accuracy always meets the preset requirements, while taking into account detection efficiency and cost control, and realizing flexible adaptation and optimization of technical solutions.

[0089] Figure 2 This application provides a schematic diagram of an anomaly detection device based on multi-band multi-illumination modes, which is applied to detection equipment. The device may include: The acquisition module 210 is used to acquire image data of the optical film to be tested; Extraction module 220 is used to extract anomaly-related features from the image data, wherein the anomaly-related features are used to characterize the difference in appearance between anomaly regions and the background in the image data; The generation module 230 is used to generate multiple candidate lighting strategies based on the anomaly-related features; The selection module 240 is used to calculate the display score of each candidate illumination strategy according to a preset display score function, and to select the candidate illumination strategy corresponding to the maximum display score as the first illumination strategy, so as to control the detection device to perform anomaly detection on the optical film to be detected.

[0090] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown in the embodiment of this application, an anomaly detection device 300 based on multi-band multi-illumination mode is provided, including a memory 330, a processor 310 and a computer program 320 stored in the memory. The processor 310 executes the computer program 320 to implement the anomaly detection method based on multi-band multi-illumination mode described in any of the above embodiments.

[0091] An anomaly detection device based on multi-band multi-illumination mode provided in this application embodiment may include a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the anomaly detection method based on multi-band multi-illumination mode described in any of the above embodiments.

[0092] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the anomaly detection method based on multi-band multi-illumination modes described in any of the above embodiments.

[0093] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0094] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0095] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An anomaly detection method based on multi-band multi-illumination modes, characterized in that, Applied to a testing device, the method includes: Acquire image data of the optical film to be tested; Extracting anomaly-related features from the image data, wherein the anomaly-related features are used to characterize the difference in appearance between anomaly regions and the background in the image data; Based on the aforementioned anomaly-related features, multiple candidate lighting strategies are generated; The display score of each candidate illumination strategy is calculated according to a preset display score function. The candidate illumination strategy corresponding to the maximum display score is taken as the first illumination strategy to control the detection device to perform anomaly detection on the optical film to be detected.

2. The method according to claim 1, characterized in that, The extraction of anomaly-related features from the image data includes: The image data is used to locate abnormal regions and identify candidate abnormal regions. General imaging features, anomaly features, and difference features are extracted from the candidate anomaly regions, respectively. The general imaging features are used to characterize the overall image quality and interference level; the anomaly features are used to characterize the morphological and structural properties of the anomaly region; and the difference features are used to characterize the enhancement of anomaly visibility under different illumination modes. The general imaging features, the anomaly features, and the difference features are fused to obtain the anomaly-related features.

3. The method according to claim 2, characterized in that, The step of generating multiple candidate lighting strategies based on the anomaly-related features includes: The abnormality-related features are quantitatively analyzed to determine the abnormality type and imaging interference type corresponding to the candidate abnormality region; Based on the anomaly type and the imaging interference type, the corresponding lighting parameters are matched using a preset anomaly feature and lighting parameter matching rule library. The lighting parameters include band parameters, lighting layout mode, light incident angle, polarization adjustment parameters, and imaging exposure parameters. Several lighting parameters are combined to generate multiple candidate lighting strategies.

4. The method according to claim 1, characterized in that, The preset display scoring function includes a single-mode scoring function and a combined-mode scoring function, and the candidate lighting strategy includes a single-mode lighting strategy and a combined-mode lighting strategy consisting of multiple single modes. The step of calculating the display score of each candidate lighting strategy according to a preset display score function, and taking the candidate lighting strategy corresponding to the maximum display score as the first lighting strategy, includes: The single-mode display score for each single-mode lighting strategy is calculated using the single-mode scoring function; the combined-mode display score for each combined-mode lighting strategy is calculated using the combined-mode scoring function. Under preset constraints, a preset search algorithm is used to select the candidate lighting strategy corresponding to the maximum value from all the single-mode display scores and the combined-mode display scores as the first lighting strategy. The constraints include one or more of total acquisition time, total number of exposures and device switching cost. The preset search algorithm includes one or more of exhaustive search, greedy search, bundle search, dynamic programming, genetic algorithm, Bayesian optimization and reinforcement learning.

5. The method according to claim 4, characterized in that, After selecting the candidate lighting strategy corresponding to the maximum display score as the first lighting strategy, the method further includes: The image of the optical film to be tested is acquired according to the first illumination strategy, and the acquisition result is obtained. Based on the collected results, calculate the abnormal manifestation gain of the first lighting strategy; Determine whether the abnormal display gain is greater than or equal to a preset gain threshold; If the abnormal display gain is greater than or equal to a preset gain threshold, then the first lighting strategy is used as the target lighting strategy. If the abnormality manifestation gain is less than a preset gain threshold, a second illumination strategy is selected from the multiple candidate illumination strategies to acquire an image of the optical film to be tested, and the abnormality manifestation gain is recalculated.

6. The method according to any one of claims 1 to 5, characterized in that, The anomaly-related features include one or more of the following: brightness features, contrast features, edge intensity features, texture stability features, saturation ratio features, and reflectivity ratio features.

7. An anomaly detection device based on multi-band multi-illumination modes, characterized in that, Applied to testing equipment, the device includes: The acquisition module is used to acquire image data of the optical film to be tested; An extraction module is used to extract anomaly-related features from the image data, wherein the anomaly-related features are used to characterize the difference in appearance between anomaly regions and the background in the image data; The generation module is used to generate multiple candidate lighting strategies based on the anomaly-related features; The selection module is used to calculate the display score of each candidate illumination strategy according to a preset display score function, and select the candidate illumination strategy corresponding to the maximum display score as the first illumination strategy to control the detection device to perform anomaly detection on the optical film to be detected.

8. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images