A Defect Detection Method and System for Cadmium Telluride Thin Film Modules Based on Visual Feature Fusion

By suppressing interference and enhancing local contrast in the surface images of cadmium telluride thin film modules, and combining the adaptive fusion of shallow and deep visual features, the problems of low-contrast defect feature extraction and inaccurate localization in the surface defect detection of cadmium telluride thin film modules are solved, and higher precision defect detection is achieved.

CN123118489APending Publication Date: 2026-11-17ZHONGSHENG ENERGY TECHNOLOGY (DAYE) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611256864.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-19
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing methods for detecting surface defects in cadmium telluride thin film modules suffer from difficulties in extracting low-contrast defect features, low defect localization accuracy, and susceptibility to imaging interference, making it difficult to meet the accuracy and automation requirements of industrial-scale batch testing.

Method used

By acquiring surface images of cadmium telluride thin film modules, the imaging state is determined based on local brightness distribution and texture response. Interference suppression and local contrast enhancement are performed. Shallow and deep visual features are extracted using a defect feature extraction network and adaptively fused to generate multi-scale defect representations. The defect region and type are determined by combining the spatial continuity of pixel positions and feature differences.

Benefits of technology

It improves the accuracy of low-contrast defect identification, enhances the ability to suppress imaging interference, improves the accuracy of defect localization, and generates more accurate surface defect detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118489A_ABST
    Figure CN123118489A_ABST
Patent Text Reader

Abstract

This application provides a method and system for defect detection of cadmium telluride thin-film modules based on visual feature fusion. The method includes: acquiring a surface image of the cadmium telluride thin-film module, determining the thin-film imaging state, and obtaining a defect-enhanced image; inputting the defect-enhanced image into a defect feature extraction network to extract shallow and deep visual features, generating a multi-scale defect representation; determining the defect response state at each pixel location, generating candidate defect regions, correcting the region boundaries based on the feature differences between the candidate defect regions and surrounding normal regions, and obtaining the target defect region; extracting regional defect features corresponding to the target defect region from the multi-scale defect representation, determining the defect type based on the matching state between the regional defect features and spatial attributes, determining the defect level based on the hazard assessment relationship corresponding to the spatial attributes and defect type, and generating a surface defect detection result. This application can improve the accuracy of low-contrast defect recognition, improve defect localization accuracy, and enhance imaging interference suppression capabilities.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of artificial intelligence and computer vision technology, and in particular to a method and system for defect detection of cadmium telluride thin film modules based on visual feature fusion. Background Technology

[0002] Cadmium telluride thin-film solar modules have advantages such as good low-light response performance, low manufacturing cost, and wide applicability, and have been gradually applied in distributed photovoltaics, building-integrated photovoltaics, and outdoor power generation. During the production, transportation, and operation of these modules, factors such as fluctuations in the thin-film deposition process, mechanical vibration, friction and collision, and environmental pollution can easily cause defects such as cracks, scratches, pinholes, abnormal coatings, and localized damage to the module surface. Therefore, accurate surface inspection is necessary to ensure module quality and subsequent operational reliability.

[0003] Current methods for detecting surface defects in cadmium telluride thin-film modules primarily rely on manual visual inspection or optical inspection methods based on traditional image processing. Manual inspection depends on human experience and visual judgment, resulting in low efficiency and consistency, and is prone to missed detections during long-term continuous inspections. Traditional optical inspection typically uses fixed threshold segmentation, edge response, or grayscale differences to identify abnormal areas, offering some detection capability for obvious defects.

[0004] However, cadmium telluride thin films exhibit imaging characteristics such as significant reflectivity and complex substrate texture. Defects like pinholes, shallow scratches, and uneven coating have weak visual differences from normal areas. Existing methods are easily affected by changes in illumination, surface reflection, and texture interference, making it difficult to stably extract low-contrast defect features. Furthermore, they lack effective feature representation and precise localization capabilities for defects of different scales and with blurred boundaries, leading to missed detections, false detections, and insufficient classification accuracy during defect detection. This makes it difficult to meet the requirements of industrial-scale batch testing for detection accuracy and automation. Summary of the Invention

[0005] In view of this, the present application provides a method and system for defect detection of cadmium telluride thin film modules based on visual feature fusion, in order to solve the problems of difficulty in extracting low-contrast defect features, low defect positioning accuracy, and susceptibility to imaging interference in the prior art.

[0006] A first aspect of this application provides a defect detection method for cadmium telluride thin-film modules based on visual feature fusion, comprising: acquiring a surface image of a cadmium telluride thin-film module; determining the thin-film imaging state based on the local brightness distribution and texture response of the surface image; and performing interference suppression and local contrast enhancement on the surface image according to the thin-film imaging state to obtain a defect-enhanced image; inputting the defect-enhanced image into a defect feature extraction network; extracting shallow visual features that preserve spatial details and deep visual features that characterize regional semantics; determining fusion weights based on the feature responses of each spatial location; adaptively fusing the shallow visual features and deep visual features to generate a multi-scale defect representation; determining the defect response state of each pixel position based on the multi-scale defect representation; generating candidate defect regions by combining the spatial continuity relationship of adjacent pixels; correcting the region boundaries based on the feature differences between the candidate defect regions and the surrounding normal regions to obtain a target defect region; extracting regional defect features corresponding to the target defect region from the multi-scale defect representation; determining the spatial attributes of the target defect region; determining the defect type based on the matching state of the regional defect features and the spatial attributes; determining the defect level based on the hazard judgment relationship corresponding to the spatial attributes and the defect type; and generating a surface defect detection result.

[0007] A second aspect of this application provides a defect detection system for cadmium telluride thin-film modules based on visual feature fusion, comprising: an acquisition module for acquiring a surface image of the cadmium telluride thin-film module, determining the thin-film imaging state based on the local brightness distribution and texture response of the surface image, and performing interference suppression and local contrast enhancement on the surface image according to the thin-film imaging state to obtain a defect-enhanced image; and an extraction module for inputting the defect-enhanced image into a defect feature extraction network, extracting shallow visual features that preserve spatial details and deep visual features that characterize regional semantics, determining fusion weights based on the feature responses at each spatial location, and fusion the shallow visual features and deep visual features. The system employs adaptive fusion to generate multi-scale defect representations. A determination module is used to determine the defect response state of each pixel location based on the multi-scale defect representations, generate candidate defect regions by combining the spatial continuity of adjacent pixels, and correct the region boundaries based on the feature differences between the candidate defect regions and the surrounding normal regions to obtain the target defect region. A generation module is used to extract regional defect features corresponding to the target defect region from the multi-scale defect representations, determine the spatial attributes of the target defect region, determine the defect type based on the matching state between the regional defect features and the spatial attributes, determine the defect level based on the hazard assessment relationship corresponding to the spatial attributes and defect type, and generate surface defect detection results.

[0008] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: By acquiring surface images of a cadmium telluride thin-film module, the thin-film imaging state is determined based on the local brightness distribution and texture response of the surface image. Interference suppression and local contrast enhancement are then applied to the surface image according to the thin-film imaging state to obtain a defect-enhanced image. This enhanced image is input into a defect feature extraction network to extract shallow visual features that preserve spatial details and deep visual features that characterize the semantics of the region. Fusion weights are determined based on the feature responses at each spatial location, and the shallow and deep visual features are adaptively fused to generate a multi-scale defect representation. The defect response state at each pixel location is determined based on the multi-scale defect representation. Candidate defect regions are generated by combining the spatial continuity of adjacent pixels. The region boundaries are corrected based on the feature differences between the candidate defect regions and the surrounding normal regions to obtain the target defect region. Regional defect features corresponding to the target defect region are extracted from the multi-scale defect representation, and the spatial attributes of the target defect region are determined. The defect type is determined based on the matching state between the regional defect features and the spatial attributes. The defect level is determined based on the hazard assessment relationship corresponding to the spatial attributes and defect type, generating a surface defect detection result. This application can improve the accuracy of low-contrast defect recognition, improve defect localization accuracy, and enhance imaging interference suppression capabilities. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of the defect detection method for cadmium telluride thin film modules based on visual feature fusion provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the cadmium telluride thin film module defect detection system based on visual feature fusion provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0012] In existing technologies, surface defect detection of cadmium telluride thin-film solar modules typically employs manual visual inspection or traditional optical image processing methods. Manual visual inspection relies on the experience and visual judgment of the inspectors, and is easily affected by factors such as fatigue during continuous batch inspections, resulting in low inspection efficiency and consistency. Traditional optical inspection usually utilizes fixed threshold segmentation, edge detection, or grayscale differences to identify abnormal areas on the module surface. It has a certain detection capability for defects with obvious visual differences, but its feature extraction methods are relatively fixed, limiting its adaptability to complex imaging environments.

[0013] Due to the significant reflectivity and complex substrate texture of cadmium telluride thin-film modules, and the weak visual difference between defects such as pinholes, shallow scratches, and localized coating anomalies and the surrounding normal areas, existing detection methods easily misidentify changes in illumination, surface reflection, or normal film texture as defects. Furthermore, they struggle to consistently preserve the detailed features of minute defects. For defects with significant scale differences or blurred edges, existing methods also suffer from inaccurate defect region segmentation. Therefore, current technologies face challenges in extracting low-contrast defect features, achieving low defect localization accuracy, and being susceptible to imaging interference.

[0014] To address the aforementioned technical problems, this application acquires surface images of cadmium telluride thin-film modules, determines the thin-film imaging state based on the local brightness distribution and texture response of the surface images, and applies interference suppression and local contrast enhancement to the surface images according to the thin-film imaging state to obtain defect-enhanced images. Based on this, a defect feature extraction network is used to extract shallow visual features that preserve spatial details and deep visual features that characterize regional semantics. Fusion weights are adaptively determined based on the feature responses at different spatial locations, and visual features at different levels are fused to form a multi-scale defect representation that simultaneously contains local detail information and regional semantic information.

[0015] Furthermore, this application determines the defect response state of each pixel location based on multi-scale defect characterization, and forms candidate defect regions using the spatial continuity between adjacent pixels. Then, it corrects the region boundaries based on the feature differences between the candidate defect regions and the surrounding normal regions to obtain the target defect region. After obtaining the target defect region, the corresponding regional defect features are extracted from the multi-scale defect characterization, and the defect type and corresponding defect level are determined by combining the spatial attributes of the target defect region. Finally, a surface defect detection result containing defect location, type, and level information is generated.

[0016] Through the above technical solutions, this application can specifically suppress interference such as reflection and texture based on the actual imaging state of the cadmium telluride thin film module, and enhance the effective characterization of low-contrast micro-defects through adaptive fusion of shallow spatial details and deep regional semantics; at the same time, it improves the accuracy of defect contour determination by utilizing the spatial continuity of pixel response and the feature differences on both sides of the defect region boundary. Therefore, this application can improve the accuracy of low-contrast defect identification, improve defect localization accuracy, and enhance the ability to suppress complex imaging interference.

[0017] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] Figure 1 This is a schematic flowchart of the cadmium telluride thin-film module defect detection method based on visual feature fusion provided in the embodiment. Figure 1 As shown, the method may specifically include: S101, acquire the surface image of the cadmium telluride thin film module, determine the thin film imaging state based on the local brightness distribution and texture response of the surface image, and perform interference suppression and local contrast enhancement on the surface image according to the thin film imaging state to obtain a defect-enhanced image. S102, input the defect enhancement image into the defect feature extraction network, extract the shallow visual features that preserve spatial details and the deep visual features that represent the semantics of the region, determine the fusion weights according to the feature responses of each spatial location, and adaptively fuse the shallow visual features and the deep visual features to generate a multi-scale defect representation. S103, determine the defect response state of each pixel position based on the multi-scale defect characterization, generate candidate defect regions by combining the spatial continuity relationship of adjacent pixels, and correct the region boundary based on the feature difference between the candidate defect regions and the surrounding normal regions to obtain the target defect region. S104: Extract regional defect features corresponding to the target defect area from the multi-scale defect characterization, determine the spatial attributes of the target defect area, determine the defect type based on the matching status of regional defect features and spatial attributes, determine the defect level based on the hazard judgment relationship corresponding to the spatial attributes and defect type, and generate surface defect detection results.

[0019] In some embodiments, acquiring a surface image of a cadmium telluride thin-film assembly and determining the thin-film imaging state based on the local brightness distribution and texture response of the surface image includes: Surface images of the cadmium telluride thin film module were acquired according to preset imaging conditions, and the surface images were divided into multiple local regions; Determine the brightness change state and texture response intensity of each local region, and determine the illumination perturbation state based on the brightness difference between adjacent local regions; By associating the texture response intensity with the corresponding illumination perturbation state, the texture response of the thin film substrate and the abnormal imaging response are distinguished, and imaging characterization of each local region is generated. By integrating the imaging characteristics of each local region, the thin film imaging state, which characterizes the degree of reflective interference and texture interference, is determined.

[0020] Specifically, the cadmium telluride thin-film solar module to be tested can be fixed on a horizontal testing fixture, ensuring that the module surface is substantially parallel to the imaging plane of the industrial camera. The industrial camera is then positioned at a fixed height above the module surface. An area-array industrial camera, in conjunction with a ring-shaped shadowless light source, can be used to acquire images. Before acquisition, the camera's field of view is calibrated according to the module specifications, ensuring partial overlap between adjacent acquisition fields of view to guarantee complete coverage of the module surface.

[0021] For the same batch of tests, the camera exposure time, gain, shooting distance, and basic brightness of the light source are kept constant. For example, the shooting distance is set to 600mm, the brightness of the ring light source is kept within a pre-calibrated stable range, and the angle between the camera optical axis and the normal of the component surface is controlled to be less than a preset angle, thereby reducing the difference in specular reflection caused by changes in shooting angle from the source. The system sequentially acquires high-definition images corresponding to each acquisition field of view according to the component number, and completes the field of view stitching according to the calibrated spatial mapping relationship to form a surface image covering the surface of the component to be tested.

[0022] After obtaining the surface image, it can be divided into multiple local regions with overlapping boundaries according to a uniform grid scale. For example, a basic local region can be formed using 256×256 pixels, with a 32-pixel overlap between adjacent local regions. For insufficiently sized regions located at the component edges, they can be supplemented by mirroring. The system calculates the average brightness, brightness dispersion, and local brightness gradient of each local region to form the brightness variation state.

[0023] In some examples, areas affected by reflections from a ring light source typically exhibit a significant increase in average brightness, while also forming a large brightness gradient between adjacent areas. Conversely, shadowed or poorly lit areas show a decrease in average brightness and a contraction in the grayscale dynamic range. The system compares the brightness difference between the current local area and its adjacent areas (up, down, left, and right) and determines the local brightness offset by combining this with the baseline brightness distribution of the entire surface image. When the brightness offset exceeds the adaptive allowable range for the corresponding batch, the area is marked as having illumination perturbation, and an illumination perturbation state is established based on the offset direction and duration.

[0024] Texture response intensity can be determined by combining multi-scale local gradient response and high-frequency energy. The system calculates edge responses in different directions within each local region and statistically analyzes the spatial distribution of local high-frequency components. When the texture response exhibits a continuous, repetitive, and directionally stable distribution over a large region, it can be considered as a texture response formed by the cadmium telluride thin film substrate itself. When the texture response exhibits local abrupt changes, isolated high responses, or continuous extension along non-periodic paths, it is further considered as a candidate for anomalous imaging responses.

[0025] For example, normal thin film regions may exhibit fine textures with similar orientations and stable amplitudes, while fine cracks typically form elongated and continuous local high-gradient responses, and pinholes usually form isolated responses with small areas and abrupt changes at the center. For highly reflective regions, if the texture response increases synchronously with local brightness and lacks stable boundaries, it is preferentially classified as a spurious response generated by illumination perturbation.

[0026] The system further correlates texture response intensity with the illumination perturbation state of the corresponding local region, calculating the independence of anomalous responses relative to brightness changes. For texture responses that change synchronously with brightness perturbations, their anomalous contribution is reduced; for texture responses that still exhibit significant abrupt changes under basically stable brightness conditions, their anomalous contribution is increased, thereby obtaining a local imaging characterization that includes brightness stability, texture interference level, and anomalous response reliability. Subsequently, the local imaging characterizations are fused according to the spatial location of each local region on the component surface. The reflective interference range is determined for continuous high-brightness areas, and the texture interference range is determined for periodic or continuous substrate responses. The interference intensity of each region is then combined to form the thin-film imaging state.

[0027] In some examples, the thin-film imaging state can be stored using a state matrix corresponding to the image space, allowing subsequent image preprocessing to call different interference suppression intensities and local contrast enhancement intensities for different regions. In this way, before entering the defect feature extraction network, it is possible to distinguish between stable background responses formed by thin-film material and light reflection and anomalous responses that may correspond to real defects, providing more stable imaging input for low-contrast defects such as shallow scratches, minute pinholes, and local coating anomalies.

[0028] In another optional implementation, a baseline brightness map and a baseline texture map can be established using a reference component without obvious defects before the start of each inspection batch. The current local area is then differentiated from the baseline area at the same spatial location to correct the overall offset caused by light source aging or batch color changes of the component. If the brightness difference in a certain area is large but the texture difference is small, the weight of reflective interference is increased; if the brightness difference is small but the local texture deviation is significant, the weight of abnormal response is increased.

[0029] For continuous responses spanning multiple local regions, corresponding states can be merged based on the consistency of responses within overlapping areas to avoid repeated judgments of the same defect due to region division. Ultimately, a state vector containing reflection interference values, texture interference values, and anomalous response confidence values ​​can be formed for each local region, and written into the thin film imaging state according to the original spatial location, providing direct parameter basis for subsequent regional enhancement.

[0030] In some embodiments, interference suppression and local contrast enhancement are performed on the surface image according to the thin film imaging state to obtain a defect-enhanced image, including: Determine the interference suppression intensity and contrast enhancement intensity of each local region based on the thin film imaging status; The abnormal brightness response of the corresponding local area is corrected according to the interference suppression intensity, and the image components that match the texture response of the thin film substrate are suppressed to obtain the corrected image; Adjusting the local grayscale distribution in the corrected image based on the contrast enhancement intensity amplifies the response difference between the anomalous imaging response and the neighboring region. The adjusted local regions are continuously fused to obtain a defect-enhanced image.

[0031] Specifically, after obtaining the thin film imaging state corresponding to the spatial location of the surface image, the system reads the reflection interference value, texture interference value, and anomaly response confidence value corresponding to each local region, and normalizes each state value to a unified range. For local regions with high reflection interference values, the anomalous brightness suppression ratio is increased; for local regions with high texture interference values, the substrate texture suppression ratio is increased; and for local regions with high anomaly response confidence values, excessive smoothing is limited and the local contrast enhancement amplitude is increased, so that the edges and grayscale abrupt changes of real defects can be preserved.

[0032] For example, when a local area produces continuous high-brightness reflections due to a ring light source and the texture response increases synchronously with the brightness, the interference suppression intensity can be set to a higher level than that of ordinary areas, while the contrast enhancement intensity is controlled in a lower range to avoid the formation of pseudo-defects after the high-brightness area is enhanced; when the brightness of a local area is stable but there are abnormal responses corresponding to isolated pinholes or fine scratches, a lower interference suppression intensity and a higher contrast enhancement intensity can be used.

[0033] After determining the parameters, the system first performs abnormal brightness correction on each local area. A brightness compensation amount is constructed based on the basic brightness difference between the local area and its neighboring areas. Continuously bright areas are compressed towards the brightness center of the adjacent normal area, while local dark areas are compensated towards their corresponding normal brightness range, thus preserving local gradient abrupt changes. For areas with significant reflective interference, low-frequency illumination components can be estimated based on the aforementioned illumination disturbance state and separated from the original grayscale response, thereby weakening background fluctuations caused by slow changes in illuminance. The brightness correction magnitude is constrained by the interference suppression strength to avoid simultaneously eliminating genuine local grayscale differences such as shallow scratches and tiny pinholes.

[0034] In some examples, for thin-film substrate texture, the system establishes a regional texture reference using the substrate texture response distribution determined in the previous processing stage. The local image can be decomposed into response components with different spatial frequencies. The matching degree between each response component and the texture reference in terms of direction, periodicity, and response intensity is compared. Image components with a high degree of matching and continuous occurrence over a large spatial range are attenuated, while local abrupt changes, aperiodic extensions, or isolated concentrated responses are preserved. For example, regular fine stripes formed by a normal thin film of the component can be suppressed as a stable background response, while crack responses extending across the normal texture direction and locally isolated pinhole responses do not attenuate synchronously with the substrate texture. After abnormal brightness correction and texture suppression, a corrected image is formed.

[0035] Subsequently, the local grayscale distribution of the corrected image is adjusted based on the contrast enhancement intensity corresponding to each local region. Specifically, the local grayscale histogram distribution can be statistically analyzed, and the effective grayscale stretching range can be determined according to the confidence value of the anomalous response. The mapping slope is increased for grayscale intervals containing confident anomalous responses, and the mapping amplitude is decreased for grayscale intervals that are close to the background brightness and recur. For low-contrast shallow scratches, the grayscale spacing between scratch pixels and adjacent normal film pixels can be increased; for tiny pinholes, the local response difference between the center and the surrounding area of ​​the hole can be enhanced; for areas with uneven coating, a slowly changing anomalous distribution is maintained over a large range to avoid only enhancing the edges and losing regional defect information. Local enhancement can also set a maximum gain constraint to prevent the acquisition noise from being amplified synchronously during the contrast enhancement process.

[0036] Since each local region uses processing parameters corresponding to the thin-film imaging state, the brightness and contrast of adjacent regions may exhibit abrupt boundary changes. Therefore, the system performs continuous fusion within the overlapping region range. For cases where the same pixel is covered by multiple local regions, the fusion coefficient is determined based on the distance of the pixel from the center of each local region and the corresponding anomalous response confidence value, and the processing results are weighted and combined. For locations where there are still significant gray-level jumps at the region edges, neighborhood transition constraints can be used to further smooth the background response between regions, while maintaining the high gradient structure at the defect boundary. After completing the fusion of all local regions, the complete image is reconstructed according to the original spatial location to obtain the defect-enhanced image.

[0037] Furthermore, for defects spanning multiple local regions, the system can examine the directional continuity and grayscale variation trend of anomalous responses within overlapping regions. When anomalous responses in adjacent regions have a continuous extension relationship, similar enhancement parameters are used while maintaining the corresponding response amplitude to prevent the same crack or scratch from being weakened or truncated at the region splicing location. For high-frequency variations that only appear at the edge of a single region and are inconsistent with the responses of neighboring regions, their enhancement contribution is reduced by combining the confidence value of the anomalous response.

[0038] In another optional implementation, the interference suppression intensity and contrast enhancement intensity can be dynamically adjusted based on the reference components of the detection batch. When the overall reflectance brightness of a batch of components increases, the brightness compensation ratio of the corresponding area is increased, instead of uniformly increasing the enhancement amplitude of all areas; when the substrate texture of the component surface is significantly enhanced compared to the previous batch, only the suppression degree of the component matching the texture reference is increased.

[0039] This regionalized and adaptive processing method enables the enhancement of defect images to reduce the effects of reflection, substrate texture and non-uniform lighting, while maintaining the subtle edges, local grayscale differences and regional morphology corresponding to the real defects, providing stable input for subsequent shallow and deep visual feature extraction.

[0040] In some embodiments, the defect-enhanced image is input into a defect feature extraction network to extract shallow visual features that preserve spatial details and deep visual features that characterize the semantics of the region, including: Hierarchical convolutional coding is performed on the defect enhancement image to obtain the feature responses corresponding to different network layers; High-resolution features that preserve defect edges and local grayscale changes are extracted from the previous network layers to generate shallow visual features; The feature receptive range is expanded along subsequent network layers, and spatially correlated feature responses are aggregated to generate deep visual features; Establish the spatial correspondence between shallow visual features and deep visual features according to the network hierarchy.

[0041] Specifically, the defect-enhanced images, after reflection suppression, substrate texture reduction, and local contrast enhancement, are uniformly adjusted to the input size required by the defect feature extraction network and normalized according to a pre-statistical image grayscale range to ensure consistent input scale for images corresponding to different components and different acquisition fields of view. The defect feature extraction network can use a lightweight convolutional neural network as the backbone encoding network, which, while ensuring the inspection speed of industrial production lines, forms multi-layer features that gradually change from high spatial resolution to high semantic level through continuous convolutional encoding. During the training phase, supervised training is conducted using labeled defect samples such as scratches, cracks, pinholes, coating anomalies, and edge defects. The enhanced samples, after rotation, flipping, cropping, and brightness fine-tuning, are used to improve the network's adaptability to component batch variations and imaging disturbances.

[0042] In the hierarchical convolutional coding process, the preceding convolutional units first perform feature calculations within the local receptive range on the defect-enhanced image. For example, the input image is convolved sequentially to form the first and second level feature maps, extracting gray-level gradients, edge directions, and local texture changes within the pixel neighborhood while maintaining high spatial resolution.

[0043] In some examples, for fine cracks with a width occupying only a few pixels, the pre-processor network can preserve high-gradient responses that appear continuously along the crack's extension direction; for pinhole defects, it can form point-like responses dominated by local central anomalies; for shallow scratches, it can preserve weak but continuous grayscale variations between the scratch and the normal film area. The system selects feature maps with larger spatial dimensions from the pre-processor network layers, normalizes and compresses the channel responses to form shallow visual features, thereby avoiding premature attenuation of minute defects during subsequent downsampling.

[0044] Subsequently, shallow features are input into subsequent convolutional units, where straddle convolutions or pooling are used to gradually reduce the spatial size and expand the image receptive range corresponding to individual feature locations. As the network depth increases, scattered local responses within the same region are gradually aggregated, enabling the network to combine the thin film state around the defect to determine the semantics of the region corresponding to the abnormal response.

[0045] For example, for localized uneven coating, relying solely on single-pixel grayscale differences is insufficient to distinguish them from changes in illumination. Subsequent networks can combine consistent shifts in color or grayscale distribution over a larger area to form regional anomalous responses. For cracks, multiple local edge fragments can be aggregated into semantic responses with continuous morphology. For normal substrate textures, a larger receptive range is used to identify repetitive distribution features and reduce the semantic response to defects. The system selects feature maps with relatively stable defect semantic discrimination capabilities from the subsequent networks, and generates deep visual features after channel aggregation.

[0046] To balance detection accuracy and network computation, some convolutions in conventional convolutional units can be replaced with lightweight convolutional structures. The channel size can be progressively expanded layer by layer based on the number of input feature channels, allowing the network to prioritize preserving spatial information in the earlier stages and gradually improve semantic representation in later stages. For downsampling, the number of consecutive downsampling operations can be limited to prevent tiny targets such as pinholes and fine cracks from completely disappearing from the feature map. During training, auxiliary supervision can be set in both shallow and deep layers to ensure that shallow features retain defect edge responses and that deep features enhance defect category-related responses, thus forming a feature extraction state with a clear hierarchical division of labor.

[0047] Because shallow and deep visual features have different spatial dimensions, the system further records the downsampling ratio and feature location mapping relationship of each network layer. For example, a feature location in a shallow visual feature corresponds to a small pixel region in a defect-enhanced image, while the corresponding location in a deep visual feature covers a larger receptive area including that region. The system reverse-maps the deep feature locations to the input image coordinates according to the cumulative step size of each network layer and establishes a corresponding index between them and the shallow feature locations. For locations that cannot be directly matched one-to-one, the corresponding feature range can be determined based on spatial proximity, and interpolation or scaling transformation can be used to align features at different levels under a unified spatial reference.

[0048] For example, when the size of the shallow feature map is 1 / 4 of the input image and the size of the deep feature map is 1 / 16 of the input image, the deep visual features can be upsampled to the shallow scale, and spatial correspondences can be established according to the same receptive centers in the original image. For shallow high-response locations located at the edge of a shallow scratch, semantic responses in the deep network covering the scratch and the surrounding normal film region can be associated; for the center of a pinhole, deep responses containing the local morphology of the pinhole can be associated.

[0049] Ultimately, we obtain shallow visual features that preserve the spatial details of defects, deep visual features that represent the semantics of regions, and the spatial correspondence between the two, which provides a foundation for subsequent adaptive fusion based on the feature contributions from different spatial locations and the generation of multi-scale defect representations.

[0050] In some embodiments, fusion weights are determined based on the feature responses of each spatial location, and shallow visual features are adaptively fused with deep visual features to generate a multi-scale defect representation, including: The shallow visual features and deep visual features are scale-aligned according to their spatial location correspondence. Determine the differences in local feature responses at each spatial location and the degree of cross-layer semantic consistency, and generate the corresponding feature contribution states; The fusion weights of shallow and deep visual features are determined based on the feature contribution status. Multi-scale defect characterization is generated by aggregating scale-aligned features according to fusion weights and preserving spatial details with anomalous responses.

[0051] Specifically, after obtaining shallow visual features, deep visual features, and their spatial correspondence, the system uses the spatial resolution of the shallow visual features as a unified fusion scale to perform scale alignment on features at different network layers. For deep visual features with spatial dimensions smaller than the shallow visual features, interpolation upsampling is performed according to the network downsampling ratio, and the feature center position is corrected based on the aforementioned spatial correspondence. In cases where the number of channels differs between shallow and deep visual features, they can be converted to a unified feature dimension through channel mapping units. During scale alignment, the correspondence between each feature position and the original region in the defect enhancement image is maintained, ensuring that the responses of the same fine crack, pinhole, or coating anomaly generated in different network layers can be mapped to the same spatial location.

[0052] After scale alignment, the system analyzes the local response changes of shallow visual features at each spatial location. Specifically, it compares the feature response of the current spatial location with the average response of neighboring locations to determine the differences in local feature responses. If the current location exhibits significant gradient abrupt changes, grayscale anomalies, or texture deviations relative to its neighborhood, the corresponding local feature response differences are substantial. For example, although shallow scratches show weak overall grayscale changes, they can create continuous local response deviations along the scratch's extension direction; tiny pinholes exhibit concentrated response differences between their central location and the surrounding normal film area. For repetitive responses formed by normal substrate textures, the differences in local feature responses remain within a relatively stable range due to the relatively stable periodic changes at adjacent locations.

[0053] The system simultaneously extracts deep visual features at the same location based on spatial correspondence and compares the correlation between shallow anomalous responses and deep semantic responses to determine the degree of cross-layer semantic consistency. For example, when a shallow visual feature forms a thin edge response at a certain location, and the corresponding deep visual feature also forms a crack-related semantic response over a larger area encompassing that location, the degree of cross-layer semantic consistency is considered high. Conversely, when a shallow layer exhibits a localized bright response, and the deep features, combined with the surrounding area, determine that it belongs to a continuous reflective region, the degree of cross-layer semantic consistency is low. For regional defects such as uneven coating, although the local edge response may not be prominent, deep visual features can form stable anomalous semantics over a large spatial range. Therefore, the deep response can be used to compensate for insufficient shallow detail responses.

[0054] Furthermore, the system transforms local feature response differences and cross-layer semantic consistency into a unified evaluation range and jointly generates feature contribution states for each spatial location. Feature contribution states can be used to characterize whether the current location relies more on shallow spatial details or deeper regional semantics. When local response differences are significant and cross-layer semantic consistency is high, the location is considered to have a high degree of defect confidence, while simultaneously preserving both shallow details and deep semantics. When local responses are prominent but cross-layer semantic support is weak, the direct contribution of shallow visual features is reduced to suppress false anomalies caused by acquisition noise, film texture, or local reflections. When local responses are weak but deep regional semantics remain abnormal, the contribution of deep visual features is increased, ensuring that low-boundary saliency defects such as coating inhomogeneity can still form an effective characterization.

[0055] The system calculates fusion weights for shallow and deep visual features based on their contribution states, and normalizes the weights at the same spatial location to ensure comparability of contributions from different feature levels. For example, for narrow crack regions, the fusion weights for shallow visual features can be increased to maintain crack edges and continuity; for larger coating anomaly regions, the fusion weights for deep visual features can be increased to maintain the overall anomaly semantics of the region; for pinhole regions, both types of weights are assigned simultaneously based on the consistency between the point-like local response and the semantics of the deep defect. The fusion weights can vary with spatial location, rather than using a fixed ratio across the entire image, thus adapting to situations where defects of different scales exist simultaneously on the surface of the same component.

[0056] During feature aggregation, the system weights the scale-aligned shallow and deep visual features position by position according to the fusion weights, and sets detail preservation constraints for shallow features with high anomalous responses. When a shallow anomalous response at a certain spatial location continuously exceeds the neighborhood background response and obtains deep semantic support, the attenuation of that response during the fusion process is limited to prevent fine cracks, shallow scratches, and pinholes from disappearing due to the smoothing effect of deep features. For anomalous responses with continuous extension relationships between adjacent locations, response propagation can also be combined with spatial neighborhood, so that the same defect forms a continuous region in the multi-scale defect representation.

[0057] For example, when a component under inspection simultaneously has a shallow scratch and a localized coating unevenness, the shallow scratch location is provided by shallow visual features with a high proportion of edge details, while deep visual features provide semantic confirmation of the defect. Similarly, the coating unevenness location is provided by deep visual features with a high proportion of regional anomaly information, while shallow visual features supplement local boundary changes. After adaptive aggregation of all spatial locations, a multi-scale defect representation is formed. This representation retains the spatial details corresponding to minute defects and also includes regional semantics to distinguish real defects from thin film textures and reflective interference, providing a unified feature basis for subsequent pixel-level defect response determination and defect region segmentation.

[0058] In some embodiments, determining the fusion weights of shallow visual features and deep visual features based on feature contribution states includes: Based on the degree of deviation between the shallow visual features of each spatial location and the response of neighboring locations, the significant state of local anomalies is determined; Based on the degree of response correlation between shallow and deep visual features at corresponding spatial locations, the cross-layer feature consistency state is determined. By jointly mapping local anomaly states with cross-layer feature consistent states, the feature contribution ratio of the corresponding spatial location is determined. The fusion weights of shallow and deep visual features are adjusted according to the feature contribution ratio, so that features that are significantly abnormal and have consistent cross-layer responses maintain their corresponding feature responses in multi-scale defect representation.

[0059] Specifically, the system takes the scale-aligned shallow and deep visual features as input, establishes a local neighborhood for each spatial location, for example, selecting a 3×3 or 5×5 feature window centered on the current location, and calculates the response difference between the shallow visual features at the current location and the average features at neighboring locations. The response difference can be normalized by combining feature vector distance, gradient change magnitude, and directional continuity to form the degree of response deviation.

[0060] In some examples, for normal thin film substrates, local textures typically exhibit periodic or slow variations in adjacent locations, with low response deviation. For fine cracks, shallow scratches, or pinholes, a more pronounced local abrupt change occurs between the anomalous location and the surrounding normal thin film. The system further modifies the response deviation by incorporating the continuous range and concentration of the anomalous response within the neighborhood, generating a significantly salient local anomaly state that distinguishes isolated noise from real defects with stable spatial structures.

[0061] For example, when a shallow scratch continuously spans multiple feature locations horizontally, although adjacent locations all exhibit anomalous responses, there is still a stable response difference between them and the normal area along the scratch normal. The system can enhance the corresponding local anomalous salience state based on directional continuity. For a single random noise point, although its response may be high, there is no continuous anomalous structure in the surrounding area, thus reducing the corresponding salience. For pinholes, a local anomalous salience state can be formed based on the radial difference between the high response at the center and the surrounding neighborhood. For high-brightness reflections caused by a ring light source, since the bright area usually changes smoothly over a large range, the degree of neighborhood deviation is relatively limited. Even if the local grayscale is high, a high anomalous salience state will not be directly obtained.

[0062] Subsequently, the system obtains deep visual features at the same location based on spatial correspondence, and performs channel mapping between shallow and deep visual features to place the two types of features in a comparable space. The system can then calculate the similarity of response directions, the degree of overlap of anomalous activations, and the consistency of neighborhood distribution for the two types of features, and fuse the calculation results into a response correlation degree.

[0063] In some examples, if a shallow layer exhibits a thin, elongated edge response, while deep features form a stable defect semantic activation within the corresponding receptive region, the response correlation is high. Conversely, if a shallow layer shows a strong local response, but deep features characterize the corresponding region as a normal film or illumination perturbation, the response correlation is low. The system can also compare the correlation results of adjacent locations. When multiple consecutive locations maintain the same cross-layer response relationship, its reliability is improved, forming a consistent state of cross-layer features.

[0064] Furthermore, after obtaining the locally significant anomaly state and the cross-layer feature consistent state, the system performs joint mapping through a pre-trained weight mapping unit or parameterized mapping function. Specifically, the two states can be normalized and combined into a two-dimensional contribution input, and the feature contribution ratio corresponding to different state combinations can be learned based on the loss feedback corresponding to real defects and pseudo-anomalies in the training samples.

[0065] In some examples, when both the local anomaly saliency state and the cross-layer feature consistency state are high, the location is determined to have a high probability of being a real defect, thus increasing the joint contribution of shallow details and deep semantics. When the local anomaly saliency state is high but the cross-layer feature consistency state is low, the direct entry of shallow high responses into the fusion result is restricted to reduce false detections caused by noise, reflective edges, or local texture abrupt changes. When the local anomaly saliency state is low but the cross-layer feature consistency state is high, the contribution of deep visual features is increased to preserve defects with indistinct coating unevenness boundaries but regional semantic anomalies.

[0066] The feature contribution ratio can be further converted into shallow fusion weights and deep fusion weights, and the two types of weights at the same spatial location can be normalized. Taking a fine crack as an example, if the local anomaly significance is 0.86 and the cross-layer feature consistency is 0.81, the shallow fusion weight can be set to a relatively high level, while giving the deep visual features sufficient semantic confirmation weight. For large-scale coating unevenness, when the local anomaly significance is only 0.42 and the cross-layer feature consistency reaches 0.88, the deep fusion weight can be increased. The specific values ​​are for illustrative purposes only, and the actual weights can be dynamically determined by the mapping parameters obtained through training.

[0067] Furthermore, after adjusting the fusion weights according to the feature contribution ratio, the system performs position-wise weighted aggregation on the two types of visual features after scale alignment. For positions with significant anomalies and consistent cross-layer responses, a minimum response preservation constraint can be set to ensure that the intensity of the fused feature is not lower than a preset proportion of the corresponding shallow layer's effective anomaly response, thus avoiding excessive smoothing of small targets such as pinholes and fine cracks by multi-layer feature fusion. For continuous defects, the variation amplitude of adjacent fusion weights can also be constrained along the direction of the anomaly response extension, ensuring that the same defect maintains a continuous response in multi-scale defect representation.

[0068] In another optional implementation, the system can use manually labeled real defect regions and normal regions in the training set to jointly train the weight mapping unit, so that the local abnormality salience state of the real defect location and the cross-layer feature consistency state jointly promote the enhancement of the fusion response, while the fusion response of the corresponding locations of reflection, base texture and random noise is suppressed.

[0069] Furthermore, after training, when faced with components from different production batches or with different surface reflection states, the system does not need to use a fixed shallow-to-deep layer fusion ratio, but instead dynamically allocates feature contributions based on the actual feature state of the current location. Through this method, multi-scale defect representation can simultaneously preserve the edge details and regional semantics of minute defects, and reduce pseudo-anomaly responses caused by local highlighting, repetitive textures, and random noise, providing a more stable feature basis for subsequent defect pixel determination and region boundary definition.

[0070] In some embodiments, the defect response state of each pixel location is determined based on multi-scale defect characterization, candidate defect regions are generated by combining the spatial continuity relationship of adjacent pixels, and the region boundaries are corrected based on the feature differences between the candidate defect regions and the surrounding normal regions to obtain the target defect region, including: Multi-scale defect representations are mapped to image space, and the defect response state is determined based on the intensity of abnormal features corresponding to each pixel location. Based on the similarity of responses and spatial connectivity between adjacent pixels, pixels that meet the defect response conditions are aggregated to generate candidate defect regions. A normal reference region is determined along the periphery of the candidate defect region. The multi-scale defect characterizations on both sides of the region edge are compared to determine the boundary difference state. The edge pixel assignments of candidate defect regions are adjusted based on the boundary difference status, and the spatial continuity of the adjusted regions is verified to obtain the target defect region.

[0071] Specifically, after obtaining multi-scale defect representations, the system first extracts the scale mapping relationship corresponding to the network based on the defect features, restoring the multi-scale defect representations to an image space consistent with the defect enhancement image. For feature representations with spatial resolution lower than the original image, a combination of interpolation upsampling and convolution thinning can be used to restore pixel-level features, and the edge positions preserved in the shallow visual features are used to correct the position of the upsampling results, so that minute defects such as fine cracks and pinholes maintain their original spatial positions during the feature mapping process.

[0072] The system calculates the intensity of abnormal features for each restored pixel location. The intensity of abnormal features is determined by the degree of deviation of the response at that location from the neighboring normal feature distribution and the degree of defect activation after multi-scale fusion, and is normalized to a uniform range. When the intensity of abnormal features exceeds the defect response boundary of the current image, the pixel is identified as a defect response pixel; for pixels near the response boundary, their continuous response state is retained for subsequent region aggregation judgment.

[0073] Furthermore, after determining the defect response state at each pixel location, the system uses the defect response pixel as the initial node to analyze the response similarity and spatial connectivity between adjacent pixels. The response similarity can be determined based on the distance between the corresponding feature vectors of adjacent pixels and the difference in the intensity of abnormal features, while the spatial connectivity is determined based on the adjacency relationship of pixels in the horizontal, vertical, and diagonal directions.

[0074] In some examples, when adjacent pixels all meet the defect response conditions and the response differences are within an acceptable range, the corresponding pixels are grouped into the same connected region. For fine cracks and shallow scratches, the connectivity tolerance along the direction of abnormal extension can be appropriately increased to maintain regional continuity even when there are slight discontinuities in the local response; for pinholes, pixels with radial correlation around the central response can be preferentially aggregated. After connectivity aggregation, one or more candidate defect regions are formed, and isolated responses with too small an area and lacking neighborhood support are filtered out.

[0075] For example, when a low-contrast fine crack exists on the surface of a component, a strong anomalous response usually forms at the center of the crack, while the pixel response near the crack tip and edge is relatively weak. If only a fixed response threshold is used, the crack area is prone to breakage. The system can use confirmed defect response pixels as seeds to determine the continuity of neighboring boundary response pixels. When adjacent pixels have similar feature directions to the seed region and extend continuously in space, the corresponding weak response pixels are added to the candidate defect region, thereby obtaining a more complete initial crack range. For large-area gradually varying high-brightness regions generated by ring light source reflection, although the pixels are spatially continuous, the anomalous features lack stable defect semantic support, and their defect response contribution can be reduced during the region aggregation stage.

[0076] Furthermore, after the candidate defect regions are formed, the system expands outward by a preset pixel width along the periphery of each candidate defect region to form a ring-shaped neighborhood, and removes other candidate defect regions and pixels with obvious abnormal responses from this ring, leaving the remaining area as a normal reference region. The normal reference region is used to characterize the actual background state of the cadmium telluride film around the current defect, thus adapting to changes in brightness and substrate texture at different locations. The system extracts multi-scale defect representations from the inner pixels of the candidate defect region's edge and the corresponding outer pixels in the normal reference region, comparing the degree of change in abnormal response intensity, feature direction, and regional semantics on both sides to generate a boundary difference state. The more obvious the boundary difference, the more likely the current location belongs to the true defect contour; if the feature changes on both sides of the edge are small, it indicates that the candidate region may be over-expanded.

[0077] Subsequently, the system adjusts the edge pixel assignment of candidate defect regions based on boundary difference status. For positions where the inner edge pixels differ little from the normal reference region, the corresponding boundary can be gradually shrunk; for positions where the outer edge pixels, although initially insufficient in response, have high feature consistency with the region's interior and significant differences from the normal reference region, they can be added to the defect region. Taking shallow scratches as an example, the edges on both sides of the scratch may not be fully covered by the initial response conditions due to a smooth grayscale transition. By comparing the features inside and outside the edges, the system can include transition pixels with continuous abnormal textures in the scratch region; for uneven coating, the boundary can be adjusted according to the semantic changes of the region over a larger area to avoid retaining only locally high-response positions.

[0078] After boundary adjustment, the system further performs spatial continuity verification. Specifically, it checks whether there are narrow voids formed by response fluctuations within the region, whether the same continuous defect is divided into multiple closely spaced regions, and whether isolated protrusions appear on the adjusted boundary. Narrow voids with internal features consistent with the surrounding defect regions are filled, adjacent regions with continuous orientation and spacing that meet the connection conditions are merged, and isolated edge pixels lacking neighborhood feature support are removed.

[0079] In some examples, for locations with natural structural boundaries, such as component corners, the effective detection area of ​​the component can be combined to limit boundary expansion and avoid identifying the background outside the component as a defect. After boundary correction and spatial continuity verification, the target defect region is generated, and the pixel mask, contour position, and region identifier corresponding to the target defect region are recorded, providing a unified regional basis for subsequent region defect feature extraction, spatial attribute calculation, defect type identification, and hazard level determination.

[0080] In some embodiments, extracting regional defect features corresponding to the target defect region from the multi-scale defect representation and determining the spatial attributes of the target defect region includes: Based on the location of the target defect region in the image space, the feature response of the corresponding region is extracted from the multi-scale defect representation; Spatial aggregation of feature responses yields regional defect features that characterize the distribution of features within the target defect region and the changes in its boundaries. The region outline is determined based on the pixel distribution of the target defect region, and the corresponding size and position are determined by combining the image scale conversion relationship. By integrating the region outline, size status, and location status, spatial attributes of the target defect region are generated.

[0081] Specifically, after completing the boundary correction and spatial continuity verification of the target defect region, the system establishes an independent region identifier for each target defect region and records the corresponding pixel mask and its coordinate range in the defect enhancement image. Based on the position of the target defect region in the image space, the pixel coordinates are transformed to the feature coordinates corresponding to the multi-scale defect representation according to the aforementioned feature scale mapping relationship, and the feature response covering the target defect region is extracted from the multi-scale defect representation.

[0082] To avoid the loss of region edge information due to coordinate rounding or scale transformation, a feature neighborhood of a preset width can be extended around the target defect region, ensuring that the intercepted range simultaneously includes the internal response of the defect and the response of adjacent normal films. For example, for fine cracks, 2 to 4 feature locations can be extended along the outer edge of the crack contour; for larger coating anomalies, the relative extension ratio can be determined according to the region scale. When multiple target defect regions are adjacent to each other, the responses of other defect regions are masked by region identifiers during the extension process to avoid feature mixing between different defects.

[0083] Furthermore, after obtaining the feature responses of the corresponding regions, the system distinguishes the interior, boundary, and surrounding reference parts of the target defect region based on the pixel mask, and performs spatial aggregation for each. For the interior region, the average response, maximum anomalous response, and spatial response dispersion of each feature channel can be calculated to characterize the overall state of gray-level anomalies and texture changes within the defect. For the boundary region, feature changes on both the inner and outer sides can be extracted according to the contour direction to determine the edge response intensity and continuity of change. For defects with a clear extension direction, continuous feature responses can be aggregated along the main extension direction to preserve the elongated features corresponding to cracks and scratches. The system correlates the internal aggregation results with the boundary change results to generate regional defect features, thus enabling the regional defect features to simultaneously include the internal response distribution of the defect and the boundary changes between the defect and the normal film.

[0084] For example, pinhole defects typically exhibit small-area, concentrated anomaly characteristics within the region, while the boundary shows a closed variation forming around the central anomaly. Spatial aggregation can create a region defect feature with both concentration and closure. For shallow scratches, the anomaly response extends along a main direction, with continuous but weak feature differences on both sides of the boundary, maintaining directional continuity during aggregation. For uneven coating, the range of response variation within the region is large, while the boundary may not be obvious, thus increasing the contribution of internal feature distribution to the regional defect feature. Therefore, even if the local grayscale values ​​of different types of defects are similar, differences can be formed through the joint characterization of the region's interior and boundary.

[0085] Furthermore, the system extracts the outermost connected pixels based on the pixel mask of the target defect region, generates closed or open region contours according to pixel adjacency order, and smooths local jagged edges and isolated protrusions. For narrow defects such as cracks and scratches, the principal axis direction and maximum extension range along the principal axis can be determined based on the region contour; for pinholes or local defects, the region occupancy scale can be determined based on the contour's enclosed area. The system utilizes the scale conversion relationship established during the image acquisition stage to convert the pixel scale into the actual scale of the component surface. The scale conversion relationship can be obtained in advance through a calibration plate, for example, by determining the actual length corresponding to each pixel, and then calculating the actual area and feature size of the defect based on the number of region pixels, contour span, and calibration ratio to form the dimensional state.

[0086] In some examples, the location status can be determined based on the center position of the target defect area, the contour coverage, and the spatial relationship between the effective detection areas of the component. Specifically, based on the component coordinate system established during image stitching and camera calibration, the center pixel of the defect area can be converted into the actual position on the component surface, and the relative position between the defect and the component edge and functional area can be calculated. For example, if a crack is located in the middle of the effective area of ​​the component, its center coordinates and extension direction can be recorded; if a corner defect intersects with the outer contour of the component, its contact range with the component edge can be recorded simultaneously. For defects spanning multiple acquisition fields of view, they are uniformly converted to the same component coordinate system according to the field stitching relationship to avoid position duplication or offset due to different local image coordinates.

[0087] Finally, the system associates the region outline, size, and location status according to the region identifier to form the spatial attributes of the target defect region. Spatial attributes may include outline representations describing the region's geometric boundaries, size representations describing the region's occupied area, and location representations describing the distribution position on the component surface; it does not require pre-approved fixed geometric shapes for different defects. For cases where multiple defects exist on the same component, a one-to-one correspondence is established between the region defect features and spatial attributes for each. Therefore, subsequent defect type determination can not only utilize the visual semantics in multi-scale defect representations but also combine the defect's regional morphology, actual scale, and distribution location for matching, providing quantifiable spatial evidence for determining the defect hazard level.

[0088] In some embodiments, the defect type is determined based on the matching status of regional defect characteristics and spatial attributes, the defect level is determined based on the hazard assessment relationship corresponding to the spatial attributes and defect type, and surface defect detection results are generated, including: Map the regional defect features to a preset defect feature space to determine the feature matching status between the regional defect features and each candidate defect type. The feature matching status is corrected by combining spatial attributes, and the defect type of the target defect area is determined based on the corrected matching status. Obtain the hazard assessment relationship corresponding to the defect type, match the spatial attributes with the hazard assessment relationship, and determine the defect level of the target defect area; By associating defect type, defect level, and spatial attributes with the corresponding components, surface defect detection results are generated.

[0089] Specifically, the system pre-trains a defect classification model using cadmium telluride thin film component samples with completed defect annotations, and establishes a defect feature space based on the regional defect characteristics corresponding to the training samples. The training samples can cover typical surface anomalies such as cracks, scratches, pinholes, coating abnormalities, edge defects, and stain oxidation, while retaining samples of different sizes, locations, and imaging states.

[0090] For each type of defect, the system uses a classification model to learn stable correlations between the internal feature distribution, boundary changes, and high-level semantic responses of the region, and forms a reference feature distribution for the corresponding category. During actual detection, the regional defect features corresponding to the target defect region are input into the classification model, transformed into a unified defect feature space through a feature mapping layer, so that the current regional defect features can be compared with the reference feature distributions of each candidate defect type.

[0091] The system can generate corresponding feature matching states based on the distance between the current region's defect features and the reference features of each candidate defect type, the consistency of their response directions, and the intensity of their classification responses. These feature matching states represent the visual-semantic proximity between the current target defect region and different defect types.

[0092] For example, fine cracks typically exhibit continuous extension, pronounced edge response, and a small internal width, showing a high degree of matching with crack reference features; pinholes usually form locally concentrated and nearly closed anomalous responses, showing a high degree of matching with pinhole reference features; coating anomalies are more often characterized by continuous regional response changes over a larger area. The system does not directly determine the final category based on a single local texture, but instead retains multiple candidate defect types and their corresponding matching degrees for subsequent correction based on spatial attributes.

[0093] Furthermore, the system reads the region contour, size, and location corresponding to the target defect area, and corrects the feature matching state based on the typical spatial distribution patterns of each candidate defect type. For example, when the region defect features have a high degree of matching with both cracks and scratches, further differentiation can be made based on the region's length-width relationship, main axis extension state, and contour continuity. When the candidate type includes pinholes, it can be determined whether it conforms to the spatial characteristics of pinholes based on the actual size of the region, the degree of contour closure, and the degree of response concentration. For corner defects, the corresponding matching degree can be corrected based on whether the target defect area intersects with the effective boundary of the component and the coverage area along the component edge. Spatial attributes are only used as a basis for correcting the feature matching state and do not replace the semantic judgment of the region defect features themselves, so as to avoid misclassification caused by simply relying on geometric dimensions.

[0094] The system can normalize the matching status of each candidate defect type after correction, and select the candidate type with the highest matching degree that meets the category determination criteria as the defect type of the target defect region. When the highest matching degree does not reach the preset confidence boundary, or when the differences between the top-ranked candidate types are insufficient, the corresponding region can be marked as a defect to be reviewed, and the candidate type and matching status can be saved to avoid forcibly outputting the wrong category under uncertain conditions. During model training and actual operation, defect regions that have been manually reviewed and confirmed can also be added back to the training samples for subsequent updates to the reference distribution in the defect feature space, improving the adaptability to the defect morphology of components from different production batches.

[0095] Furthermore, after determining the defect type, the system invokes the hazard assessment relationship corresponding to that defect type. Different spatial attribute evaluation rules can be set for different defect types. For example, cracks focus on the actual extension length, coverage location, and whether they are close to the module edge; pinholes focus on the area and the aggregation state within a unit area; coating anomalies focus on the proportion of the abnormal area and the distribution range within the effective power generation area; and corner defects focus on the defect range and the degree of overlap with the effective area of ​​the module. The hazard assessment relationship can be pre-established based on historical quality inspection data and production acceptance rules, and each spatial parameter can be mapped to the corresponding level ranges of minor, general, and severe.

[0096] For example, upon detecting a crack, the system calculates its actual length to be 18mm based on image scale conversion and determines that the crack is located in the middle of the component's effective area and does not intersect with the component's outer edge. The system matches the length, location, and area contour with the corresponding hazard assessment relationship for the crack. If the overall condition falls within the general defect range, the defect level is determined to be general. If another crack reaches a higher risk range and crosses a critical effective area, it is determined to be a severe defect and an early warning is triggered. For multiple pinholes clustered in a small area, even if a single pinhole is small, the corresponding hazard level can be increased based on the clustering distribution. Therefore, the defect level does not use a uniform area threshold for all defects, but rather calls an appropriate hazard assessment relationship based on the defect type.

[0097] Finally, the system associates the defect type, defect level, and spatial attributes of each target defect area according to the component number, and records the position mask and contour information of the corresponding defect area in the component image, generating surface defect detection results. When multiple defects exist in the same component, each defect area can be recorded separately, and the component quality status can be determined according to the highest defect level. For severe defects, the system marks the corresponding area in the visualization interface and generates warning information; for components without defects or with only minor defects that meet the acceptance criteria, corresponding quality judgment results are generated. At the same time, the detection time, component number, defect location, actual size, defect type, and defect level are recorded and written into the detection archive, so that subsequent production quality inspection, process analysis, and operation and maintenance review can trace the corresponding surface defect detection results according to the component number.

[0098] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0099] Figure 2 This is a schematic diagram of the structure of the cadmium telluride thin-film module defect detection system based on visual feature fusion provided in an embodiment of this application. Figure 2 As shown, the system includes: The acquisition module 201 is used to acquire the surface image of the cadmium telluride thin film module, determine the thin film imaging state based on the local brightness distribution and texture response of the surface image, and perform interference suppression and local contrast enhancement on the surface image according to the thin film imaging state to obtain a defect-enhanced image. The extraction module 202 is used to input the defect enhancement image into the defect feature extraction network, extract shallow visual features that preserve spatial details and deep visual features that represent the semantics of the region, determine the fusion weights based on the feature responses of each spatial location, and adaptively fuse the shallow visual features and deep visual features to generate a multi-scale defect representation. The determination module 203 is used to determine the defect response state of each pixel position based on the multi-scale defect characterization, generate candidate defect regions by combining the spatial continuity relationship of adjacent pixels, and correct the region boundary based on the feature difference between the candidate defect regions and the surrounding normal regions to obtain the target defect region. The generation module 204 is used to extract regional defect features corresponding to the target defect region from the multi-scale defect characterization, determine the spatial attributes of the target defect region, determine the defect type based on the matching status of the regional defect features and spatial attributes, determine the defect level based on the hazard judgment relationship corresponding to the spatial attributes and defect type, and generate surface defect detection results.

[0100] In some embodiments, Figure 2 The acquisition module 201 acquires surface images of the cadmium telluride thin film module according to preset imaging conditions and divides the surface images into multiple local regions; determines the brightness change state and texture response intensity of each local region, and determines the illumination disturbance state based on the brightness difference between adjacent local regions; associates the texture response intensity with the corresponding illumination disturbance state, distinguishes between the texture response of the thin film substrate and the abnormal imaging response, and generates the imaging characterization of each local region; and fuses the imaging characterizations of each local region to determine the thin film imaging state that characterizes the degree of reflective interference and texture interference.

[0101] In some embodiments, Figure 2 The acquisition module 201 determines the interference suppression intensity and contrast enhancement intensity corresponding to each local region based on the thin film imaging state; corrects the abnormal brightness response of the corresponding local region according to the interference suppression intensity, and suppresses the image components that match the texture response of the thin film substrate to obtain a corrected image; adjusts the local grayscale distribution in the corrected image according to the contrast enhancement intensity to expand the response difference between the abnormal imaging response and the neighboring region; and continuously fuses the adjusted local regions to obtain a defect enhancement image.

[0102] In some embodiments, Figure 2 The extraction module 202 performs hierarchical convolutional coding on the defect enhancement image to obtain the feature responses corresponding to different network layers; it extracts high-resolution features that preserve the defect edges and local grayscale changes from the previous network layer to generate shallow visual features; it expands the feature perception range along the subsequent network layer and aggregates the feature responses with spatial correlation to generate deep visual features; and it establishes the spatial positional correspondence between shallow visual features and deep visual features according to the network layer.

[0103] In some embodiments, Figure 2The extraction module 202 performs scale alignment of shallow visual features and deep visual features according to the spatial position correspondence; determines the local feature response differences and cross-layer semantic consistency of each spatial position, and generates the corresponding feature contribution state; determines the fusion weight of shallow visual features and deep visual features according to the feature contribution state; aggregates the scale-aligned features according to the fusion weight, and maintains the spatial details with abnormal response to generate multi-scale defect representation.

[0104] In some embodiments, Figure 2 The extraction module 202 determines the local anomaly salience state based on the degree of response deviation between the shallow visual features of each spatial location and the neighboring locations; it determines the cross-layer feature consistency state based on the degree of response correlation between the shallow visual features and deep visual features of the corresponding spatial location; it performs joint mapping between the local anomaly salience state and the cross-layer feature consistency state to determine the feature contribution ratio of the corresponding spatial location; and it adjusts the fusion weight of the shallow visual features and deep visual features according to the feature contribution ratio so that the features with significant anomalies and consistent cross-layer responses maintain the corresponding feature responses in the multi-scale defect representation.

[0105] In some embodiments, Figure 2 The determination module 203 maps the multi-scale defect representation to the image space and determines the defect response state based on the abnormal feature intensity corresponding to each pixel position. According to the response similarity and spatial connectivity between adjacent pixels, the pixels that meet the defect response conditions are aggregated to generate candidate defect regions. A normal reference region is determined along the periphery of the candidate defect region, and the multi-scale defect representations on both sides of the region edge are compared to determine the boundary difference state. The edge pixel assignment of the candidate defect region is adjusted according to the boundary difference state, and the spatial continuity of the adjusted region is checked to obtain the target defect region.

[0106] In some embodiments, Figure 2 The generation module 204 extracts the feature response of the corresponding region from the multi-scale defect representation according to the position of the target defect region in the image space; performs spatial aggregation on the feature response to obtain the regional defect features that represent the internal feature distribution and boundary changes of the target defect region; determines the region contour according to the pixel distribution of the target defect region, and determines the corresponding size state and position state in combination with the image scale conversion relationship; and fuses the region contour, size state and position state to generate the spatial attributes of the target defect region.

[0107] In some embodiments, Figure 2The generation module 204 maps the regional defect features to a preset defect feature space and determines the feature matching status between the regional defect features and each candidate defect type; it corrects the feature matching status by combining spatial attributes and determines the defect type of the target defect area based on the corrected matching status; it obtains the hazard judgment relationship corresponding to the defect type, matches the spatial attributes with the hazard judgment relationship, and determines the defect level of the target defect area; it associates the defect type, defect level, and spatial attributes with the corresponding components to generate surface defect detection results.

[0108] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments.

[0109] Figure 3 This is a schematic diagram of the electronic device 3 provided in an embodiment of this application. Figure 3 As shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the various system embodiments described above.

[0110] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or different components.

[0111] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0112] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 3. The memory 302 can also include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0114] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which may be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0115] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A defect detection method for cadmium telluride thin-film modules based on visual feature fusion, characterized in that, include: A surface image of a cadmium telluride thin film module is acquired. The thin film imaging state is determined based on the local brightness distribution and texture response of the surface image. Interference suppression and local contrast enhancement are performed on the surface image according to the thin film imaging state to obtain a defect-enhanced image. The enhanced image of the defect is input into the defect feature extraction network, which extracts shallow visual features that preserve spatial details and deep visual features that represent the semantics of the region. The fusion weights are determined based on the feature responses of each spatial location, and the shallow visual features and the deep visual features are adaptively fused to generate a multi-scale defect representation. Based on the multi-scale defect characterization, the defect response state of each pixel position is determined, and a candidate defect region is generated by combining the spatial continuity relationship of adjacent pixels. The region boundary is corrected based on the feature difference between the candidate defect region and the surrounding normal region to obtain the target defect region. Extract regional defect features corresponding to the target defect region from the multi-scale defect characterization, determine the spatial attributes of the target defect region, determine the defect type based on the matching status of the regional defect features and the spatial attributes, determine the defect level based on the hazard judgment relationship corresponding to the spatial attributes and the defect type, and generate surface defect detection results.

2. The method according to claim 1, characterized in that, The process of acquiring a surface image of the cadmium telluride thin film assembly and determining the thin film imaging state based on the local brightness distribution and texture response of the surface image includes: Surface images of the cadmium telluride thin film assembly are acquired according to preset imaging conditions, and the surface images are divided into multiple local regions; Determine the brightness change state and texture response intensity of each local region, and determine the illumination perturbation state based on the brightness difference between adjacent local regions; The texture response intensity is correlated with the corresponding illumination disturbance state to distinguish between the texture response of the thin film substrate and the abnormal imaging response, thereby generating an imaging characterization of each local region. By integrating the imaging characteristics of each local region, the thin film imaging state, which characterizes the degree of reflective interference and texture interference, is determined.

3. The method according to claim 2, characterized in that, The step of performing interference suppression and local contrast enhancement on the surface image according to the thin film imaging state to obtain a defect-enhanced image includes: The interference suppression intensity and contrast enhancement intensity corresponding to each local region are determined based on the thin film imaging state. The abnormal brightness response of the corresponding local area is corrected according to the interference suppression intensity, and the image components that match the texture response of the thin film substrate are suppressed to obtain the corrected image; The local grayscale distribution in the corrected image is adjusted according to the contrast enhancement intensity to amplify the response difference between the abnormal imaging response and the neighboring region. The adjusted local regions are continuously fused to obtain the defect-enhanced image.

4. The method according to claim 1, characterized in that, The step of inputting the enhanced image of the defect into the defect feature extraction network to extract shallow visual features that preserve spatial details and deep visual features that represent the semantics of the region includes: The defect enhancement image is subjected to hierarchical convolutional encoding to obtain the feature responses corresponding to different network layers; High-resolution features that preserve defect edges and local grayscale changes are extracted from the previous network layer to generate the shallow visual features. The feature receptive range is expanded along subsequent network layers, and spatially correlated feature responses are aggregated to generate the deep visual features. Establish the spatial correspondence between the shallow visual features and the deep visual features according to the network hierarchy.

5. The method according to claim 4, characterized in that, The step of determining fusion weights based on the feature responses of each spatial location, adaptively fusing the shallow visual features with the deep visual features to generate a multi-scale defect representation includes: The shallow visual features and the deep visual features are scale-aligned according to the spatial position correspondence. Determine the differences in local feature responses at each spatial location and the degree of cross-layer semantic consistency, and generate the corresponding feature contribution states; The fusion weights of shallow visual features and deep visual features are determined based on the feature contribution status. The multi-scale defect characterization is generated by aggregating the scale-aligned features according to the fusion weights and preserving the spatial details with anomalous responses.

6. The method according to claim 5, characterized in that, Determining the fusion weights of shallow visual features and deep visual features based on the feature contribution state includes: Based on the degree of deviation between the shallow visual features of each spatial location and the response of neighboring locations, the significant state of local anomalies is determined; Based on the degree of response correlation between shallow and deep visual features at corresponding spatial locations, the cross-layer feature consistency state is determined. The local anomaly salience state is jointly mapped with the cross-layer feature consistent state to determine the feature contribution ratio of the corresponding spatial location; The fusion weights of the shallow visual features and the deep visual features are adjusted according to the feature contribution ratio, so that features that are significantly abnormal and have consistent cross-layer responses maintain their corresponding feature responses in the multi-scale defect representation.

7. The method according to claim 1, characterized in that, The process of determining the defect response state of each pixel location based on the multi-scale defect characterization, generating candidate defect regions by combining the spatial continuity relationship of adjacent pixels, and correcting the region boundaries based on the feature differences between the candidate defect regions and the surrounding normal regions to obtain the target defect region includes: The multi-scale defect characterization is mapped to the image space, and the defect response state is determined based on the intensity of the abnormal features corresponding to each pixel position. Based on the similarity of responses and spatial connectivity between adjacent pixels, pixels that meet the defect response conditions are aggregated to generate candidate defect regions. A normal reference region is determined along the periphery of the candidate defect region, and the multi-scale defect characterizations on both sides of the region edge are compared to determine the boundary difference state. The edge pixel assignments of the candidate defect region are adjusted according to the boundary difference state, and the spatial continuity of the adjusted region is verified to obtain the target defect region.

8. The method according to claim 1, characterized in that, The step of extracting regional defect features corresponding to the target defect region from the multi-scale defect representation and determining the spatial attributes of the target defect region includes: Based on the location of the target defect region in the image space, the feature response of the corresponding region is extracted from the multi-scale defect representation; Spatial aggregation of the feature responses yields regional defect features characterizing the internal feature distribution and boundary changes of the target defect region. The region outline is determined based on the pixel distribution of the target defect region, and the corresponding size and position are determined by combining the image scale conversion relationship. By fusing the region's contour, size, and location, the spatial attributes of the target defect region are generated.

9. The method according to claim 8, characterized in that, The process of determining the defect type based on the matching status between the regional defect features and the spatial attributes, determining the defect level based on the hazard assessment relationship corresponding to the spatial attributes and the defect type, and generating surface defect detection results includes: The regional defect features are mapped to a preset defect feature space to determine the feature matching status between the regional defect features and each candidate defect type. The feature matching state is corrected by combining spatial attributes, and the defect type of the target defect area is determined based on the corrected matching state. Obtain the hazard assessment relationship corresponding to the defect type, match the spatial attributes with the hazard assessment relationship, and determine the defect level of the target defect area; The surface defect detection results are generated by associating the defect type, defect level, and spatial attributes with the corresponding components.

10. A defect detection system for cadmium telluride thin-film modules based on visual feature fusion, characterized in that, include: The acquisition module is used to acquire a surface image of a cadmium telluride thin film assembly, determine the thin film imaging state based on the local brightness distribution and texture response of the surface image, and perform interference suppression and local contrast enhancement on the surface image according to the thin film imaging state to obtain a defect-enhanced image. The extraction module is used to input the defect enhancement image into the defect feature extraction network, extract shallow visual features that preserve spatial details and deep visual features that represent the semantics of the region, determine the fusion weights based on the feature responses of each spatial location, and adaptively fuse the shallow visual features and the deep visual features to generate a multi-scale defect representation. The determination module is used to determine the defect response state of each pixel position based on the multi-scale defect characterization, generate candidate defect regions by combining the spatial continuity relationship of adjacent pixels, and correct the region boundary based on the feature difference between the candidate defect regions and the surrounding normal regions to obtain the target defect region. The generation module is used to extract regional defect features corresponding to the target defect region from the multi-scale defect characterization, determine the spatial attributes of the target defect region, determine the defect type based on the matching state between the regional defect features and the spatial attributes, determine the defect level based on the hazard judgment relationship between the spatial attributes and the defect type, and generate surface defect detection results.