A real-time visual detection system for household appliance glass defect deep learning

Abstract

The application discloses a kind of real-time visual detection systems of household appliance glass defect deep learning, it is related to visual inspection technical field, the system includes: polarization image acquisition module, polarization feature decoupling module, double branch deep learning detection module and result fusion output module;The application is by the way of pixel polarization imaging to collect the multi-path polarization angle image of the measured household appliance glass, the polarization optical response difference that exists between surface defect and subsurface defect is used to process original imaging information into the feature image corresponding to two kinds of defects respectively, which can obtain multi-dimensional polarization information relying on single imaging trigger, without multiple sets of acquisition equipment combination layout, avoid the field deviation of multiple images and additional registration operation, while based on the optical characteristic difference of defect completes feature layering processing, so that the feature expression of surface texture class defect and internal morphology class defect is more distinguishable, to ensure the recognizability of different levels of defects from the level of imaging and feature processing.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to a real-time visual inspection system for defects in household appliance glass based on deep learning. Background Technology

[0002] As a component of various home appliances such as refrigerator doors, touch panels, cooktop covers, and display panels, the surface and internal quality of appliance glass directly determines the appearance, optical performance, and reliability of the final product. With the continuous deepening of the consumption upgrade trend in the home appliance industry, the downstream market is constantly raising its requirements for the appearance smoothness, structural consistency, and yield rate of appliance glass. Various defects such as scratches, abrasions, dents, bubbles, stones, and impurities generated during the production process can cause product downgrading or even direct scrapping. Currently, mass production lines for appliance glass generally have high conveying speeds. The online quality inspection process needs to simultaneously consider detection efficiency, coverage, and identification accuracy. The traditional sampling inspection mode that relies on manual visual inspection has problems such as low efficiency, fatigue, and unstable missed detection rate, which is no longer suitable for the large-scale, high-standard production quality control requirements. There is an urgent need in the industry for automated, high-precision online inspection solutions, and the accurate differentiation of defect depth levels has become a core pain point in the field of appliance glass quality inspection.

[0003] However, existing automated glass defect detection solutions mostly employ a single-modal visible light imaging architecture using ordinary industrial cameras and conventional lighting. This architecture can only acquire grayscale information of the surface of the object being tested, making it difficult to capture the differences in optical characteristics of defects at different depths. It also lacks sufficient ability to identify defects such as bubbles and stones on the subsurface and inside the glass, easily leading to missed detections and misjudgments. Most detection solutions incorporate various defects into the same detection model for unified identification, without designing differentiated processing paths for the imaging characteristics of defects at different depths. This makes it difficult to simultaneously address the compatibility of surface texture defects and internal morphological defects, and cannot guarantee the detection effect of both types of defects. Furthermore, the results of existing detection only include the location coordinates and category information of the defects, lacking the ability to distinguish defect depth levels. This makes it impossible to provide accurate data support for subsequent product grading and production process parameter adjustments, and it is difficult to meet the actual application needs of refined quality control in home appliance glass. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a real-time visual inspection system for defects in household appliance glass based on deep learning. This invention acquires multi-channel polarization angle images of the tested household appliance glass through pixel-by-pixel polarization imaging. Utilizing the difference in polarization optical response between surface defects and subsurface defects, the original imaging information is processed into feature images corresponding to the two types of defects respectively. This method can acquire multi-dimensional polarization information based on a single imaging trigger, eliminating the need for multiple sets of acquisition devices and avoiding field-of-view deviations and additional registration calculations for multiple images. At the same time, feature layering processing is completed based on the differences in the optical characteristics of defects, making the feature expressions of surface texture defects and internal morphology defects more distinguishable. From the imaging and feature processing levels, the identifiability of defects at different levels is guaranteed, providing a targeted data foundation for subsequent defect layering detection.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a real-time visual inspection system for defects in household appliance glass based on deep learning, the system comprising: The polarization image acquisition module acquires four polarization angle images of the glass of the household appliance to be inspected through a polarization illumination unit and a polarization industrial camera. The polarization feature decoupling module calculates three types of polarization parameters based on four polarization angle images. It then uses the difference in optical properties between surface defects and subsurface defects to generate surface polarization feature maps and subsurface light intensity feature maps, respectively. The dual-branch deep learning detection module includes a surface defect detection branch and a subsurface defect detection branch. The two branches take the surface polarization feature map and the subsurface light intensity feature map as inputs, respectively, and each completes the feature extraction, target localization and category recognition of defects in the corresponding mode, and outputs the preliminary detection results of the two types of defects in parallel. The results fusion output module performs spatial registration and information integration based on a unified pixel coordinate system for the preliminary detection results of the two types of defects, and outputs structured detection results with depth level labels.

[0006] Furthermore, in the polarization image acquisition module, the polarization illumination unit adopts a coaxial diffuse polarization illumination structure, and the polarization direction of the emitted light matches the reference polarization direction of the polarization industrial camera. The illumination emission direction forms a set angle with the glass surface of the appliance. The polarization industrial camera adopts a pixel-level polarization sensor. Each physical pixel unit of the pixel-level polarization sensor integrates a photosensitive detection structure for four polarization angles. A single trigger acquisition synchronously outputs four polarization angle images. The grayscale value of each pixel coordinate in the four polarization angle images is the light intensity value. The four polarization angles are 0°, 45°, 90°, and 135°, respectively. The resolution of the four polarization angle images is completely consistent and the hardware native pixel coordinates are matched, and they share the same set of pixel coordinate systems.

[0007] Furthermore, in the polarization image acquisition module, the polarization illumination unit and the polarization industrial camera are positioned above the quality inspection station on the production conveyor line, and a photoelectric triggering unit is provided in conjunction with them. When the glass of the appliance to be inspected moves into the inspection field of view, the photoelectric triggering unit outputs a trigger signal to the polarization industrial camera to acquire a single frame image. The acquisition frame rate is matched with the running speed of the production conveyor line. The single frame acquisition field of view covers the entire area of ​​the glass of the appliance to be inspected in the lateral width direction, and the image is continuously acquired along the conveying direction to cover the complete outer surface of the glass of the appliance to be inspected.

[0008] Furthermore, in the polarization feature decoupling module, the three types of polarization parameters are the total light intensity parameter, the degree of polarization parameter, and the polarization angle parameter. When calculating the three types of polarization parameters, the light intensity values ​​of the same pixel coordinate in the four polarization angle images are extracted at the four polarization angles. The total light intensity parameter is obtained by summing the light intensity values ​​corresponding to 0° and 90°. The first orthogonal light intensity difference parameter is obtained by subtracting the light intensity values ​​corresponding to 0° and 90°. The second orthogonal light intensity difference parameter is obtained by subtracting the light intensity values ​​corresponding to 45° and 135°. The first orthogonal light intensity difference parameter and the second orthogonal light intensity difference parameter are respectively compared with the total light intensity parameter. The degree of polarization parameter is obtained by taking the square root of the sum of the squares of the two ratios. Then, the polarization angle parameter is obtained by arctangent operation based on the ratio of the second orthogonal light intensity difference parameter to the first orthogonal light intensity difference parameter.

[0009] Furthermore, in the polarization feature decoupling module, when generating the surface polarization feature map, the polarization degree parameter and the polarization angle parameter are respectively normalized to map the numerical range of these two types of polarization parameters to the same interval. Then, the surface defect polarization responsivity corresponding to each pixel coordinate is calculated using the surface defect polarization responsivity calculation formula. The surface defect polarization responsivity corresponding to all pixel coordinates is integrated into a single-channel response feature map. Finally, the normalized polarization degree parameter, polarization angle parameter, and single-channel response feature map are concatenated and fused along the channel dimension to form the surface polarization feature map. The surface defect polarization responsivity calculation formula is: ,in, The pixel values ​​at pixel coordinates x and y represent the surface defect polarization responsivity, forming the single-channel response feature map. This is the polarization degree parameter at pixel coordinates x and y, with a value range of 0-1. The polarization angle spatial gradient magnitude at pixel coordinates x and y is determined by the polarization angle parameter through neighborhood difference, and its range is determined by the numerical range of the polarization angle parameter and the spatial resolution of the four polarization angle images.

[0010] Furthermore, in the polarization feature decoupling module, when generating the subsurface light intensity feature map, the total light intensity parameter is used as the basic data. Gray-level normalization is applied to unify the brightness range of the image corresponding to the total light intensity parameter. For each group of pixel coordinates, the subsurface defect light intensity attenuation degree is calculated using the subsurface defect light intensity attenuation degree calculation formula. The subsurface defect light intensity attenuation degrees corresponding to all pixel coordinates are integrated into a single-channel attenuation feature map. Then, the normalized total light intensity parameter and the single-channel attenuation feature map are concatenated and fused along the channel dimension to form the subsurface light intensity feature map. The subsurface defect light intensity attenuation degree calculation formula is: in, The pixel values ​​at pixel coordinates x and y represent the light intensity attenuation of subsurface defects, forming the pixel values ​​of the single-channel attenuation feature map. Let x be the total light intensity parameter at pixel coordinates x and y. The average light intensity value within a preset neighborhood at pixel coordinates x and y is defined. This neighborhood is determined by the spatial resolution of the four polarization angle images and the minimum defect detection size. The polarization degree parameter is located at pixel coordinates x and y, and its value ranges from 0 to 1.

[0011] Furthermore, in the dual-branch deep learning detection module, the two branches adopt a lightweight target detection network as a common basic architecture. The network parameters of the two branches are independent of each other, and the inference process is executed in parallel. Both branches include a feature extraction backbone network, a feature fusion neck network, and a detection head network, which are adapted and optimized for the distribution characteristics of the corresponding input feature maps, and each outputs the bounding box coordinates, defect category identifier, and confidence score of the corresponding category of defects.

[0012] Furthermore, in the dual-branch deep learning detection module, the surface defect detection branch takes the surface polarization feature map as input, and the feature extraction backbone network sets the appropriate convolution kernel size and stride parameters for the polarization texture features. The identifiable defect categories of the surface defect detection branch include deep scratches, shallow scratches, abrasions and pits, and the output preliminary detection results are labeled as surface-level defects.

[0013] Furthermore, in the dual-branch deep learning detection module, the subsurface defect detection branch takes the subsurface light intensity feature map as input, and the feature extraction backbone network sets an appropriate downsampling ratio and feature channel number for grayscale morphological features. The identifiable defect categories of the subsurface defect detection branch include bubbles, stones, and impurity points, and the output preliminary detection results are labeled as subsurface level defects.

[0014] Furthermore, in the result fusion output module, using the pixel coordinate system shared by the four polarization angle images as a unified reference, the bounding box coordinates output by the surface defect detection branch and the subsurface defect detection branch are uniformly mapped to this pixel coordinate system. For each pixel region enclosed by the bounding box coordinates corresponding to a defect, the depth level confidence calibration value corresponding to that pixel region is calculated using the depth level confidence calibration calculation formula. Based on the positive or negative value of the depth level confidence calibration value, the defect is classified and labeled with the corresponding depth level identifier. The depth level identifier is divided into surface level identifier and subsurface level identifier. The bounding box coordinates, defect category identifier, depth level identifier, and confidence value corresponding to a single defect are integrated to output the defect localization and classification results with depth level identifiers. The depth level confidence calibration calculation formula is: ,in, This is the confidence level calibration value for the depth level of a single defect. At that time, assign surface level identifiers. At that time, assign subsurface level identifiers. At the same time, two types of depth level identifiers are labeled simultaneously. This represents the confidence level of the corresponding defect output by the surface defect detection branch, with a value ranging from 0 to 1. The average surface defect polarization responsivity, representing the coordinates of all pixels within the corresponding defect pixel region, is determined using a single-channel response feature map. This represents the corresponding defect confidence level output by the subsurface defect detection branch, with a value ranging from 0 to 1. The average value of the subsurface defect light intensity attenuation for all pixel coordinates within the corresponding defect pixel region is determined by a single-channel attenuation feature map.

[0015] Compared with existing technologies, this real-time visual inspection system for defects in household appliance glass based on deep learning has the following advantages: I. This invention acquires multi-channel polarization angle images of the tested household appliance glass using a pixel-by-pixel polarization imaging method. Utilizing the difference in polarization optical response between surface defects and subsurface defects, the original imaging information is processed into feature images corresponding to the two types of defects. This method relies on a single imaging trigger to acquire multi-dimensional polarization information, eliminating the need for multiple sets of acquisition devices and avoiding field-of-view deviations and additional registration calculations associated with multi-channel images. Simultaneously, feature layering processing is performed based on the differences in the optical properties of the defects, making the feature expressions of surface texture defects and internal morphology defects more distinguishable. This ensures the identifiability of defects at different levels from both imaging and feature processing perspectives, providing a targeted data foundation for subsequent defect layering detection.

[0016] Second, this invention processes two types of feature images through independent dual detection paths, adapts corresponding feature extraction logic to the feature attributes of defects at different depths, and completes the localization and category identification of the two types of defects in parallel. Then, the dual-path detection results are integrated and judged through a depth-level confidence calibration mechanism, and defect information with depth-level annotation is output. This method avoids the adaptation bias of single detection to defects at different levels, can specifically match the identification logic of various defects, and solves the problem of the attribution conflict of dual-path detection results through the level calibration mechanism, clarifying the depth-level attributes of defects, providing an accurate basis for product classification processing in subsequent production stages, and matching the actual application scenario of online quality inspection in mass production lines.

[0017] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 A flowchart of a real-time visual inspection system for defects in household appliance glass based on deep learning; Figure 2 This is a framework diagram of a real-time visual inspection system for defects in household appliance glass based on deep learning. Figure 3 This is a framework diagram of a dual-branch deep learning detection module in a real-time visual inspection system for defects in household appliance glass. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] Example: In the quality inspection station of the refrigerator door tempered glass mass production line of a home appliance glass manufacturing enterprise, in order to realize online automated layered detection of surface and subsurface defects of glass, replace the traditional manual visual sampling mode, and ensure the quality inspection accuracy and operational efficiency of the mass production process, a real-time visual inspection system based on deep learning for home appliance glass defects is adopted to carry out continuous online quality inspection. This real-time visual inspection system based on deep learning for home appliance glass defects includes a polarization image acquisition module, a polarization feature decoupling module, a dual-branch deep learning detection module, and a result fusion output module. The corresponding conveyor line running speed of this station is 0.5m / s, the size of the tempered glass for the refrigerator door to be inspected is 600mm×400mm, and the production cycle is 12 seconds / piece. The real-time visual inspection system based on deep learning for home appliance glass defects is required to be able to identify the smallest defect size of 0.1mm, and at the same time distinguish between surface and subsurface level defects, and output structured inspection results that can be directly used for product grading.

[0022] The hardware layout and imaging configuration of the polarization image acquisition module are implemented as follows: The polarization illumination unit and the polarization industrial camera are fixedly installed on the gantry bracket directly above the quality inspection station. The illumination emission direction forms a 45° angle with the glass surface of the appliance. This angle was determined through optical simulation and actual verification. It ensures that the polarized illumination light enters the polarization industrial camera after reflection from the glass surface, effectively acquiring the polarization characteristics corresponding to the surface morphology. It also allows some illumination light to penetrate the glass surface, carrying the light intensity attenuation information of subsurface defects back to the imaging surface, thus accommodating the imaging needs of both types of defects. The polarization illumination unit adopts a coaxial diffuse polarization illumination structure. The polarization direction of the emitted light matches the reference polarization direction of the polarization industrial camera, which can suppress specular reflection glare from the glass surface and improve imaging contrast. The polarization industrial camera uses a pixel-level polarization sensor. Each physical pixel unit of the pixel-level polarization sensor integrates four photosensitive detection structures for polarization angles. The four photosensitive detection structures correspond to linear polarization angles of 0°, 45°, 90°, and 135°, respectively. The four photosensitive detection structures share the entrance pupil of the same pixel. A single trigger acquisition can simultaneously output four polarization angle images. Figure 1As shown; since the four polarization angle images are generated from the same physical pixel unit of the same sensor, there is no need for multi-camera stitching or external polarizer rotation. Therefore, the resolution of the four images is completely consistent and the hardware native pixel coordinates are matched. The four polarization angle images share the same set of pixel coordinates, and there is no field of view shift or geometric deformation, so no additional image registration is required. Each pixel coordinate of the four polarization angle images corresponds to an 8-bit grayscale output. The grayscale value is linearly related to the incident light intensity, that is, the grayscale value of each pixel coordinate is the light intensity value at that position, providing basic data for subsequent polarization parameter calculation. The quality inspection station is equipped with a photoelectric trigger unit on the feeding side, which adopts a through-beam photoelectric sensor. When the glass of the household appliance to be inspected moves into the detection field of view, the light... The electrical sensor is obscured by the edge of the glass, and outputs a trigger signal to the polarization industrial camera. The polarization industrial camera responds to the trigger signal to complete the acquisition of a single frame image. The acquisition frame rate is matched with the running speed of the production conveyor line. In this embodiment, it is set to 20 frames / second. There is a 10% overlap area between adjacent frames along the conveying direction to ensure that there are no blind spots during continuous acquisition. The horizontal width of the single frame acquisition field of view is 650mm, which is greater than the 600mm horizontal width of the glass to be inspected, covering the entire area of ​​the glass in the horizontal width direction. The 50mm margin is reserved to compensate for the left and right positioning tolerance of the glass on the conveyor line. After continuously acquiring multiple frames of images along the conveying direction, the entire upper surface of the glass to be inspected can be covered, realizing full-area blind-spot-free detection.

[0023] The polarization feature decoupling module, based on four polarization angle images, completes polarization parameter calculation and feature map generation according to the following process: First, it calculates the pixel-by-pixel coordinates of three types of polarization parameters: total light intensity parameter, degree of polarization parameter, and polarization angle parameter. During the calculation, it extracts the light intensity values ​​of the same pixel coordinate at the four polarization angles from the four polarization angle images. The total light intensity parameter is obtained by summing the light intensity values ​​corresponding to 0° and 90°. The total light intensity parameter corresponds to the total intensity information of the incident light, is unaffected by the polarization state, and reflects the overall light intensity distribution of the glass. For 0°... The first orthogonal light intensity difference parameter is obtained by subtracting the light intensity value corresponding to 90°, and the second orthogonal light intensity difference parameter is obtained by subtracting the light intensity values ​​corresponding to 45° and 135°. These two sets of orthogonal light intensity difference parameters correspond to the polarization component differences in two orthogonal directions, respectively. Based on this, the ratios of the first and second orthogonal light intensity difference parameters to the total light intensity parameter are calculated. The square root of the sum of the squares of the two ratios yields the polarization degree parameter, which ranges from 0 to 1; a higher value indicates a stronger degree of polarization at that location. Then, based on the second orthogonal light intensity difference parameter... The ratio of the intensity difference parameter to the first orthogonal light intensity difference parameter is used to obtain the polarization angle parameter through arctangent calculation. The polarization angle parameter ranges from 0 to π, representing the vibration direction angle of the linearly polarized light at that location. Surface defects and subsurface defects in household appliance glass exhibit fundamentally different response characteristics to polarized light. This difference is the core physical basis for achieving layered detection: deep scratches, shallow scratches, abrasions, pits, and other surface defects represent discontinuities in the glass surface morphology, which alter the reflection direction and polarization state of the incident polarized light. This manifests as a significant difference in the degree of polarization and polarization angle between the defective area and the surrounding normal area. The discriminative power of polarization features is far greater than that of simple light intensity differences. Subsurface defects such as bubbles, stones, and impurities are located in the internal region below the glass surface. When polarized light penetrates the surface and reaches the defect location, the polarization state has already partially depolarized. The defects mainly change the intensity of transmitted light through scattering and absorption, manifesting as local light intensity attenuation. The amplitude of polarization state change is smaller than that of surface defects. Based on this difference in optical characteristics, dedicated feature maps for the two types of defects are constructed respectively, so that the defect features at different levels can be targeted and enhanced, improving the discrimination and accuracy of subsequent detection. When generating the surface polarization feature map, the polarization degree parameter and the polarization angle parameter are first subjected to numerical normalization, mapping their numerical ranges to the same interval. The polarization degree parameter itself has a value range of 0-1 and requires no additional scaling. The polarization angle parameter originally has a value range of 0-π, which is mapped to the 0-1 interval using linear normalization to eliminate dimensional differences and provide a unified numerical basis for subsequent channel fusion. After normalization, the pixel coordinates are substituted into the surface defect polarization responsivity calculation formula to solve for the surface defect polarization responsivity corresponding to each pixel coordinate. The surface defect polarization responsivity calculation formula is as follows: ,in, Let x be the polarization responsivity of the surface defect at pixel coordinates x, y. The polarization degree parameter at pixel coordinates x and y. Let x be the spatial magnitude of the polarization angle gradient at pixel coordinates x, y. In the calculation process, the spatial magnitude of the polarization angle gradient is first solved using a neighborhood difference method. A 3×3 neighborhood Sobel difference operator is used to calculate the gradient components of the polarization angle parameter in the horizontal and vertical directions. Then, the square root of the sum of the squares of the two components is taken to obtain the spatial magnitude of the polarization angle gradient at that pixel coordinate. This reflects the degree of change of the polarization angle in space. Abrupt changes in the polarization angle at surface defects correspond to higher gradient magnitudes. Multiplying the degree of polarization parameter by the spatial magnitude of the polarization angle yields the polarization responsivity of the surface defect. This value can simultaneously reflect the degree of polarization and the amplitude of polarization angle change, exhibiting strong response characteristics to surface morphology defects and suppressing noise in uniform regions. The surface defect polarization response corresponding to all pixel coordinates is arranged according to the original pixel coordinates and integrated into a single-channel response feature map. Finally, the normalized polarization degree parameter, polarization angle parameter and single-channel response feature map are spliced ​​and fused along the channel dimension to form a three-channel surface polarization feature map. This surface polarization feature map simultaneously contains three types of information: polarization degree, polarization direction and defect response intensity, which is suitable for the detection requirements of surface defects. When generating the subsurface light intensity feature map, the total light intensity parameter is used as the basic data. First, grayscale normalization is performed to unify the brightness range of the image corresponding to the total light intensity parameter. The original value range is 0-510, and linear normalization is used to map it to the standard grayscale range of 0-255 to eliminate the influence of imaging brightness fluctuations. After normalization, for each set of pixel coordinates, the subsurface defect light intensity attenuation formula is substituted to solve for the subsurface defect light intensity attenuation corresponding to each set of pixel coordinates. The subsurface defect light intensity attenuation calculation formula is as follows: ,in, Let x be the light intensity attenuation of the subsurface defect at pixel coordinates x and y. Let x be the total light intensity parameter at pixel coordinates x and y. The average light intensity value within a preset neighborhood at pixel coordinates x and y. Let x and y be the polarization degree parameters at pixel coordinates. During the calculation, the average light intensity value within the neighborhood of each pixel coordinate is first determined. The neighborhood range is determined based on the spatial resolution of the four polarization angle images and the minimum defect detection size. In this embodiment, the image spatial resolution is approximately 0.1 mm / pixel, and the minimum detection size is 0.1 mm. Therefore, an 11×11 pixel square neighborhood is selected, corresponding to a physical size of 1.1 mm × 1.1 mm. This size is larger than the minimum defect size, ensuring that the neighborhood contains sufficient normal background area. The calculated average light intensity represents the background light intensity level at that location. The average light intensity of the neighborhood is then subtracted from the current pixel's average light intensity. The total light intensity value is divided by the average light intensity of the neighborhood to obtain the relative light intensity attenuation rate of the pixel. This is then multiplied by (1 - polarization parameter) for weighting. Since the polarization degree of subsurface defects is low, the value of 1 - polarization degree is relatively large, which can further strengthen the response weight of subsurface defects while suppressing interference from highly polarized areas on the surface, ultimately obtaining the light intensity attenuation degree of subsurface defects. The light intensity attenuation degrees of subsurface defects corresponding to all pixel coordinates are arranged according to the original pixel coordinates and integrated into a single-channel attenuation feature map. Finally, the normalized total light intensity parameter and the single-channel attenuation feature map are spliced ​​and fused along the channel dimension to form a two-channel subsurface light intensity feature map, such as... Figure 2 As shown, this subsurface light intensity feature map contains two types of information: basic light intensity distribution and defect attenuation response, which is specifically adapted to the detection needs of subsurface defects.

[0024] The dual-branch deep learning detection module includes a surface defect detection branch and a subsurface defect detection branch. Both branches use a lightweight object detection network as a common architecture. In this embodiment, a lightweight object detection network is chosen as the foundation to balance detection accuracy and inference speed, meeting the frame rate requirements of real-time detection in mass production lines. The network parameters of the two branches are independent of each other. The training process is completed using labeled datasets of corresponding categories, and the inference process is executed in parallel. The multi-stream parallel processing capability of the GPU is used to run both networks simultaneously, ensuring that the overall processing latency meets real-time requirements. Both branches include a feature extraction backbone network, a feature fusion neck network, and a detection head network, which are adapted and optimized for the distribution characteristics of the corresponding input feature maps. Each branch outputs the bounding box coordinates, defect category identifier, and confidence score of the corresponding category of defect. Among them, the surface defect detection branch focuses on surface... The polarization feature map is used as input. The feature extraction backbone network sets appropriate convolution kernel size and stride parameters for polarization texture features: Since surface scratches, abrasions and other defects are mainly fine texture and high-frequency detail features, the convolution kernel size of the shallow layer of the backbone network is uniformly set to 3×3 and the stride is set to 1 to retain more high-frequency texture details and avoid losing small scratch information during downsampling. At the same time, the fusion of shallow and deep features is strengthened in the feature fusion neck network to take into account both detail features and semantic features. The identifiable defect categories of the surface defect detection branch include deep scratches, shallow scratches, abrasions and pits. The output preliminary detection results are labeled as surface-level defects. Each detection result includes the bounding box coordinates of the defect, the defect category label and the confidence value. The confidence value ranges from 0 to 1. The higher the value, the higher the confidence of the network in determining that the area is the corresponding defect. The subsurface defect detection branch takes the subsurface light intensity feature map as input, such as... Figure 3 As shown, the feature extraction backbone network sets an appropriate downsampling ratio and number of feature channels for grayscale morphological features: Since subsurface defects such as bubbles and stones are mainly characterized by regional morphology and grayscale differences, with low proportion of detailed textures, the downsampling ratio of the backbone network is appropriately increased to reduce the computation of shallow features and allocate more channel resources to deep semantic feature extraction, thereby enhancing the ability to perceive regional morphological differences. The identifiable defect categories of the subsurface defect detection branch include bubbles, stones, and impurity points. The output preliminary detection results are labeled as subsurface level defects. Each detection result also includes bounding box coordinates, defect category identifier, and confidence score.

[0025] After receiving the preliminary detection results from the two branches, the result fusion output module completes spatial registration, hierarchical calibration, and result integration, ultimately outputting structured detection results with depth level identifiers. First, spatial registration is performed: using the pixel coordinate system shared by the four polarization angle images as a unified reference, the bounding box coordinates output by the surface defect detection branch and the subsurface defect detection branch are uniformly mapped to this pixel coordinate system. Since the input feature maps of both branches are generated based on the original pixel coordinate system, and only a fixed downsampling ratio exists during network inference, the output bounding box coordinates can be multiplied by the corresponding downsampling factor to restore the coordinate values ​​to the original pixel coordinate system, completing the unified alignment of spatial positions. The registration process does not require complex deformation correction, resulting in low computational load and high accuracy. After spatial registration, for each pixel region enclosed by the bounding box coordinates corresponding to a defect, the depth level confidence calibration value corresponding to that pixel region is calculated using the depth level confidence calibration calculation formula. The depth level confidence calibration calculation formula is as follows: ,in, This is the confidence level calibration value for the depth level of a single defect. This represents the corresponding defect confidence level output by the surface defect detection branch. This represents the average surface defect polarization responsivity of all pixel coordinates within the corresponding defect pixel region. This represents the corresponding defect confidence level output by the subsurface defect detection branch. The calculation involves first calculating the surface defect polarization responsivity of all pixel coordinates within the corresponding defect pixel region and then taking the arithmetic mean to obtain the average surface defect polarization responsivity. Similarly, the calculation also involves calculating the subsurface defect light intensity attenuation of all pixel coordinates within the region and taking the arithmetic mean. The confidence score of the corresponding defect output from the surface defect detection branch is multiplied by the average surface defect polarization responsivity to obtain the comprehensive response intensity of the surface branch. The confidence score of the corresponding defect output from the subsurface defect detection branch is then multiplied by the average subsurface defect light intensity attenuation to obtain the comprehensive response intensity of the subsurface branch. Subtracting the subsurface comprehensive response intensity from the surface comprehensive response intensity yields the depth-level confidence calibration value. Based on the depth-level confidence calibration value, defects are assigned and labeled with corresponding depth-level identifiers. Depth-level identifiers are divided into surface-level identifiers and subsurface-level identifiers. When the surface feature response intensity of the defect area is higher, a surface level identifier is assigned; when When the subsurface feature response intensity representing the defect region is higher, a subsurface level identifier is assigned; when When the response intensity of the two types of features is similar and it is impossible to clearly distinguish the dominant level, two levels of depth are simultaneously labeled for subsequent manual review or graded handling reference. After the level labeling is completed, the bounding box coordinates, defect category identifier, depth level identifier and confidence level corresponding to a single defect are integrated to summarize the information of all defects into a structured detection result. The output format adopts the standard JSON format. Each defect record contains five fields: unique defect number, bounding box coordinates, defect category, depth level and confidence level. It can be directly connected to the production management system or graded handling equipment to provide accurate data support for product grading and process parameter optimization.

[0026] In summary, this embodiment relies on pixel-level polarization imaging hardware to acquire multi-channel polarization angle images. Based on the differences in optical properties between surface and subsurface defects, it completes feature decoupling and constructs dedicated feature maps. Combined with a dual-branch parallel deep learning detection architecture and a deep hierarchical calibration mechanism, it can achieve online layered detection of surface and subsurface defects on the mass production line of home appliance glass. The detection accuracy and operating speed can be adapted to the mass production cycle, effectively replacing manual visual sampling inspection, improving the automation level and reliability of the quality inspection process of home appliance glass, and providing stable technical support for the refined quality control of glass production.

[0027] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A real-time visual inspection system for defects in household appliance glass using deep learning, characterized in that, The system includes: The polarization image acquisition module acquires four polarization angle images of the glass of the household appliance to be inspected through a polarization illumination unit and a polarization industrial camera. The polarization feature decoupling module calculates three types of polarization parameters based on four polarization angle images. It then uses the difference in optical properties between surface defects and subsurface defects to generate surface polarization feature maps and subsurface light intensity feature maps, respectively. The dual-branch deep learning detection module includes a surface defect detection branch and a subsurface defect detection branch. The two branches take the surface polarization feature map and the subsurface light intensity feature map as inputs, respectively, and each completes the feature extraction, target localization and category recognition of defects in the corresponding mode, and outputs the preliminary detection results of the two types of defects in parallel. The results fusion output module performs spatial registration and information integration based on a unified pixel coordinate system for the preliminary detection results of the two types of defects, and outputs structured detection results with depth level labels.

2. The real-time visual inspection system for defects in household appliance glass based on deep learning according to claim 1, characterized in that, In the polarization image acquisition module, the polarization illumination unit adopts a coaxial diffuse polarization illumination structure. The polarization direction of the emitted light matches the reference polarization direction of the polarization industrial camera, and the illumination emission direction forms a set angle with the glass surface of the appliance. The polarization industrial camera adopts a pixel-level polarization sensor. Each physical pixel unit of the pixel-level polarization sensor integrates a photosensitive detection structure for four polarization angles. A single trigger acquisition synchronously outputs four polarization angle images. The grayscale value of each pixel coordinate in the four polarization angle images is the light intensity value. The four polarization angles are 0°, 45°, 90°, and 135°, respectively. The resolution of the four polarization angle images is completely consistent, and the hardware native pixel coordinates are matched, and they share the same set of pixel coordinate systems.

3. The real-time visual inspection system for defects in household appliance glass based on deep learning according to claim 1, characterized in that, In the polarization image acquisition module, the polarization illumination unit and the polarization industrial camera are deployed above the quality inspection station of the production conveyor line, and a photoelectric triggering unit is provided in conjunction with them. When the glass of the appliance to be inspected moves into the inspection field of view, the photoelectric triggering unit outputs a trigger signal to the polarization industrial camera to perform single-frame image acquisition. The acquisition frame rate is matched with the running speed of the production conveyor line. The single-frame acquisition field of view covers the entire area of ​​the glass of the appliance to be inspected in the lateral width direction, and the continuous acquisition along the conveying direction covers the complete outer surface of the glass of the appliance to be inspected.

4. The real-time visual inspection system for defects in household appliance glass based on deep learning according to claim 1, characterized in that, In the polarization feature decoupling module, the three types of polarization parameters are the total light intensity parameter, the degree of polarization parameter, and the polarization angle parameter. When calculating the three types of polarization parameters, the light intensity values ​​of the same pixel coordinate in the four polarization angle images are extracted at the four polarization angles. The total light intensity parameter is obtained by summing the light intensity values ​​corresponding to 0° and 90°. The first orthogonal light intensity difference parameter is obtained by subtracting the light intensity values ​​corresponding to 0° and 90°. The second orthogonal light intensity difference parameter is obtained by subtracting the light intensity values ​​corresponding to 45° and 135°. The first orthogonal light intensity difference parameter and the second orthogonal light intensity difference parameter are respectively compared with the total light intensity parameter. The degree of polarization parameter is obtained by taking the square root of the sum of the squares of the two ratios. Then, the polarization angle parameter is obtained by arctangent operation based on the ratio of the second orthogonal light intensity difference parameter to the first orthogonal light intensity difference parameter.

5. A real-time visual inspection system for defects in household appliance glass based on deep learning, as described in claim 4, is characterized in that... In the polarization feature decoupling module, when generating the surface polarization feature map, the polarization degree parameter and the polarization angle parameter are respectively normalized to map the numerical range of the two types of polarization parameters to the same interval. Then, the surface defect polarization responsivity corresponding to each pixel coordinate is calculated using the surface defect polarization responsivity calculation formula. The surface defect polarization responsivity corresponding to all pixel coordinates is integrated into a single-channel response feature map. Finally, the normalized polarization degree parameter, polarization angle parameter and single-channel response feature map are spliced ​​and fused along the channel dimension to form the surface polarization feature map. The formula for calculating the polarization responsivity of surface defects is: ,in, Let x be the polarization responsivity of the surface defect at pixel coordinates x, y. The polarization degree parameter at pixel coordinates x and y. Let x be the magnitude of the spatial gradient of the polarization angle at pixel coordinates x and y.

6. The real-time visual inspection system for defects in household appliance glass based on deep learning according to claim 4, characterized in that, In the polarization feature decoupling module, when generating the subsurface light intensity feature map, the total light intensity parameter is used as the basic data. The brightness range of the image corresponding to the total light intensity parameter is unified through grayscale normalization. For each group of pixel coordinates, the subsurface defect light intensity attenuation degree corresponding to each group of pixel coordinates is calculated using the subsurface defect light intensity attenuation degree calculation formula. The subsurface defect light intensity attenuation degree corresponding to all pixel coordinates is integrated into a single-channel attenuation feature map. Then, the normalized total light intensity parameter and the single-channel attenuation feature map are spliced ​​and fused along the channel dimension to form the subsurface light intensity feature map. The formula for calculating the light intensity attenuation of subsurface defects is: ,in, Let x be the light intensity attenuation of the subsurface defect at pixel coordinates x, y. Let x be the total light intensity parameter at pixel coordinates x and y. The average light intensity value within a preset neighborhood at pixel coordinates x and y. Let x and y be the polarization parameters at pixel coordinates.

7. The real-time visual inspection system for defects in household appliance glass based on deep learning according to claim 1, characterized in that, In the dual-branch deep learning detection module, the two branches adopt a lightweight target detection network as a common basic architecture. The network parameters of the two branches are independent of each other, and the inference process is executed in parallel. Both branches include a feature extraction backbone network, a feature fusion neck network, and a detection head network, which are adapted and optimized for the distribution characteristics of the corresponding input feature maps. They each output the bounding box coordinates, defect category identifier, and confidence score of the corresponding category of defects.

8. The real-time visual inspection system for defects in household appliance glass based on deep learning according to claim 7, characterized in that, In the dual-branch deep learning detection module, the surface defect detection branch takes the surface polarization feature map as input, and the feature extraction backbone network sets the appropriate convolution kernel size and stride parameters for the polarization texture features. The identifiable defect categories of the surface defect detection branch include deep scratches, shallow scratches, abrasions and pits, and the output preliminary detection results are labeled as surface-level defects.

9. A real-time visual inspection system for defects in household appliance glass based on deep learning, as described in claim 7, is characterized in that... In the dual-branch deep learning detection module, the subsurface defect detection branch takes the subsurface light intensity feature map as input, and the feature extraction backbone network sets an appropriate downsampling ratio and number of feature channels for grayscale morphological features. The identifiable defect categories of the subsurface defect detection branch include bubbles, stones, and impurity points, and the output preliminary detection results are labeled as subsurface-level defects.

10. A real-time visual inspection system for defects in household appliance glass based on deep learning, as described in claim 7, is characterized in that... In the result fusion output module, the pixel coordinate system shared by the four polarization angle images is used as a unified reference. The bounding box coordinates output by the surface defect detection branch and the subsurface defect detection branch are uniformly mapped to this pixel coordinate system. For the pixel region enclosed by the bounding box coordinates corresponding to each defect, the depth level confidence calibration value corresponding to the pixel region is calculated using the depth level confidence calibration calculation formula. Based on the positive and negative values ​​of the depth level confidence calibration value, the defect is classified and labeled with the corresponding depth level identifier. The depth level identifier is divided into surface level identifier and subsurface level identifier. The bounding box coordinates, defect category identifier, depth level identifier and confidence value corresponding to a single defect are integrated to output the defect localization and classification results with depth level identifiers. The formula for calculating the confidence level calibration at depth levels is: ,in, This is the confidence level calibration value for the depth level of a single defect. At that time, assign surface level identifiers. At that time, assign subsurface level identifiers. At the same time, two types of depth level identifiers are labeled simultaneously. This represents the corresponding defect confidence level output by the surface defect detection branch. This represents the average surface defect polarization responsivity of all pixel coordinates within the corresponding defect pixel region. This represents the corresponding defect confidence level output by the subsurface defect detection branch. This represents the average subsurface defect light intensity attenuation rate for all pixel coordinates within the corresponding defect pixel region.

Images