A defect detection method and system for ambient lights

Abstract

This invention relates to the field of defect detection technology, and discloses a defect detection method and system for ambient lights. The method includes: acquiring RGB and near-infrared images of the target ambient light; preprocessing all images and binding them with product structure information, chip model, and light guide structure type; inputting the images into a multi-task convolutional neural network to obtain image recognition results, and determining the pass / fail status of the image recognition results based on a dynamic tolerance threshold; acquiring assembly structure measurement data of the target ambient light to determine the assembly recognition results; and determining the defect detection results of the target ambient light based on the image recognition results and the assembly recognition results. This application improves the accuracy and comprehensiveness of defect detection by fusing multi-angle multispectral image acquisition, multi-task convolutional neural network recognition, and three-dimensional assembly measurement, achieving intelligent, standardized, and traceable quality management.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and more specifically, to a defect detection method and system for ambient lighting. Background Technology

[0002] With the rapid development of the automotive industry and the continuous improvement of in-vehicle intelligent experience, ambient lighting, as an important component to enhance the aesthetic quality of vehicle interiors and the user's perceived interactive experience, has been widely used in various interior modules of mid-to-high-end models, such as door panels, dashboards, footwells, and cup holders. Current ambient lighting products exhibit a diversified structure, often employing injection-molded light guides, laser-dotted light guides, or soft light guide solutions, combined with intelligent chips such as the MLX81108 and IND83212 to achieve RGB control and dynamic flowing effects. The requirements for appearance and functional indicators such as luminous uniformity, color consistency, and structural fit are becoming increasingly stringent.

[0003] To ensure the lighting effect and assembly quality of ambient lights, traditional quality inspection methods mainly rely on manual visual inspection and fixed-point brightness measurement. However, this approach has significant limitations: manual inspection is highly subjective and struggles to accurately identify minor bright spots, dark areas, or color shifts, especially with poor repeatability in dynamic lighting conditions; existing image recognition solutions are mostly single-task networks or static threshold recognition, unable to distinguish complex defect types caused by light guide structure defects, chip color temperature shifts, or assembly misalignments; and inspection systems are often decoupled from product structure, chip model, and other parameters, lacking specificity and adaptability. Furthermore, some ambient light defects do not originate from the optical body itself, but are caused by minor assembly misalignments at the light guide interface and bracket joint, such as misaligned connecting slots or abnormal light refraction due to uneven bracket stress. These assembly defects are often missed.

[0004] Therefore, it is necessary to design a defect detection method and system for ambient lights to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a defect detection method and system for ambient lights, aiming to solve the problems of poor detection repeatability, poor reliability of detection results and easy omission in the current technology.

[0006] In one aspect, the present invention proposes a defect detection method for ambient lighting, comprising:

[0007] An image acquisition device is set up to acquire RGB and near-infrared images of the target ambient light. The image acquisition device includes lighting units at least two angles and a multispectral exposure device.

[0008] All images are preprocessed, and the preprocessed images are bound to product structure information, chip model and light guide structure type. The preprocessing includes noise reduction, alignment, cropping and color calibration.

[0009] The preprocessed image is input into a multi-task convolutional neural network, which includes a first branch network for uneven light emission recognition, a second branch network for color difference discrimination, and a third branch network for light guide structure defect detection. The image recognition result output by the multi-task convolutional neural network is obtained. The image recognition result includes defect category, location coordinates, and confidence score. The image recognition result is judged to be qualified based on the dynamic tolerance threshold.

[0010] The assembly structure measurement data of the target ambient light is obtained, and the assembly structure measurement data is compared with the offset threshold of the structural connection part to determine the assembly recognition result.

[0011] The defect detection results of the target ambient light are determined based on the image recognition results and the assembly recognition results.

[0012] Furthermore, when setting up an image acquisition device to acquire RGB and near-infrared images of the target ambient light, the following steps are included:

[0013] Two lighting units are set at different angles above and to the side of the target ambient light, and the lighting units adopt a wide spectrum LED array;

[0014] A multi-channel industrial camera is installed directly above the inspection station. The multi-channel industrial camera has the ability to simultaneously acquire RGB images and near-infrared images. The multi-channel industrial camera also includes a multispectral exposure device, which adjusts the exposure time and spectral channels according to the material transmittance and structural thickness of the target ambient light.

[0015] Furthermore, when preprocessing all images and binding the preprocessed images with product structure information, chip model, and light guide structure type, the process includes:

[0016] Establish a product parameter database, which records the product structure information, chip model and light guide structure type corresponding to different models of ambient lights;

[0017] Collect the product barcode of the target ambient light, and extract the corresponding product structure information, chip model and light guide structure type from the product parameter database based on the product barcode;

[0018] The images are denoised, aligned, cropped at different angles, and calibrated in color space. The pre-processed images are then fused with the extracted parameters to generate a product dataset.

[0019] Furthermore, the multi-task convolutional neural network includes:

[0020] A shared backbone feature extraction network with three independent branch networks;

[0021] The backbone feature extraction network is based on the EfficientNet structure and is used to extract the basic spatial features and channel features of the image;

[0022] The first branch network is based on a fully convolutional architecture, outputs a probability map of brightness non-uniformity, and calculates the boundary of the defect region through local variance;

[0023] The second branch network converts the RGB image to the CIELab color space and then performs channel-wise convolution processing to output a color difference map and calculates the deviation by comparing it with the standard calibration value.

[0024] The third branch network adopts a pyramid structure to fuse global and local features and generate a binary defect mask map.

[0025] Furthermore, obtaining the image recognition result output by the multi-task convolutional neural network includes:

[0026] The defect categories in the image recognition results include brightness defects, color difference defects, and light guide defects;

[0027] The location coordinates represent the segmentation mask of the defect region in pixels, and the confidence score is obtained by weighted fusion of the output values ​​of each branch network and then normalized based on sigmoid.

[0028] Furthermore, the multi-task convolutional neural network optimizes the loss function through joint training, and the loss function includes:

[0029] The first branch network uses a combination of structural similarity (SSIM) loss and mean square error (MSE) to construct a brightness unevenness detection loss term;

[0030] The second branch network uses the perceived color difference loss and divergence in the Lab color space to form the color difference recognition loss term;

[0031] The third branch network uses a combined loss of Dice coefficient and IoU.

[0032] Furthermore, when determining the pass / fail status of image recognition results based on the dynamic tolerance threshold, the following steps are taken:

[0033] Establish a tolerance template library that matches defect categories with projects. The tolerance template library records the color difference, brightness fluctuation, and tolerance threshold of light guide defects corresponding to different vehicle models and light guide schemes.

[0034] Based on the defect category, location information, and confidence score output from the image recognition results, and combined with the product structure information parsed from the product barcode, the tolerance threshold of the corresponding item in the tolerance template library is called.

[0035] Tolerance judgment is performed on the identified defects, where:

[0036] For brightness-related defects, determine whether the uniformity index and local brightness fluctuations exceed the brightness tolerance threshold;

[0037] For color difference defects, determine whether the color difference value exceeds the color difference tolerance threshold;

[0038] For light guide defects, determine whether the mask area exceeds the light guide tolerance threshold.

[0039] Furthermore, when acquiring the assembly structure measurement data of the target ambient light and comparing the assembly structure measurement data with the offset threshold of the structural connection parts to determine the assembly identification result, the process includes:

[0040] A three-dimensional contour measurement sensor is set up, which acquires the three-dimensional geometric data of the target ambient light based on laser triangulation.

[0041] Based on the reference points and assembly tolerance zones set in the structural model of the target ambient light, coordinate fitting and boundary curvature analysis are performed on the collected three-dimensional geometric data to generate the spatial offset vector of the current measured part.

[0042] The spatial offset vector is compared with the offset threshold at each structural connection feature point;

[0043] If the spatial offset vector exceeds the offset threshold at any connection point, the generated assembly identification result is unqualified.

[0044] Furthermore, when determining the defect detection result of the target ambient light based on the image recognition result and the assembly recognition result, the following steps are included:

[0045] A joint decision is made based on the image recognition results and the assembly recognition results, wherein:

[0046] If there is a defect in the image recognition result but the corresponding area of ​​assembly recognition is qualified, it is determined to be an optical defect;

[0047] If there is a defect in the image recognition result and the spatial overlap between the defect location and the assembly offset area exceeds a preset threshold, it is determined to be a structural defect caused by assembly.

[0048] If the image recognition result is qualified but the assembly recognition is unqualified, it is marked as a potential assembly hazard and recorded in the historical work order.

[0049] Compared with existing technologies, the advantages of this invention are as follows: By constructing an ambient light defect detection process that integrates multi-angle multispectral image acquisition, structural parameter binding, image preprocessing, multi-task convolutional neural network recognition, dynamic tolerance judgment, and three-dimensional assembly measurement, this invention solves the problems of low recognition accuracy, weak defect attribution ability, fixed judgment standards, and easy omission of assembly deviations in traditional detection methods. It utilizes multi-angle lighting and multispectral exposure devices to enhance image detail representation, achieves adaptive parameter adjustment of the recognition model by binding structural information with chip model, identifies uneven brightness, abnormal color difference, and light guide structure defects in parallel based on multi-task neural networks, and combines three-dimensional contour measurement data to judge the spatial offset of key assembly connection parts. Through joint decision-making of images and assembly results, defect attribution and level determination are completed, improving the accuracy and comprehensiveness of defect detection and achieving intelligent, standardized, and traceable quality management.

[0050] On the other hand, this application also provides a defect detection system for ambient lighting, for applying the above-described defect detection method for ambient lighting, including:

[0051] The acquisition unit is configured to acquire RGB and near-infrared images of the target ambient light;

[0052] The processing unit is configured to preprocess all images and bind the preprocessed images with product structure information, chip model and light guide structure type. The preprocessing includes noise reduction, alignment, cropping and color calibration.

[0053] The judgment unit is configured to input the preprocessed image into a multi-task convolutional neural network, which includes a first branch network for uneven light emission recognition, a second branch network for color difference discrimination, and a third branch network for light guide structure defect detection, to obtain the image recognition result output by the multi-task convolutional neural network. The image recognition result includes defect category, location coordinates, and confidence score, and the image recognition result is judged to be qualified according to a dynamic tolerance threshold.

[0054] The identification unit is configured to acquire the assembly structure measurement data of the target ambient light, compare the assembly structure measurement data with the offset threshold of the structural connection part, and determine the assembly identification result.

[0055] The detection unit is configured to determine the defect detection result of the target ambient light based on the image recognition result and the assembly recognition result.

[0056] It is understandable that the above-mentioned defect detection methods and systems for ambient lighting have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0057] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0058] Figure 1 A flowchart illustrating a defect detection method for ambient lighting provided in an embodiment of the present invention;

[0059] Figure 2 This is a functional block diagram of a defect detection system for ambient lighting provided in an embodiment of the present invention. Detailed Implementation

[0060] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0061] Traditional methods for detecting defects in ambient lighting rely on manual visual inspection, which depends on the operator's subjective judgment and struggles to quantify and identify uneven brightness distribution or color drift, especially under dynamic flow effects where it cannot capture instantaneous anomalies. Fixed-point brightness measurement only covers localized areas, leading to missed defects within the light guide structure. Single-task convolutional neural networks lack multi-dimensional feature fusion capabilities, failing to distinguish between complex defects caused by chip color temperature shifts or assembly misalignments. Furthermore, the detection system is decoupled from product structural parameters, unable to dynamically adjust the recognition threshold based on the light guide material's transmittance and chip model differences, resulting in a higher false positive rate.

[0062] For example, in a production line for automotive door ambient lighting using injection-molded light guides and the MLX81108 chip, manual inspection cannot reliably identify localized dark areas caused by microcracks in the light guide at a level of 0.5mm. When the sequential lighting switches at a frequency of 10Hz, traditional single-spectrum cameras suffer from motion blur due to exposure synchronization errors, failing to accurately capture transient color temperature shifts. Existing single-branch inspection networks misjudge abnormal light guide refraction caused by assembly misalignment as defects in the light guide itself, incorrectly triggering the light guide module replacement process. The inspection system is not connected to the product parameter database and still uses the color calibration parameters of rigid light guides for soft light guide products driven by the IND83212 chip, resulting in a deviation in the color difference recognition benchmark.

[0063] If these issues are not addressed, the production line misjudgment rate will rise to over 8%, resulting in qualified optical guide modules being incorrectly scrapped. Missed dynamic defects will lead to increased customer complaints, impacting brand quality reputation. Invalid repair orders caused by misjudgments will increase unit production costs by 12%-15%. Detection deviations not adapted to product parameters will hinder multi-model mixed-line production and limit flexible production line upgrades. Misclassification of complex defects will delay root cause analysis and extend process optimization cycles.

[0064] For this, please refer to Figure 1 As shown, this application proposes a defect detection method for ambient lights, comprising:

[0065] S100: Set up an image acquisition device to acquire RGB and near-infrared images of the target ambient light, wherein the image acquisition device includes lighting units at least two angles and a multispectral exposure device.

[0066] S200: Preprocesses all images and binds the preprocessed images with product structure information, chip model and light guide structure type. Preprocessing includes noise reduction, alignment, cropping and color calibration.

[0067] S300: Input the preprocessed image into the multi-task convolutional neural network. The multi-task convolutional neural network includes a first branch network for uneven light emission recognition, a second branch network for color difference discrimination, and a third branch network for light guide structure defect detection. Obtain the image recognition result output by the multi-task convolutional neural network. The image recognition result includes defect category, location coordinates, and confidence score. The image recognition result is qualified according to the dynamic tolerance threshold.

[0068] S400: Acquire the assembly structure measurement data of the target ambient light, compare the assembly structure measurement data with the offset threshold of the structural connection part, and determine the assembly recognition result.

[0069] S500: Determine the defect detection results of the target ambient light based on the image recognition results and assembly recognition results.

[0070] The image acquisition device includes at least two lighting units at different angles and a multispectral exposure device. Specifically, it can be implemented using a combination of a broadband LED array and a multi-channel industrial camera. Different angle light sources eliminate shadow interference, and the multispectral exposure device adjusts parameters according to the material's transmittance and structural thickness to ensure the clarity of RGB and near-infrared images. Preprocessing includes noise reduction, alignment, cropping, and color calibration. Specifically, Gaussian filtering eliminates noise, feature point matching achieves image alignment, ROI region cropping removes irrelevant background, and a white balance algorithm corrects color deviations. The multi-task convolutional neural network includes three independent branches. Specifically, a shared backbone network extracts basic features, the fully convolutional branch outputs a brightness unevenness probability map, channel-specific convolution processes the color difference map, and a pyramid structure generates a light guide defect mask, enabling simultaneous detection of multiple defect types. A dynamic tolerance threshold determines the pass / fail status of the image recognition results. Specifically, a tolerance template library associated with the product model can be established, and corresponding brightness fluctuation, color difference range, and defect area thresholds can be called according to different light guide schemes to achieve differentiated judgment standards. The assembly structure measurement data is compared with the offset threshold. Specifically, a laser triangulation sensor can be used to acquire three-dimensional geometric data, and the spatial offset vector of the connection part can be calculated through coordinate fitting to determine whether it exceeds the assembly tolerance zone. The image recognition results and assembly recognition results are jointly used to determine the defect detection results. Specifically, the spatial overlap between the defect area and the assembly offset can be analyzed to distinguish between optical defects and structural defects, avoiding misjudgments caused by a single detection dimension.

[0071] This application achieves accurate identification and classification of complex defect types in ambient lighting by fusing multispectral imaging and multi-task networks, combined with dynamic tolerance and assembly data for joint decision-making. By binding product parameters and preprocessed data, a differentiated inspection benchmark is established, addressing the problem that traditional methods cannot adapt to diverse product structures. Cross-validation of 3D contour measurement and image recognition results effectively distinguishes between optical defects and anomalies caused by assembly misalignment, enhancing the overall judgment capability of the inspection system.

[0072] The working process and principle of this application are as follows: An image acquisition device is set up, including lighting units at least two angles and a multispectral exposure device, for acquiring RGB and near-infrared images of the target ambient light. The multi-angle, multispectral acquisition method can comprehensively capture the optical characteristics of the ambient light.

[0073] All acquired images undergo preprocessing, including noise reduction, alignment, cropping, and color calibration. The preprocessed images are then linked to product structure information, chip model, and light guide structure type. This linking process allows subsequent analysis to be performed based on specific product parameters.

[0074] The preprocessed image is input into a multi-task convolutional neural network for analysis. This network consists of three branches: the first branch is used for uneven light emission recognition, the second branch for color difference discrimination, and the third branch for light guide structure defect detection. This multi-task parallel processing approach enables a comprehensive analysis of various possible defects in the ambient light.

[0075] The network outputs image recognition results including defect category, location coordinates, and confidence score. These results are then judged as acceptable based on a dynamic tolerance threshold, which allows the judgment criteria to be flexibly adjusted according to different product parameters.

[0076] Simultaneously, the system acquires assembly structure measurement data of the target ambient light, compares it with the offset threshold of the structural connection parts, and determines the assembly identification result. It can detect defects caused by assembly problems.

[0077] The defect detection results of the target ambient light are determined by combining image recognition results with assembly recognition results. This multi-dimensional analysis method can accurately distinguish between optical defects and assembly defects, improving the accuracy and comprehensiveness of the detection.

[0078] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0079] On the ambient lighting production line, a testing station is set up. This station is equipped with two LED array lighting units, located at a 45-degree angle above and a 90-degree angle to the side of the ambient light, respectively. A multi-channel industrial camera is installed directly above the station, capable of simultaneously acquiring RGB and near-infrared images.

[0080] When an ambient light product arrives at the testing station, its barcode is read, and the corresponding product structure information, chip model, and light guide structure type are extracted from the product parameter database. Based on this information, the multispectral exposure device automatically adjusts the exposure time and spectral channels.

[0081] Next, the lighting unit is activated, and the multi-channel camera simultaneously acquires RGB and near-infrared images. The acquired images undergo preprocessing steps such as noise reduction, alignment, cropping, and color calibration. The preprocessed images are then fused with the previously extracted product parameters to generate a complete product dataset.

[0082] The dataset is input into a pre-trained multi-task convolutional neural network. The backbone feature extraction network of this network is based on the EfficientNet structure and extracts the basic spatial and channel features of the image. The first branch network outputs a probability map of brightness non-uniformity, the second branch network analyzes the color difference in the CIELab color space, and the third branch network generates a binary defect mask map.

[0083] The defect category, location coordinates, and confidence score output by the network are compared with a preset dynamic tolerance threshold. Simultaneously, a 3D contour measurement sensor collects assembly structure data of the ambient light and compares it with a preset offset threshold.

[0084] By comprehensively analyzing the image recognition results and assembly recognition results, the defect type and severity are determined, and a final detection conclusion is given.

[0085] Through the above-described scheme, this application enables comprehensive and accurate detection of defects in ambient lighting. Multispectral image acquisition and multi-task neural network analysis improve the ability to identify minute optical defects, especially in capturing instantaneous anomalies under dynamic flowing water effects. Binding product parameters with image data allows for dynamic adjustment of identification standards based on different materials and chip characteristics, effectively reducing the false positive rate. Joint analysis of three-dimensional geometric measurement and image recognition results accurately distinguishes between body defects and assembly defects, improving the accuracy of defect tracing and enhancing product quality control.

[0086] In some of the solutions described above in this application, when the image acquisition device acquires ambient lights with different light transmittance and structural thickness, the insufficient coverage and spectral response of the light source result in loss of detail or exposure deviation in the RGB and near-infrared images, affecting the accuracy of subsequent defect detection.

[0087] This application further proposes two lighting units positioned at different angles above and to the side of the target ambient light, each employing a broadband LED array. A multi-channel industrial camera is positioned directly above the inspection station, capable of simultaneously acquiring RGB and near-infrared images. The multi-channel industrial camera also includes a multispectral exposure device that adjusts the exposure time and spectral channels according to the material transmittance and structural thickness of the target ambient light.

[0088] The wide-spectrum LED array covers the visible to near-infrared bands, and dual-angle lighting from above and to the side eliminates shadow interference caused by a single light source. A multi-channel industrial camera simultaneously acquires RGB and near-infrared images via a beam splitter, avoiding timing errors. The multispectral exposure device dynamically adjusts exposure parameters based on a material transmittance database; for example, materials with transmittance below 30% use extended exposure times and near-infrared channel enhancement, while materials with transmittance above 70% use shorter exposure times and blue light channel suppression.

[0089] Specifically, the upper lighting unit covers the main luminous area of ​​the ambient light surface at a 45-degree incident angle, while the side lighting unit enhances the texture contrast of the light guide structure edges at a 75-degree incident angle. The broadband LED array controls the intensity ratio of different wavelengths by adjusting the current; for example, when capturing light guide defects, the intensity of the red light band is increased to enhance the visibility of the structural defects. The multispectral exposure device receives material parameters through an embedded controller and matches them to preset exposure curves. For example, a 2mm thick PMMA light guide plate corresponds to an exposure time of 8ms with both 650nm and 850nm channels activated, while a 1.5mm thick PC material corresponds to an exposure time of 5ms with only the 850nm channel activated. The multi-channel industrial camera uses time-division exposure technology to simultaneously acquire images of the RGB three channels and near-infrared channels within the same workstation cycle, and ensures image timing alignment through hardware triggering. Thus, the differences in optical properties of different materials and structures are effectively compensated, avoiding image information loss due to overexposure or underexposure.

[0090] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0091] Two lighting units are positioned at different angles above and to the side of the target ambient light. These units utilize a broadband LED array. The LED array contains multiple full-spectrum LEDs with wavelengths ranging from 380-780nm, arranged in a matrix structure. The upper lighting unit is located 30cm directly above the ambient light, while the side lighting unit is located 20cm at a 45° angle to the side of the ambient light. The two lighting units independently control brightness and color temperature to adapt to the lighting requirements of different ambient light structures.

[0092] A multi-channel industrial camera is positioned directly above the inspection station. This camera is capable of simultaneously acquiring RGB and near-infrared images. It uses a CMOS sensor with a pixel resolution of 4096×3000 and a frame rate of up to 60fps. The camera lens has a focal length of 12mm and an adjustable aperture range of F1.4-F16. The multi-channel industrial camera also includes a multispectral exposure unit. This unit adjusts the exposure time and spectral channels based on the material transmittance and structural thickness of the target ambient light. The exposure time can be precisely adjusted within the range of 10μs-1s, and the spectral channels include visible light (400-700nm) and near-infrared (700-1000nm). By adjusting the exposure parameters of different channels, optimal imaging effects for ambient lights of various materials can be achieved.

[0093] Through the above technical solution, this application achieves comprehensive optical inspection of ambient lights. The multi-angle lighting unit provides uniform and stable illumination conditions, avoiding ambient light interference and enhancing defect visibility. A multi-channel industrial camera simultaneously acquires RGB and near-infrared images, expanding the spectral range of the inspection and enabling the capture of minute defects that are difficult to detect with the naked eye. The multispectral exposure device dynamically adjusts exposure parameters to adapt to ambient lights of different materials and structures, ensuring consistent image quality. The image acquisition scheme improves the accuracy and reliability of defect detection.

[0094] In some of the solutions described above in this application, there is no effective correlation between the preprocessed image and the product structure information, chip model, and light guide structure type, which makes it impossible for the detection system to perform differentiated analysis for the characteristics of different product models, thus affecting the accuracy of defect judgment.

[0095] This application further proposes establishing a product parameter database, which records the product structure information, chip model, and light guide structure type corresponding to different models of ambient lights. The product barcode of the target ambient light is collected, and the corresponding product structure information, chip model, and light guide structure type are extracted from the product parameter database based on the product barcode. The image is then subjected to denoising, alignment, viewing angle cropping, and color space calibration. The pre-processed image is then fused with the extracted parameters to generate a product dataset.

[0096] The product parameter database stores data by pre-entering the refractive index of the light guide material, chip driving parameters, and 3D models of the light guide structure corresponding to different models. Product barcodes are embedded in the surface of the ambient light mounting bracket in the form of QR codes or RFID tags. Noise reduction uses a non-local mean filtering algorithm to eliminate image noise; alignment is based on feature point matching to achieve spatial registration of multispectral images; viewpoint cropping selects the effective detection range by pre-defined ROI regions; and color space calibration performs white balance correction based on a standard color chart. Data fusion encodes product structural information as metadata and embeds it in the image file header; chip model and light guide structure type are associated with the image hash value in the form of structured fields.

[0097] Specifically, when the target ambient light enters the inspection station, the vision sensor scans its surface QR code to obtain a unique identification code. The system then retrieves the corresponding light guide wall thickness parameters, chip color temperature compensation coefficient, and light guide groove distribution topology map via a database interface. RGB and near-infrared images simultaneously acquired by multi-channel industrial cameras are denoised, and feature points are extracted using the SIFT algorithm to complete multispectral image alignment. Subsequently, pixel-level cropping is performed based on preset inspection area coordinates. During calibration, a color conversion matrix is ​​established based on 24-color chart data under a standard D65 light source to convert the original image to a standard color space. The preprocessed image and process parameters extracted from the database are encapsulated in JSON format to form a multidimensional dataset containing pixel data, structural parameters, and optical properties. This provides learning samples with model-specific features for subsequent multi-task neural networks. This data binding mechanism enables the inspection system to automatically adjust the color difference judgment threshold according to the chip model and optimize the defect region segmentation algorithm based on the light guide structure type, thereby improving the recognition accuracy of complex defects.

[0098] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0099] A product parameter database is established, which records the product structure information, chip model, and light guide structure type for different models of ambient lights. The product parameter database adopts a relational database structure and includes fields such as product model, structural parameters, chip model, and light guide type.

[0100] Collect the product barcode of the target ambient light. Use an industrial-grade QR code scanner to scan the QR code label on the ambient light's casing and obtain a 16-digit string containing product model information.

[0101] The system retrieves the corresponding product parameter information from the database based on the product barcode. Using an SQL query with the product model as the keyword, it retrieves the corresponding structural parameters, chip model, light guide type, and other information from the database.

[0102] Image preprocessing is performed. First, Gaussian filtering is used for noise reduction. Then, multi-angle image alignment is performed based on a feature point matching algorithm. Next, the image is cropped according to a preset region of interest. Finally, color space conversion is performed using a color calibration panel.

[0103] The preprocessed images are fused with the extracted parameters to generate a product dataset. The image data and product parameter information are integrated into a structured dataset containing fields such as image matrix, product model, structural parameters, chip model, and light guide type.

[0104] Through the above technical solution, this application achieves automated association between product parameters and image data, improving data processing efficiency. Simultaneously, by establishing a product parameter database, key parameters for different product models can be quickly obtained, providing accurate reference information for subsequent defect detection. Furthermore, standardized preprocessing of the images eliminates image quality fluctuations caused by differences in shooting conditions.

[0105] In some of the solutions described above in this application, the backbone feature extraction network of the multi-task convolutional neural network may not be able to effectively balance the feature expression requirements of different defect types, resulting in feature redundancy or missing features in the branch networks when processing uneven brightness, color difference or light guide structure defects, which affects detection accuracy and efficiency.

[0106] This application further proposes a multi-task convolutional neural network comprising a shared backbone feature extraction network and three independent branch networks. The backbone feature extraction network, based on the EfficientNet architecture, is used to extract basic spatial and channel features of the image. The first branch network, based on a fully convolutional architecture, outputs a probability map of brightness non-uniformity and calculates the boundary of the defect region through local variance. The second branch network converts the RGB image to the CIELab color space and performs channel-wise convolution processing, outputting a color difference map and calculating the deviation by comparing it with a standard calibration value. The third branch network uses a pyramid structure to fuse global and local features, generating a binary defect mask map.

[0107] The backbone feature extraction network employs EfficientNet's composite scaling strategy, uniformly scaling the network's depth, width, and resolution parameters to enhance the representational ability of basic features under computationally limited conditions. The first branch's fully convolutional architecture comprises five deconvolutional layers, each using a 3×3 kernel and ReLU activation function. The last layer outputs a single-channel probability map, with the probability value corresponding to the confidence level of a pixel belonging to a region of uneven brightness. The second branch's input data undergoes color space transformation, followed by separable convolution operations in the L, a, and b channels. The convolution kernel size is 5×5, with a stride of 2, and the number of output channels remains consistent with the input. The third branch's pyramid structure contains four feature layers of different scales. The bottom feature map is 1 / 4 the size of the original image, and the top layer is 1 / 32. Features from each layer are upsampled using bilinear interpolation and then concatenated.

[0108] Specifically, after the preprocessed RGB and near-infrared images are input into the backbone network, multi-scale features are extracted layer by layer through the MBConv module of EfficientNet to generate a basic feature map containing spatial details and channel correlations. The first branch network performs a full convolution operation on the basic feature map to generate a brightness distribution probability map. Combined with local variance calculation, the defect boundary is determined. When the variance of adjacent pixels exceeds 0.15, it is identified as a brightness abrupt change region. The second branch network converts the RGB image to CIELab space, extracts brightness gradient features in the L channel, and extracts red-green and yellow-blue hue features in the a and b channels, respectively. Cross-channel interference is eliminated through channel-specific convolution. The output color difference map is compared with the standard color chart for ΔE value. When ΔE exceeds 2.3, a color difference defect marker is triggered. The third branch network fuses light guide structure features of different scales through a pyramid structure. The bottom-level features capture the edge details of the light guide strip, and the top-level features identify macroscopic deformation caused by assembly offset. Finally, when generating the binary mask map, 0.5 is used as the segmentation threshold. The three branch networks share the backbone features while maintaining independent parameter updates. The dynamic gradient allocation mechanism avoids interference between tasks and improves the parallel processing efficiency of multi-defect detection.

[0109] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0110] A multi-task convolutional neural network consists of a shared backbone feature extraction network and three independent branch networks.

[0111] The backbone feature extraction network is based on the EfficientNet architecture and is used to extract basic spatial and channel features of images. EfficientNet employs a compound scaling method, simultaneously scaling the network's width, depth, and resolution to balance computational efficiency and accuracy. Specifically, the backbone network consists of eight moving inverted bottleneck (MBConv) convolutional blocks, each containing depthwise separable convolutions, a squeeze-and-excitation module, and skip connections.

[0112] The first branch network is based on a fully convolutional architecture, outputting a probability map of brightness non-uniformity and calculating the boundary of the defect region through local variance. This branch network consists of 5 convolutional layers, each followed by batch normalization and a ReLU activation function. The last layer uses a 1x1 convolution to reduce the number of channels in the feature map to 1, outputting a single-channel probability map of brightness non-uniformity.

[0113] The second branch network converts the RGB image to the CIELab color space and then performs channel-wise convolution processing to output a color difference map and calculates the deviation by comparing it with the standard calibration value. In the specific implementation, the RGB image is first converted to Lab space using the OpenCV library. Then, three convolutional layers are applied to the L, a, and b channels respectively for feature extraction. Finally, the feature maps of the three channels are concatenated and fused through 1x1 convolution to obtain the color difference map.

[0114] The third branch network employs a pyramid structure to fuse global and local features, generating a binary defect mask. This branch consists of four downsampling convolutional layers and four upsampling deconvolutional layers, forming a U-shaped structure. During downsampling, the receptive field is gradually increased to acquire global information, while during upsampling, feature maps of different scales are fused through skip connections. The final output is a binary mask with the same resolution as the input image.

[0115] Through the above technical solution, this application achieves multi-task parallel processing, improving detection efficiency. Shared features extracted by the backbone network can be reused by the three branch networks, reducing redundant computation. The three branch networks are optimized for brightness unevenness, color difference, and light guide defects, respectively, improving the detection accuracy of each indicator. The fully convolutional structure preserves spatial information, which is beneficial for locating defects. The pyramid structure integrates multi-scale features, enhancing adaptability to defects of different sizes.

[0116] In some of the solutions described above in this application, the image recognition results output by the multi-task convolutional neural network lack standardized definitions for defect categories, location coordinates, and confidence scores. This results in the inability to accurately associate defect types with tolerance thresholds during subsequent qualification determination processes, and the inability to distinguish the spatial distribution characteristics of different defect areas, affecting the accuracy of defect location and the reliability of determination.

[0117] This application further proposes that the defect categories in the image recognition results include brightness defects, color difference defects, and light guide defects. The position coordinates represent the segmentation mask of the defect region in pixels, and the confidence score is obtained by weighted fusion of the output values ​​of each branch network and then normalized based on sigmoid.

[0118] Specifically, brightness-related defects correspond to the brightness non-uniformity probability map output by the first branch network for uneven light emission recognition; color difference-related defects correspond to the deviation of the color difference map output by the second branch network from the standard calibration value; and light guide-related defects correspond to the binary defect mask map generated by the third branch network. The segmentation mask uses pixel coordinates to locate the defect region boundaries and determines the spatial distribution range of the defect region through local variance calculation. The confidence score is obtained by weighted fusion of the output values ​​of the three branch networks, and the sigmoid function is used to compress the fused value to the 0-1 range, forming a normalized score.

[0119] Specifically, the output of the multi-task convolutional neural network maps uneven emission regions using probability maps for brightness-related defects, quantifies color deviation using color difference maps, and marks structurally abnormal regions using binary masks for light-guiding defects. The segmentation mask defines defect boundaries using pixel coordinates and determines the spatial continuity of defect regions through local variance calculation, eliminating misjudgments of isolated noise points. The confidence score is generated by assigning different weight coefficients to the three branch networks, linearly superimposing the output values ​​of each network, and inputting the result into a sigmoid function to quantify the reliability of the detection results. For example, the confidence score for brightness-related defects can be calculated based on the weighted sum of the SSIM loss value and MSE error value of the first branch network; the confidence score for color difference-related defects can be calculated based on the weighted sum of the color difference deviation and divergence value of the second branch network; and the confidence score for light-guiding defects can be calculated based on the weighted sum of the Dice coefficient and IoU value of the third branch network. Through the normalized confidence score, the judgment threshold for different defect types can be dynamically adjusted, improving the accuracy of pass / fail judgments.

[0120] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0121] The defect categories in the image recognition results include brightness defects, color difference defects, and light guide defects. The location coordinates represent the segmentation mask of the defect region in pixels. The confidence score is obtained by weighted fusion of the output values ​​from each branch network and then normalized using sigmoid.

[0122] Specifically, the three branches of the multi-task convolutional neural network output different types of defect recognition results. The first branch outputs a brightness non-uniformity probability map to identify brightness-related defects. The second branch outputs a color difference map to identify color difference-related defects. The third branch outputs a binary defect mask map to identify light guide defects.

[0123] For location coordinates, pixel-level segmentation masks are used to accurately locate defect areas. For example, for a 1920x1080 resolution image, the mask for the defect area might be a binary image of the same size, where pixels with a value of 1 represent defect areas and pixels with a value of 0 represent normal areas.

[0124] The confidence score is calculated as follows: First, the output of each branch network is weighted. For example, brightness, color difference, and light guide defects can be assigned weights of 0.4, 0.3, and 0.3, respectively. Then, the weighted output values ​​are summed to obtain a comprehensive score. Finally, the sigmoid function is used to map this score to a range of 0 to 1, yielding the final confidence score.

[0125] The formula for the sigmoid function is: f(x) = 1 / (1 + e^(-x))-x ).

[0126] Where x is the weighted composite score, and f(x) is the final confidence score.

[0127] In this way, the system can simultaneously output the type, location, and credibility of the defect, providing comprehensive information support for subsequent defect identification and processing.

[0128] Through the above technical solutions, this application can achieve comprehensive and accurate identification of defects in ambient lighting. By classifying defects into three main categories—brightness, color difference, and light guide defects—the system can specifically detect different types of defects, improving the comprehensiveness of detection. Using pixel-level segmentation masks to represent defect locations allows for precise localization of defect areas, which is beneficial for subsequent defect processing and quality control. Introducing a confidence scoring mechanism and calculating it through weighted fusion of multiple branch networks comprehensively considers the importance of different types of defects, improving the reliability of detection results. Simultaneously, using the sigmoid function for normalization ensures that the confidence score has good interpretability and comparability. This multi-dimensional defect identification output provides a rich information foundation for subsequent defect judgment, analysis, and processing, contributing to improved quality control of ambient lighting.

[0129] In some of the above-mentioned schemes of this application, the multi-task convolutional neural network processes different types of defect detection tasks through multiple branch networks. However, during joint training, the gradient directions of each branch network conflict due to differences in task objectives, making it difficult to balance the detection accuracy of different defect types. In particular, when uneven brightness, color difference deviation, and light guide structure defects coexist, the network cannot effectively optimize the weight allocation of different branches, affecting the accuracy of the overall detection results.

[0130] This application further proposes a multi-task convolutional neural network that optimizes the loss function through joint training. The loss function includes: the first branch network uses structural similarity (SSIM) loss and mean squared error (MSE) to jointly construct a brightness unevenness detection loss term. The second branch network uses perceptual color difference loss and divergence under the Lab color space to construct a color difference recognition loss term. The third branch network uses a joint loss of Dice coefficient and IoU.

[0131] The loss term for brightness unevenness detection measures the similarity between the global brightness distribution of the image and the standard sample using Structural Similarity Same (SSIM), while using Mean Squared Error (MSE) to constrain local pixel brightness differences. The loss term for color difference recognition calculates the Euclidean distance between the predicted color difference map and the standard color difference map in the Lab color space and introduces KL divergence to measure the deviation in color distribution. The loss term for light guide structure defect detection optimizes the boundary matching degree of defect region segmentation using the Dice coefficient and enhances the overlap of mask regions by combining the IoU index.

[0132] Specifically, during the training phase, the first branch network compares the brightness distribution of the input image with that of the standard samples based on structural similarity. It calculates the brightness distribution difference using the SSIM function and penalizes local pixel brightness errors using the MSE function, enabling the network to capture both overall brightness anomalies and identify local bright spots or dark areas. The second branch network converts the RGB image to the Lab color space, calculates the brightness difference in the L channel, and the chromaticity difference in the a and b channels. It quantifies the degree of color difference deviation using Euclidean distance and evaluates the deviation trend of the chromaticity distribution from the standard samples using KL divergence. The third branch network calculates the boundary overlap ratio between the predicted mask and the ground truth mask using the Dice coefficient and optimizes the area overlap of defect regions using the IoU metric. The loss terms of the three branches are weighted and summed to form a joint loss function, dynamically adjusting the gradient contribution ratio of each branch during backpropagation. This allows the network to simultaneously optimize the accuracy of brightness uniformity detection, color difference discrimination, and light guide structure defect segmentation during training. For example, in the training data, when the proportion of light guide structure defect samples is less than 10%, the loss weight of the third branch network can be adjusted to 0.8 to enhance the learning ability of small sample defect categories.

[0133] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0134] Multi-task convolutional neural networks optimize the loss function through joint training. The loss function consists of three parts: the first branch network uses structural similarity loss (SSIM) and mean squared error (MSE) to jointly construct a brightness non-uniformity detection loss term. Specifically, SSIM loss measures the structural similarity between the predicted brightness map and the true brightness map, while MSE loss measures pixel-level differences. The combination of the two can more comprehensively evaluate the detection performance of brightness non-uniformity.

[0135] The second branch network uses perceptual color difference loss and divergence in the Lab color space to form the color difference recognition loss term. The perceptual color difference loss is calculated based on the CIE76 color difference formula to determine the Euclidean distance between the predicted color and the standard color, while the divergence measures the degree of difference between the predicted color distribution and the standard color distribution. This combination can simultaneously consider the absolute magnitude and distribution characteristics of the color difference.

[0136] The third branch network employs a joint loss of Dice coefficients and IoU. Dice coefficients measure the overlap between the predicted and actual masks, while IoU calculates their cross-union ratio. The combination of these two losses can better optimize the segmentation of defects in the light guide structure.

[0137] Through the above technical solutions, this application achieves joint optimization of multi-task convolutional neural networks, improving the accuracy and robustness of ambient light defect detection. Specifically, the joint loss of SSIM and MSE can more comprehensively evaluate brightness non-uniformity and improve detection sensitivity. Perceptual color difference loss and divergence based on Lab color space can more accurately identify color difference defects and reduce false positives. The joint loss of Dice coefficient and IoU helps to improve the segmentation accuracy of light guide structure defects. This multi-task joint optimization method can simultaneously meet the detection needs of different types of defects.

[0138] In some of the solutions mentioned above in this application, the dynamic tolerance threshold has insufficient adaptability in application. Different vehicle models and light guide schemes have significantly different tolerance requirements for color difference, brightness fluctuation and light guide defects. Fixed thresholds cannot cover the detection requirements of diverse products, leading to the risk of misjudgment or missed detection.

[0139] This application further proposes establishing a tolerance template library that matches defect categories with projects. The tolerance template library records the tolerance thresholds for color difference, brightness fluctuation, and light guide defects corresponding to different vehicle models and light guide schemes. Based on the defect category, location information, and confidence score output from the image recognition results, combined with the product structure information parsed from the product barcode, the corresponding tolerance threshold in the tolerance template library is called. Tolerance judgment is performed on the identified defects. For brightness defects, the uniformity index and local brightness fluctuation are judged to see if they exceed the brightness tolerance threshold; for color difference defects, the color difference value is judged to see if it exceeds the color difference tolerance threshold; and for light guide defects, the mask area is judged to see if it exceeds the light guide tolerance threshold.

[0140] The tolerance template library is indexed using predefined vehicle model identifiers and light guide scheme codes. Each index entry stores the numerical ranges of brightness tolerance thresholds, color difference tolerance thresholds, and light guide tolerance thresholds. The brightness tolerance threshold is set between 0.8 and 0.95 based on a uniformity index; the color difference tolerance threshold is set between 1.2 and 3.0 based on the ΔE value in the CIELab color space; and the light guide tolerance threshold is set between 0.5% and 2% based on the proportion of the mask area to the total image area. When calling the tolerance thresholds, the vehicle model code and light guide scheme field in the product barcode are parsed, the corresponding index entry in the template library is matched, and the threshold parameters are extracted. During tolerance judgment, brightness defects are calculated using local variance and compared with the threshold range; color difference defects are compared with the threshold using the ΔE value in the color difference image; and light guide defects are compared with the threshold using the proportion of the mask area.

[0141] Specifically, the tolerance template library is constructed by collecting optical and structural parameters from historically qualified products, combining them with manually calibrated acceptable defect ranges to generate initial thresholds, and then using a machine learning model to iteratively optimize multiple batches of test data, forming a dynamically updated threshold database. During the inspection process, the system parses the vehicle model and light guide scheme based on the product barcode. For example, for a vehicle model with code C202, the light guide scheme is identified as G5, and the corresponding brightness tolerance threshold of 0.85, color difference tolerance threshold of 2.5, and light guide tolerance threshold of 1.2% are extracted from the template library. When detecting brightness-related defects, the local area brightness variance is calculated; if the variance value exceeds the fluctuation limit corresponding to 0.85, it is judged as unqualified. When detecting color difference-related defects, the mean ΔE of the current color difference image is calculated; if it exceeds 2.5, an alarm is triggered. When detecting light guide-related defects, the proportion of the mask area to the total image area is statistically analyzed; if it exceeds 1.2%, it is marked as a structural defect. This solution effectively solves the problem that traditional fixed thresholds cannot adapt to diverse products by dynamically matching thresholds, improving the accuracy and adaptability of defect judgment.

[0142] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0143] Establish a tolerance template library that matches defect categories with projects. The tolerance template library records the tolerance thresholds for color difference, brightness fluctuation, and light guide defects corresponding to different vehicle models and light guide schemes. For example, for a certain model of ambient light product, its color difference tolerance threshold is set to ΔE*ab≤3.0, its brightness uniformity index tolerance threshold is set to ≥0.85, and its light guide defect area tolerance threshold is set to ≤0.5mm2.

[0144] Based on the defect category, location information, and confidence score output from the image recognition results, and combined with the product structure information parsed from the product barcode, the tolerance threshold for the corresponding item in the tolerance template library is retrieved. Specifically, the model information of the ambient light is obtained through the product barcode, and the corresponding tolerance threshold parameter is extracted from the tolerance template library.

[0145] Tolerance assessment is performed on the identified defects. Among other things:

[0146] For brightness-related defects, the uniformity index and local brightness fluctuations are assessed to determine if they exceed the brightness tolerance threshold. For example, the brightness uniformity index of the ambient light's luminous area is calculated; if it is below 0.85, it is considered unacceptable. Simultaneously, local brightness fluctuations are detected; if the brightness change in a certain area exceeds ±10%, it is also considered unacceptable.

[0147] For color difference defects, determine whether the color difference value exceeds the color difference tolerance threshold. For example, calculate the CIELab color difference value ΔE*ab of the ambient light emission area; if it exceeds 3.0, it is considered unacceptable.

[0148] For light guide defects, determine whether the mask area exceeds the light guide tolerance threshold. For example, calculate the pixel area of ​​the defective region and convert it to the actual area based on the image resolution. If it exceeds 0.5 mm², it is considered unqualified.

[0149] Through the above technical solution, this application achieves ambient lighting defect detection based on dynamic tolerance thresholds. Therefore, the detection system can automatically adjust the judgment criteria according to different vehicle models and light guide schemes, improving the flexibility and adaptability of the detection. Furthermore, by combining defect category, location information, and confidence scores, the severity of defects can be assessed more comprehensively, avoiding misjudgments that may result from judging by a single indicator. Specifically, for brightness-related defects, both overall uniformity and local fluctuations are considered, enabling more accurate identification of unevenness affecting visual effects. For color difference-related defects, the CIELab color difference evaluation system is adopted, which is closer to human visual perception. For light guide defects, area threshold judgment can effectively filter out minor, harmless defects, reducing the false alarm rate. Therefore, the solution of this application improves the accuracy and reliability of ambient lighting defect detection, helping to ensure product quality consistency.

[0150] In some of the solutions described above in this application, although the combination of image recognition and assembly structure measurement data can detect some defects, in practical applications, defects caused by assembly misalignment may not be accurately identified due to the lack of precise three-dimensional geometric data. For example, minute misalignments at the light guide interface or bracket joint may not be captured by traditional two-dimensional image detection, leading to missed detection.

[0151] This application further proposes setting up a three-dimensional contour measurement sensor to acquire the three-dimensional geometric data of the target ambient light based on laser triangulation. According to the reference points and assembly tolerance zones set in the structural model, coordinate fitting and boundary curvature analysis are performed on the acquired three-dimensional geometric data to generate a spatial offset vector for the current measured component. The spatial offset vector is then compared item by item with an offset threshold at each structural connection feature point. If the spatial offset vector exceeds the offset threshold at any connection point, the assembly identification result is deemed unqualified.

[0152] The 3D contour measurement sensor employs laser triangulation technology, achieving a measurement accuracy of ±0.01 mm, capable of capturing minute deformations at the connection points of the light guide structure. Coordinate fitting eliminates positional deviations during measurement by rigidly registering the 3D point cloud data with a preset reference coordinate system. Boundary curvature analysis uses a differential geometry-based algorithm to calculate the curvature changes at the edges of the light guide structure, identifying local deformations caused by assembly offsets. The spatial offset vector quantifies the displacement direction and magnitude of each connection point through vector decomposition, calculating the offset along the X, Y, and Z axes respectively.

[0153] Specifically, a 3D contour measurement sensor scans the surface of the target ambient light in a non-contact manner, generating 3D point cloud data containing height information. A coordinate system is established using reference points (such as the center point of the light guide groove or the mounting hole of the bracket), and the measured data is registered with the theoretical model to ensure consistency of the measurement reference. Boundary curvature analysis identifies abnormal curvature peaks caused by assembly misalignment by calculating the curvature distribution at the edge of the light guide structure. For example, when the curvature at the connection between the light guide groove and the bracket exceeds a preset threshold, it is determined that there is an assembly offset in that area. The spatial offset vector is compared with the assembly tolerance zone at key connection points (such as snap-fit ​​positions or welding points). If the offset in any direction exceeds the allowable range (e.g., X-axis offset exceeds 0.2 mm), the assembly is deemed unqualified. This solution effectively distinguishes between optical defects and assembly defects through high-precision 3D measurement and multi-dimensional data analysis, avoiding misjudgments or missed detections caused by structural misalignment.

[0154] As a preferred embodiment, the solution of this application is implemented as follows: A three-dimensional contour measurement sensor is installed on a fixed bracket at the inspection station. A laser triangulation ranging module projects linear laser stripes onto the surface of the target ambient light in an oblique incidence manner. The reflected light spot is captured by a high-resolution CMOS image sensor and converted into three-dimensional point cloud data. Based on a pre-imported CAD structural model, the reference point is set as the center coordinates of the four positioning posts at the interface between the light guide plate and the bracket. The assembly tolerance zone is defined as a cylindrical spatial range of ±0.2mm. The acquired three-dimensional geometric data is fitted to coordinates using the least squares method. The boundary curvature analysis uses a Gaussian curvature algorithm to calculate the surface concave and convex features, generating a spatial offset vector containing X / Y / Z axial offsets. At the three structural connection feature points of the light guide plate buckle, bracket screw hole, and PCB fixing slot, the magnitude of the spatial offset vector is compared with the corresponding offset threshold. When the magnitude of any feature point exceeds the threshold, the assembly recognition result is marked as unqualified.

[0155] Through the above technical solution, this application achieves accurate identification of assembly defects, effectively distinguishes between light guiding abnormalities caused by structural connection offset and optical body defects, avoids the problem of missed judgment caused by the lack of quantitative detection of assembly errors in traditional methods, and improves the accuracy of defect attribution analysis.

[0156] In some of the solutions described above in this application, optical defects of ambient lights are detected by image recognition methods. However, these methods cannot effectively identify defects caused by minor assembly offsets at the light guide interface and bracket joint, such as misalignment of the connecting groove or abnormal light refraction caused by uneven force on the bracket. These assembly defects are missed due to the lack of three-dimensional geometric data support.

[0157] This application further proposes a method for acquiring assembly structure measurement data of a target ambient light, comparing this measurement data with offset thresholds for structural connection points, and determining the assembly identification result. This includes: setting up a three-dimensional contour measurement sensor, which acquires three-dimensional geometric data of the target ambient light based on laser triangulation. Based on the reference points and assembly tolerance zones set in the structural model of the target ambient light, coordinate fitting and boundary curvature analysis are performed on the acquired three-dimensional geometric data to generate a spatial offset vector for the current measured component. The spatial offset vector is then compared item by item with the offset threshold at each structural connection feature point. If the spatial offset vector exceeds the offset threshold at any connection point, the assembly identification result is deemed unqualified.

[0158] The 3D contour measurement sensor employs the laser triangulation principle, calculating 3D coordinates by emitting a laser beam and receiving changes in the position of the reflected light spot, with measurement accuracy controlled within ±0.05 mm. The reference point is determined based on key assembly positioning points in the structural model, such as the center point of the light guide strip and bracket snap-fit ​​or the apex of the injection-molded positioning column. During coordinate fitting, the measured 3D geometric data is matched with the theoretical model using least-squares to eliminate overall positional deviations. Boundary curvature analysis identifies local deformation areas by calculating the rate of change of surface curvature. The spatial offset vector contains the displacement components of each feature point in the X, Y, and Z directions. The offset threshold is set based on the material's thermal expansion coefficient and assembly process capabilities; for example, ±0.2 mm is allowed in the X direction, ±0.15 mm in the Y direction, and ±0.1 mm in the Z direction.

[0159] Specifically, the 3D contour measurement sensor moves along the surface of the target ambient light in a line scanning mode, collecting 2000 3D point cloud data points per second to form a complete 3D geometric model. Reference points and assembly tolerance zones are extracted from product design drawings; for example, the four buckle centers connecting the light guide strip and the bracket are selected as reference points, and the assembly tolerance zone is set to ±5% of the buckle groove width. During coordinate fitting, the ICP algorithm is used to align the measured point cloud with the theoretical model, eliminating differences in installation posture. Boundary curvature analysis identifies abnormal areas exceeding 0.5 radians / mm by calculating the rate of change of the angle between the normal vectors of adjacent point clouds. The spatial offset vector is decomposed into axial components at feature points; if any directional component exceeds a threshold, for example, an X-direction offset of 0.25 mm at the light guide strip buckle, the assembly is deemed unqualified. This process quantifies 3D deformation data to accurately identify structural defects caused by assembly misalignment, avoiding misjudgments due to a disconnect between optical and structural inspection.

[0160] As a preferred embodiment, the specific implementation of this application is as follows: In the assembly structure measurement stage, a three-dimensional contour measurement sensor acquires the three-dimensional geometric data of the target ambient light through the laser triangulation principle. Based on the preset structural model reference point, the collected point cloud data is aligned with the theoretical model in the coordinate system, and spatial registration is achieved through the least squares method. Discrete differential analysis is performed on the boundary curvature at the assembly interface to calculate the normal vector deviation between the actual assembly surface and the theoretical surface, generating a spatial offset vector containing X / Y / Z three-axis offsets. At six key connection points, such as the light guide strip buckle mounting point and the PCB board fixing slot, the offsets in each direction are compared with the preset 0.2mm assembly tolerance zone item by item. When the X-axis offset of a certain connection point reaches 0.25mm and the Y-axis offset reaches 0.18mm, it is determined that the offset exceeds the offset threshold. At this time, the assembly recognition result is marked as unqualified, and the offset data is spatially mapped with the light guide structure defect mask in the image recognition result. If the overlap between the mask area and the offset area of ​​the PCB board fixing groove in three-dimensional space reaches 65% or more, it is determined to be a structural defect caused by assembly. When image recognition shows a color difference defect but the offset of all assembly connection points is below the threshold, it is determined to be an optical material color temperature abnormality defect. If image recognition shows no defect but the light guide strip buckle mounting point has an offset of 0.22mm, the potential assembly hazard is marked in the detection result and associated with the production batch number.

[0161] Through the above technical solution, this application effectively solves the problems of missed detection and misjudgment of defect types in traditional detection methods. By jointly analyzing the results of three-dimensional contour measurement and image recognition, it is possible to clearly distinguish between optical defects and indirect defects caused by assembly misalignment, avoiding misjudgments caused by a single detection dimension. For cases where the assembly is qualified but optical defects exist, the problem can be directly located to material or process issues. For areas where assembly misalignment and image defects overlap, the adjustment requirements of the assembly process can be quickly traced. At the same time, a hidden danger recording mechanism is established for cases where assembly misalignment does not cause immediate defects, providing data support for product reliability analysis and improving the accuracy of defect attribution analysis and the decision-making value of detection results.

[0162] The above embodiments address the problems of low recognition accuracy, weak defect attribution ability, fixed judgment criteria, and easy omission of assembly deviations in traditional detection methods by constructing an ambient light defect detection process that integrates multi-angle multispectral image acquisition, structural parameter binding, image preprocessing, multi-task convolutional neural network recognition, dynamic tolerance judgment, and 3D assembly measurement. Multi-angle lighting and multispectral exposure devices enhance image detail representation; structural information is bound to chip model for adaptive parameter adjustment of the recognition model; multi-task neural networks are used for parallel recognition of uneven brightness, color differences, and light guide structure defects; and 3D contour measurement data is combined to determine spatial offset of key assembly connection parts. Defect attribution and level determination are completed through joint decision-making of images and assembly results, improving the accuracy and comprehensiveness of defect detection and achieving intelligent, standardized, and traceable quality management.

[0163] In another preferred embodiment based on the above embodiments, see [reference] Figure 2 As shown, this embodiment provides a defect detection system for ambient lights, which applies the above-described defect detection method for ambient lights, including:

[0164] The acquisition unit is configured to acquire RGB and near-infrared images of the target ambient light.

[0165] The processing unit is configured to preprocess all images and bind the preprocessed images with product structure information, chip model and light guide structure type. The preprocessing includes noise reduction, alignment, cropping and color calibration.

[0166] The judgment unit is configured to input the preprocessed image into a multi-task convolutional neural network, which includes a first branch network for uneven light emission recognition, a second branch network for color difference discrimination, and a third branch network for light guide structure defect detection, to obtain the image recognition result output by the multi-task convolutional neural network. The image recognition result includes defect category, location coordinates, and confidence score, and the image recognition result is judged to be qualified according to the dynamic tolerance threshold.

[0167] The identification unit is configured to acquire the assembly structure measurement data of the target ambient light, compare the assembly structure measurement data with the offset threshold of the structural connection part, and determine the assembly identification result.

[0168] The detection unit is configured to determine the defect detection results of the target ambient light based on the image recognition results and the assembly recognition results.

[0169] Specifically, the acquisition unit simultaneously acquires RGB and near-infrared images using a multi-channel industrial camera, combined with a wide-spectrum LED array lighting unit to cover imaging requirements for different material transmittances and structural thicknesses. The processing unit dynamically binds images and parameters by associating product barcodes with product structure information, chip models, and light guide structure types from a database. The judgment unit employs a multi-task convolutional neural network, with three branches handling brightness unevenness, color difference, and light guide structure defects respectively, and optimizing the loss function through joint training to improve detection accuracy. The recognition unit uses a 3D contour measurement sensor to acquire 3D geometric data and performs spatial offset analysis based on a reference point and assembly tolerance zone. The detection unit makes a joint decision based on the spatial overlap between image recognition results and assembly recognition results to distinguish between optical and structural defects.

[0170] Specifically, the acquisition unit uses a broadband LED array lighting unit positioned above and to the side of the target ambient light, combined with a multi-channel industrial camera's multi-spectral exposure device, to adjust the exposure time and spectral channels, ensuring imaging quality for materials with different transmittance. The processing unit performs denoising, alignment, cropping, and color calibration on the images, then fuses the preprocessed data with structural information from the product parameter database to generate a product dataset containing chip model and light guide type. The judgment unit inputs the dataset into a multi-task convolutional neural network. The first branch network outputs a brightness unevenness probability map based on a fully convolutional architecture; the second branch network performs channel-wise convolution processing in the CIELab color space to generate a color difference map; and the third branch network uses a pyramid structure to fuse global and local features to generate a light guide defect mask map. The recognition unit acquires three-dimensional geometric data through laser triangulation, fits a reference point to the assembly tolerance zone, calculates the spatial offset vector, and compares it with a threshold. The detection unit determines the defect type as optical or structural based on the overlap between the defect area and the assembly offset area, and records the assembly defect in the historical work order.

[0171] As a preferred embodiment, the solution of this application is implemented as follows: The acquisition unit uses a multi-channel industrial camera and a wide-spectrum LED array to construct an image acquisition module. The multi-channel industrial camera is vertically mounted above the inspection station. The wide-spectrum LED array includes two sets of lighting units respectively positioned 30 cm above the target ambient light and 45 degrees to the side. The two sets of lighting units use LED beads with color temperatures of 6500K and 3000K, respectively, to be alternately lit. The processing unit performs Gaussian noise reduction on the acquired RGB and near-infrared images through an image preprocessing module, uses SIFT feature point matching to achieve multispectral image alignment, and crops the effective detection area of ​​the light guide strip according to the preset ROI region. At the same time, the processed image data is data-bound with the chip model and light guide structure code read from the MES system. The judgment unit deploys a multi-task convolutional neural network model, with the backbone network adopting the EfficientNet-B4 structure. The three branch networks are configured to output a 256×256 resolution brightness distribution map, a color difference distribution map, and a binary mask for light guide defects, respectively. The dynamic tolerance judgment module automatically matches standard parameters of brightness fluctuation threshold ±5% and color difference ΔE ≤ 1.5 based on the product code. The recognition unit acquires 3D point cloud data of the light guide strip buckle mounting surface using a line laser 3D scanner. It uses the ICP algorithm to register the measured point cloud with the CAD model, outputting the XYZ axial offset of each connection point. The detection unit establishes a rule engine. When the image recognition module detects excessive color difference and the corresponding assembly offset is < 0.2 mm, it determines it as an optical defect. If the area with excessive color difference is accompanied by an assembly offset ≥ 0.2 mm, a structural defect alarm is triggered. Simultaneously, detection results with assembly offsets between 0.1 and 0.2 mm are marked as potential hazards and stored in the database.

[0172] Through the above technical solutions, this application effectively solves the technical problem that traditional detection methods cannot distinguish between optical defects and assembly defects. By combining multispectral image fusion and three-dimensional measurement data analysis, it achieves precise location of color difference and brightness anomalies, as well as intelligent identification of defect causes. The adoption of a dynamic tolerance judgment mechanism avoids misjudgment caused by fixed thresholds, and the binding of product parameters enhances the adaptability of the detection system. The introduction of a joint decision rule engine improves the accuracy of identifying optical defects caused by assembly offsets, while ensuring closed-loop feedback between detection results and production processes.

[0173] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0175] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A defect detection method for an atmosphere lamp, characterized by, The method comprises the following steps: An image acquisition device is arranged to acquire RGB images and near-infrared images of the target atmosphere lamp, wherein the image acquisition device comprises at least two angle light distribution units and a multi-spectral exposure device; All images are preprocessed, and the preprocessed images are bound with product structure information, chip model and light guide structure type, wherein the preprocessing includes denoising, alignment, cutting and color calibration processing; The preprocessed images are input into a multi-task convolutional neural network, wherein the multi-task convolutional neural network comprises a first branch network for light emission unevenness identification, a second branch network for color difference discrimination and a third branch network for light guide structure defect detection, image recognition results output by the multi-task convolutional neural network are obtained, the image recognition results comprise defect categories, position coordinates and confidence scores, and the image recognition results are qualified according to a dynamic tolerance threshold; The assembly recognition result is determined by comparing the assembly structure measurement data of the target atmosphere lamp with a structure connection part offset threshold; The defect detection result of the target atmosphere lamp is determined according to the image recognition result and the assembly recognition result.

2. The defect detection method for an atmosphere lamp according to claim 1, characterized by, When the image acquisition device is arranged to acquire RGB images and near-infrared images of the target atmosphere lamp, the method comprises the following steps: Two light distribution units at different angles above and on the side of the target atmosphere lamp are arranged, and the light distribution units adopt a wide-spectrum LED array; A multi-channel industrial camera is arranged directly above the detection station, the multi-channel industrial camera simultaneously acquires RGB images and near-infrared images, and the multi-channel industrial camera further comprises a multi-spectral exposure device, the multi-spectral exposure device adjusts the exposure time and the spectral channel according to the material light transmittance and the structure thickness of the target atmosphere lamp.

3. The defect detection method for an atmosphere lamp according to claim 1, characterized by, When all images are preprocessed and the preprocessed images are bound with product structure information, chip model and light guide structure type, the method comprises the following steps: A product parameter database is established, which records product structure information, chip model and light guide structure type corresponding to different models of atmosphere lamps; The product barcode of the target atmosphere lamp is acquired, and the product structure information, chip model and light guide structure type of the corresponding product are extracted from the product parameter database according to the product barcode; The images are subjected to denoising, alignment, perspective cutting and color space calibration, and the preprocessed images are data-fused with the extracted parameters to generate a product data set.

4. The defect detection method for an atmosphere lamp according to claim 3, characterized by, The multi-task convolutional neural network comprises: A shared main feature extraction network and three independent branch networks; The main feature extraction network is based on the EfficientNet structure and is used to extract the basic spatial features and channel features of the image; The first branch network is based on a full convolution architecture, outputs a brightness unevenness probability map, and calculates the defect area boundary through local variance; The second branch network converts the RGB image into a CIELab color space, then performs channel-by-channel convolution processing, outputs a color difference map and calculates the deviation amount by comparing a standard calibration value; The third branch network adopts a pyramid structure to fuse global and local features and generates a binary defect mask map.

5. The defect detection method for an atmosphere lamp according to claim 4, characterized by, The image recognition result output by the multi-task convolutional neural network is obtained, comprising: The defect categories in the image recognition result include brightness category defects, color difference category defects, and light guide category defects; The position coordinates express the segmentation mask of the defect area in units of pixels, and the confidence score is obtained by weighting and fusing the output values of each branch network and based on sigmoid normalization processing.

6. The defect detection method for an atmosphere lamp according to claim 5, characterized by, The multi-task convolutional neural network optimizes the loss function through joint training, and the loss function comprises: The first branch network adopts a structural similarity SSIM loss and a mean square error MSE to jointly construct a brightness unevenness detection loss term; The second branch network adopts a perceptual color difference loss and a divergence in the Lab color space to constitute a color difference recognition loss term; The third branch network adopts a Dice coefficient and an IoU joint loss.

7. The defect detection method for an atmosphere lamp according to claim 6, characterized by, The image recognition result is qualified when the dynamic tolerance threshold is determined, comprising: A tolerance template library is established to match the defect categories and the project, and the tolerance template library records the tolerance thresholds of color difference, brightness fluctuation, and light guide defects corresponding to different vehicle models and light guide schemes; According to the defect categories, position information, and confidence score output in the image recognition result, combined with the product structure information parsed from the product barcode, the tolerance threshold of the corresponding item in the tolerance template library is called; The identified defects are tolerance judged, wherein: For the brightness category defects, it is judged whether the uniformity index and the local brightness fluctuation exceed the brightness tolerance threshold; For the color difference category defects, it is judged whether the color difference value exceeds the color difference tolerance threshold; For the light guide category defects, it is judged whether the mask area exceeds the light guide tolerance threshold.

8. The defect detection method for an atmosphere lamp according to claim 7, characterized by, When obtaining the assembly recognition result by comparing the assembly structure measurement data of the target atmosphere lamp with the structure connection part offset threshold, comprising: A three-dimensional profile measurement sensor is set, which collects three-dimensional geometric data of the target atmosphere lamp based on laser triangulation; According to the reference point and the assembly tolerance band set in the structure model of the target atmosphere lamp, the collected three-dimensional geometric data is coordinate fitted and boundary curvature analyzed to generate a space offset vector of the current measured piece; The space offset vector is compared with the offset threshold at the structure connection feature point; If the space offset vector exceeds the offset threshold at any connection point, the assembly recognition result is generated as unqualified.

9. The defect detection method for an atmosphere lamp according to claim 8, characterized by, When determining the defect detection result of the target atmosphere lamp according to the image recognition result and the assembly recognition result, comprising: The image recognition result and the assembly recognition result are jointly decided, wherein: If there is a defect in the image recognition result but the corresponding area is qualified in the assembly recognition, it is judged as an optical defect; If there is a defect in the image recognition result and the spatial overlap degree of its defect position and the assembly offset area exceeds a preset threshold, it is judged as a structural defect caused by assembly; If the image recognition result is qualified but the assembly recognition is unqualified, it is marked as a potential assembly hidden danger and recorded in the history work order.

10. A defect detection system for an atmosphere lamp for applying the defect detection method for an atmosphere lamp according to any one of claims 1 to 9, characterized by Comprising: The acquisition unit is configured to acquire RGB images and near-infrared images of the target atmosphere lamp; The processing unit is configured to pre-process all images, and bind the pre-processed images with product structure information, chip model and light guide structure type, wherein the pre-processing includes denoising, alignment, cutting and color calibration processing; The judging unit is configured to input the pre-processed images into a multi-task convolutional neural network, wherein the multi-task convolutional neural network includes a first branch network for light emission unevenness identification, a second branch network for color difference discrimination, and a third branch network for light guide structure defect detection, obtain image recognition results output by the multi-task convolutional neural network, wherein the image recognition results include defect categories, position coordinates and confidence scores, and perform qualified judgment on the image recognition results according to a dynamic tolerance threshold; The identifying unit is configured to obtain assembly structure measurement data of the target atmosphere lamp, compare the assembly structure measurement data with a structure connection part offset threshold, and determine an assembly recognition result; The detecting unit is configured to determine a defect detection result of the target atmosphere lamp according to the image recognition result and the assembly recognition result.

Images