Automobile LED module production monitoring system and method based on Internet of Things

Abstract

The invention relates to the field of LED production, in particular to an automobile LED module production monitoring system and method based on the Internet of Things, and the system comprises a chip calibration module, a wafer recognition module, a light source training module, a quality monitoring module and a defect supervision module. The wafer recognition module is used for extracting chip wafer images, the light source training module is used for toning the images, the quality monitoring module is used for screening defect areas, and the defect supervision module is used for outputting surface defect types. The defect detection speed is improved, the production cost is reduced, the detection generalization ability in a complex illumination environment is improved, the detection efficiency and consistency are improved, the LED chip production efficiency is improved, the dispersion correction precision is improved, and the automatic detection efficiency of a production line is improved.

Description

Technical Field

[0001] This invention relates to the field of LED manufacturing, specifically to an Internet of Things-based automotive LED module production monitoring system and method. Background Technology

[0002] An LED module is a lighting component assembled and packaged with LED light-emitting diodes. It has an independent circuit structure and boasts advantages such as low power consumption, long lifespan, and high luminous efficacy. It is commonly used in large-scale LED displays or automotive lighting systems to improve adaptability to LED lighting needs in various scenarios. Automotive LED modules typically consist of LED beads, LED chips, and connection interfaces. The LED chip, which controls the brightness and color of the LED, is the key component of the entire LED module, and its production is the most crucial step in module manufacturing.

[0003] In the LED chip manufacturing process, a die bonder is needed to pick up the chips and solder different LED wafers into the chip. Due to the complex factory environment and variable production conditions, the operating status of the die bonder is affected, which can easily lead to problems such as chip surface defects, misalignment, and breakage. It is necessary to inspect the soldering quality of the wafers. Manual inspection is inefficient and has a high rate of missed detection. Photoelectric sensors have poor inspection capabilities for small defects and are easily affected by changes in light, resulting in insufficient detection capabilities.

[0004] In addition, during chip inspection, three-color ring light is often used to illuminate the chip to improve the defect recognition capability under different colors. Compared with point light source, it can achieve more refined texture anomaly detection. However, in actual production, problems such as uneven illumination and color distortion often occur, which affect the inspection efficiency. Ring light can also affect camera calibration parameters, resulting in calibration errors introduced by camera quality, and reducing the identifiability of chip faults. Summary of the Invention

[0005] The purpose of this invention is to provide an Internet of Things-based automotive LED module production monitoring system and method to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an Internet of Things-based automotive LED module production monitoring system, comprising: a chip calibration module, a wafer identification module, a light source training module, a quality monitoring module, and a defect monitoring module;

[0007] The chip calibration module is used to deploy different cameras to capture the chip production area, calibrate the LED chip position and perform chain code contour tracking, extract the edge contour of the LED chip image, decompose the edge contour points into a fixed number, make the reference image and the target image point set correspond, obtain the transformation matrix between the camera's image coordinate system and the reference coordinate system, use the projection positioning algorithm to project all chip images onto the standard coordinate system, select the image with the smallest projection standard deviation, correct the calibration deviation based on geometric coordinate transformation, and straighten the chip to obtain the standard image.

[0008] The wafer recognition module is used to set the contrast between the solder line and the image edge line by setting the second derivative threshold, segment the solder joint by adaptive threshold, obtain the solder line by dilation algorithm, divide the region by the solder line as the boundary, match the gray-scale features of the wafer region by normalized cross-correlation algorithm, determine the wafer connected region, perform a closing operation on the connected region, locate the wafer by the minimum circumscribed moment of the connected region, and obtain the chip wafer image.

[0009] The light source training module is used to apply the particle swarm optimization algorithm to adjust the brightness of each LED in the ring lamp, so that the grayscale of the chip image is uniform. The visual parameters under monochromatic light illumination and the corresponding grayscale images are used as the training set. The training set is enhanced by a pseudo-anomaly simulation strategy. The training set is input into the neural network model and outputs the grayscale mapping equation under point light source radiation. A grayscale mapping equation system is established for all point light sources under three-color ring illumination conditions. The grayscale dispersion parameters of the chip image are obtained by solving the equation system. A color space conversion model is established according to the grayscale dispersion parameters and the chip image is color-corrected.

[0010] The quality monitoring module is used to acquire the image features of LED chips, including the spacing between wafers, average grayscale value, length and width, area and center distance, and compare them with the image features of standard chips. The comparison results determine the chip production quality, and the chip is judged as a good product according to the production quality. At the same time, for defective chips, defective areas are screened out and defective area images are output.

[0011] The defect monitoring module is used to detect sub-pixel edges of defect areas, perform Euclidean clustering on edge points, fit the gray-level gradient direction along the image edge using a fourth-order polynomial, detect the gray-level curve of foreign objects, fit the gray-level curve by an ellipse, calculate the geometric centroid coordinates and deflection angle of the foreign object defect, and generate feature vectors using the geometric centroid coordinates, deflection angle, gray level, and pixel area. All feature vectors in the defect area constitute a feature matrix, which is input into an SVM classifier of a support vector machine to classify surface defect traces and output the surface defect type.

[0012] Furthermore, the chip calibration module includes: a contour tracking unit and a coordinate projection unit;

[0013] The contour tracking unit is used to set up the camera and calibration plate so that the field of view covers the stage area, uniformly sample the contour points of the LED chip, select an auxiliary coordinate system, align the sample point set by iterating the nearest point and perform homography matrix transformation, and calculate the transformation matrix from the image coordinate system to the reference coordinate system.

[0014] The coordinate projection unit is used to project all chip images to the standard coordinate system using a transformation matrix, calculate the standard deviation of each projected image in the standard coordinate system, select the image with the smallest standard deviation as the reference to correct the image, eliminate pose deviation, and obtain the standard image of the chip.

[0015] Furthermore, the wafer identification module includes: a wire bonding extraction unit and a connectivity segmentation unit;

[0016] The wire bonding extraction unit is used to enhance the contrast between lines and background at the junction of the wafer and the substrate on a standard image using a second derivative operator, perform morphological recognition based on regions with contrast higher than a threshold, extract wire bonding endpoints, and dilate the wire bonding endpoints to obtain connected wire bonding regions.

[0017] The connected segmentation unit is used to divide the image into ROI blocks with the bonding wire area and the outer contour of the chip as the boundary. Within the ROI, normalized cross-correlation is used to match with the standard wafer grayscale texture template. Regions with high matching confidence are binarized and connected component analysis is performed to obtain a standardized wafer image.

[0018] Furthermore, the light source training module includes: an illumination unit, a grayscale mapping unit, and a feature extraction unit;

[0019] The ensemble illumination unit is used to construct a fitness function with the goal of minimizing the grayscale standard deviation of the wafer region, simulate all LED brightness configuration schemes as particles, construct a particle swarm model, use the fitness function to evaluate the quality of wafer images captured under different particle configurations, output the configuration scheme with the highest quality and make adjustments.

[0020] The grayscale mapping unit is used to illuminate the chip sample with a monochromatic point light source in a dark room environment, record visual parameters, including incident angle, wavelength, intensity and wafer image grayscale, form training data pairs, construct a neural network model, input the visual parameter vector of the point light source, and output the grayscale distribution equation of the wafer surface.

[0021] The feature extraction unit is used to superimpose the gray-level distribution equations under all light sources to form a mapping equation set. Solving the mapping equation set yields the gray-level dispersion parameters of the wafer image surface, correcting the color shift caused by uneven illumination spectrum.

[0022] Furthermore, the quality monitoring module includes: an image comparison unit and a good product classification unit;

[0023] The image comparison unit is used to extract image feature vectors from known good chip images and compare them item by item with the feature vectors of the current chip wafer image. By judging whether each feature is within the tolerance threshold, it is determined whether the chip is good.

[0024] The good product classification unit is used to filter out areas that do not meet the standards based on the comparison results of image features, and mark them as defective areas. The filtered defective areas are then labeled to generate defective area images.

[0025] Furthermore, the defect monitoring module includes: a foreign object segmentation unit, a surface defect unit, and a classification and identification unit;

[0026] The foreign object segmentation unit is used to delineate the invalid area by using a trapezoidal region. Within the delineated trapezoidal effective area, the edge of the foreign object area is delineated by Zernike moment or grayscale moment, and the sub-pixel edge points are clustered based on Euclidean distance.

[0027] The surface defect unit is used to calculate the gray-level gradient direction for the clustered edge points, extract a narrow band region along the gray-level gradient normal direction, fit the gray-level value change curve in the normal direction using a fourth-order polynomial, connect all the edge points within the cluster to form a defect contour, and perform least-squares ellipse fitting on the extracted defect contour to construct structured data.

[0028] The classification and recognition unit is used to generate high-dimensional feature vectors for structured data, load defect vector training sets from the cloud to train a support vector machine classifier, and output chip defect type and confidence level.

[0029] A method for monitoring the production of automotive LED modules based on the Internet of Things (IoT) includes the following steps:

[0030] Step S1. Photograph the chip production area, mark the LED chip position, extract the edge contour of the LED chip image, project all chip images onto the standard coordinate system, select and correct the image with the smallest standard deviation of the projection to obtain the standard chip image;

[0031] Step S2. Identify solder joints by contrasting the solder lines and image edge lines. Solder joints expand to obtain solder lines. Divide regions using solder lines as boundaries, match grayscale features of each region, determine connected regions of the wafer, perform a closing operation on the connected regions, and locate the wafer using the minimum circumscribed moment of the connected regions to obtain the wafer image.

[0032] Step S3. Apply the particle swarm optimization algorithm to adjust the brightness of each LED in the ring lamp. Use the visual parameters and grayscale images under monochromatic light illumination as the training set to train the neural network model, output the grayscale mapping equation under point light source radiation, solve the grayscale mapping equation set established by all point light sources, obtain the grayscale dispersion parameters of the image, and adjust the color of the chip image.

[0033] Step S4. Obtain chip image features, compare them with the image features of standard chips, determine chip production quality, determine whether the chip is a good product, and for defective chips, screen out the defective areas of the wafer image and output the defective area image.

[0034] Step S5. Detect sub-pixel edges of the defect area, fit the edge gradient direction, detect the grayscale curve of the foreign object defect, fit the grayscale curve by ellipse, calculate the geometric centroid coordinates and deflection angle of the foreign object defect, generate a feature vector, input it into the SVM classifier for surface defect classification, and output the surface defect type.

[0035] Furthermore, step S1 includes:

[0036] Step S11. Set up the camera and calibration board so that the field of view covers the stage area, uniformly sample the outline points of the LED chip, select an auxiliary coordinate system, align the sample point set by iterating the nearest point and perform homography matrix transformation, and calculate the transformation matrix from the image coordinate system to the reference coordinate system.

[0037] Step S12. Project all chip images to the standard coordinate system using the transformation matrix, calculate the standard deviation of each projected image in the standard coordinate system, select the image with the smallest standard deviation as the reference to correct the image, eliminate pose deviation, and obtain the standard image of the chip.

[0038] Furthermore, step S2 includes:

[0039] Step S21. On the standard image, the second derivative operator is used to enhance the contrast between the lines and the background at the junction of the wafer and the substrate. Based on the region with contrast higher than the threshold, morphological recognition is performed to extract the endpoints of the bonding wires. The endpoints of the bonding wires are then expanded to obtain the connected bonding wire regions.

[0040] Step S22. Divide the image into ROI blocks using the bonding wire area and the outer contour of the chip as boundaries. Within the ROI, use normalized cross-correlation to match with the standard wafer grayscale texture template. Binarize the regions with high matching confidence and perform connected component analysis to obtain a standardized wafer image.

[0041] Furthermore, step S3 includes:

[0042] Step S31. With minimizing the grayscale standard deviation of the wafer region as the optimization objective, construct a fitness function, simulate all LED brightness configuration schemes as particles, construct a particle swarm model, use the fitness function to evaluate the quality of wafer images captured under different particle configurations, output the configuration scheme with the highest quality and make adjustments.

[0043] Step S32. In a darkroom environment, illuminate the chip sample with a monochromatic point light source and record the visual parameters, including: incident angle, wavelength, intensity and wafer image grayscale, to form training data pairs, construct a neural network model, input the visual parameter vector of the point light source, output the wafer surface grayscale distribution equation, superimpose the grayscale distribution equations under all light sources to form a mapping equation set, solve the mapping equation set to obtain the wafer image surface grayscale dispersion parameters, and correct the color shift caused by uneven illumination spectrum.

[0044] Furthermore, step S4 includes:

[0045] Step S41. Extract the image feature vector from the known good chip image and compare it with the feature vector of the current chip wafer image item by item. By judging whether each feature is within the tolerance threshold, determine whether the chip is good.

[0046] Step S42. Based on the comparison results of image features, filter out areas that do not meet the standards and mark them as defective areas. Generate defective area images by annotating the filtered defective areas.

[0047] Furthermore, step S5 includes:

[0048] Step S51. Delineate the invalid area by trapezoidal region. Within the delineated effective trapezoidal region, delineate the edge of the foreign object region by Zernike moment or gray-level moment. Cluster the sub-pixel edge points based on Euclidean distance. For the clustered edge points, calculate the gray-level gradient direction. Extract a narrow band region along the gray-level gradient normal direction. Fit the gray-level value change curve in the normal direction using a fourth-order polynomial. Connect all edge points within the cluster to form the defect contour.

[0049] Step S52. Perform least-squares ellipse fitting on the extracted defect contours to construct structured data. Generate high-dimensional feature vectors for the structured data. Load the defect vector training set from the cloud to train the support vector machine classifier and output the chip defect type and confidence level.

[0050] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0051] 1. This invention uses different cameras to capture images of the chip production area, extracts the edge contours of LED chip images, calibrates image deviations, segments solder joints and solder lines, locates connected areas of the wafer, and identifies wafer mounting position deviations during the production process. This enables quantitative analysis of defects, reduces reliance on manual labor, meets the needs of industrial automation production, ensures the stability of the calibration process, reduces chip false detection rates, improves defect detection speed, and reduces production costs.

[0052] 2. This invention adjusts the brightness of each LED in the ring lamp to achieve uniform grayscale in the chip image, performs color processing on the chip image, obtains the image features of the LED chip, determines the chip production quality, and screens out defective areas. This improves the grayscale uniformity of the LED chip image, enables efficient detection of surface defects in the chip while ensuring accuracy, enhances the detection generalization ability under complex lighting conditions, improves detection efficiency and consistency, and increases LED chip production efficiency.

[0053] 3. This invention detects the grayscale curve of foreign object defects by detecting the edge of the defect area, generates feature vectors by ellipse fitting, identifies and calibrates the defect area through morphological analysis, and classifies surface defect traces. This can improve the extraction accuracy of defect areas, achieve accurate detection and classification of defect areas, improve dispersion correction accuracy, and enhance the automated detection efficiency of the production line. Attached Figure Description

[0054] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0055] Figure 1 This is a schematic diagram of the structure of an IoT-based automotive LED module production monitoring system according to the present invention.

[0056] Figure 2 This is a schematic diagram illustrating the steps of an IoT-based automotive LED module production monitoring method according to the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figures 1 to 2 The present invention provides a technical solution: an Internet of Things-based automotive LED module production monitoring system, comprising: a chip calibration module, a wafer identification module, a light source training module, a quality monitoring module, and a defect monitoring module;

[0059] The chip calibration module is used to deploy different cameras to capture the chip production area, calibrate the LED chip position and perform chain code contour tracking, extract the edge contour of the LED chip image, decompose the edge contour points into a fixed number, make the reference image and the target image point set correspond, obtain the transformation matrix between the camera's image coordinate system and the reference coordinate system, use the projection positioning algorithm to project all chip images onto the standard coordinate system, select the image with the smallest projection standard deviation, correct the calibration deviation based on geometric coordinate transformation, and straighten the chip to obtain the standard image.

[0060] The chip calibration module includes: a contour tracking unit and a coordinate projection unit;

[0061] The contour tracking unit is used to set up the camera and calibration plate so that the field of view covers the stage area, uniformly sample the contour points of the LED chip, select an auxiliary coordinate system, align the sample point set by iterating the nearest point and perform homography matrix transformation, and calculate the transformation matrix from the image coordinate system to the reference coordinate system.

[0062] The coordinate projection unit is used to project all chip images to the standard coordinate system using a transformation matrix, calculate the standard deviation of each projected image in the standard coordinate system, select the image with the smallest standard deviation as the reference to correct the image, eliminate pose deviation, and obtain the standard image of the chip.

[0063] The wafer recognition module is used to set the contrast between the solder line and the image edge line by setting the second derivative threshold, segment the solder joint by adaptive threshold, obtain the solder line by dilation algorithm, divide the region by the solder line as the boundary, match the gray-scale features of the wafer region by normalized cross-correlation algorithm, determine the wafer connected region, perform a closing operation on the connected region, locate the wafer by the minimum circumscribed moment of the connected region, and obtain the chip wafer image.

[0064] The wafer identification module includes: a wire bonding extraction unit and a connectivity segmentation unit;

[0065] The wire bonding extraction unit is used to enhance the contrast between lines and background at the junction of the wafer and the substrate on a standard image using a second derivative operator, perform morphological recognition based on regions with contrast higher than a threshold, extract wire bonding endpoints, and dilate the wire bonding endpoints to obtain connected wire bonding regions.

[0066] The connected segmentation unit is used to divide the image into ROI blocks with the bonding wire area and the outer contour of the chip as the boundary. Within the ROI, normalized cross-correlation is used to match with the standard wafer grayscale texture template. Regions with high matching confidence are binarized and connected component analysis is performed to obtain a standardized wafer image.

[0067] The light source training module is used to apply the particle swarm optimization algorithm to adjust the brightness of each LED in the ring lamp, so that the grayscale of the chip image is uniform. The visual parameters under monochromatic light illumination and the corresponding grayscale images are used as the training set. The training set is enhanced by a pseudo-anomaly simulation strategy. The training set is input into the neural network model and outputs the grayscale mapping equation under point light source radiation. A grayscale mapping equation system is established for all point light sources under three-color ring illumination conditions. The grayscale dispersion parameters of the chip image are obtained by solving the equation system. A color space conversion model is established according to the grayscale dispersion parameters and the chip image is color-corrected.

[0068] The light source training module includes: an illumination unit, a grayscale mapping unit, and a feature extraction unit;

[0069] The ensemble illumination unit is used to construct a fitness function with the goal of minimizing the grayscale standard deviation of the wafer region, simulate all LED brightness configuration schemes as particles, construct a particle swarm model, use the fitness function to evaluate the quality of wafer images captured under different particle configurations, output the configuration scheme with the highest quality and make adjustments.

[0070] The grayscale mapping unit is used to illuminate the chip sample with a monochromatic point light source in a dark room environment, record visual parameters, including incident angle, wavelength, intensity and wafer image grayscale, form training data pairs, construct a neural network model, input the visual parameter vector of the point light source, and output the grayscale distribution equation of the wafer surface.

[0071] The feature extraction unit is used to superimpose the gray-level distribution equations under all light sources to form a mapping equation set. Solving the mapping equation set yields the gray-level dispersion parameters of the wafer image surface, correcting the color shift caused by uneven illumination spectrum.

[0072] The quality monitoring module is used to acquire the image features of LED chips, including the spacing between wafers, average grayscale value, length and width, area and center distance, and compare them with the image features of standard chips. The comparison results determine the chip production quality, and the chip is judged as a good product according to the production quality. At the same time, for defective chips, defective areas are screened out and defective area images are output.

[0073] The quality monitoring module includes: an image comparison unit and a good product classification unit;

[0074] The image comparison unit is used to extract image feature vectors from known good chip images and compare them item by item with the feature vectors of the current chip wafer image. By judging whether each feature is within the tolerance threshold, it is determined whether the chip is good.

[0075] The good product classification unit is used to filter out areas that do not meet the standards based on the comparison results of image features, and mark them as defective areas. The filtered defective areas are then labeled to generate defective area images.

[0076] The defect monitoring module is used to detect sub-pixel edges of defect areas, perform Euclidean clustering on edge points, fit the gray-level gradient direction along the image edge using a fourth-order polynomial, detect the gray-level curve of foreign objects, fit the gray-level curve by an ellipse, calculate the geometric centroid coordinates and deflection angle of the foreign object defect, and generate feature vectors using the geometric centroid coordinates, deflection angle, gray level, and pixel area. All feature vectors in the defect area constitute a feature matrix, which is input into an SVM classifier of a support vector machine to classify surface defect traces and output the surface defect type.

[0077] The defect monitoring module includes: a foreign object segmentation unit, a surface defect unit, and a classification and identification unit;

[0078] The foreign object segmentation unit is used to delineate the invalid area by using a trapezoidal region. Within the delineated trapezoidal effective area, the edge of the foreign object area is delineated by Zernike moment or grayscale moment, and the sub-pixel edge points are clustered based on Euclidean distance.

[0079] The surface defect unit is used to calculate the gray-level gradient direction for the clustered edge points, extract a narrow band region along the gray-level gradient normal direction, fit the gray-level value change curve in the normal direction using a fourth-order polynomial, connect all the edge points within the cluster to form a defect contour, and perform least-squares ellipse fitting on the extracted defect contour to construct structured data.

[0080] The classification and recognition unit is used to generate high-dimensional feature vectors for structured data, load defect vector training sets from the cloud to train a support vector machine classifier, and output chip defect type and confidence level.

[0081] A method for monitoring the production of automotive LED modules based on the Internet of Things (IoT) includes the following steps:

[0082] Step S1. Photograph the chip production area, mark the LED chip position, extract the edge contour of the LED chip image, project all chip images onto the standard coordinate system, select and correct the image with the smallest standard deviation of the projection to obtain the standard chip image;

[0083] Step S1 includes:

[0084] Step S11. Set up the camera and calibration board so that the field of view covers the stage area, uniformly sample the outline points of the LED chip, select an auxiliary coordinate system, align the sample point set by iterating the nearest point and perform homography matrix transformation, and calculate the transformation matrix from the image coordinate system to the reference coordinate system.

[0085] Step S12. Project all chip images to the standard coordinate system using the transformation matrix, calculate the standard deviation of each projected image in the standard coordinate system, select the image with the smallest standard deviation as the reference to correct the image, eliminate pose deviation, and obtain the standard image of the chip.

[0086] Step S2. Identify solder joints by contrasting the solder lines and image edge lines. Solder joints expand to obtain solder lines. Divide regions using solder lines as boundaries, match grayscale features of each region, determine connected regions of the wafer, perform a closing operation on the connected regions, and locate the wafer using the minimum circumscribed moment of the connected regions to obtain the wafer image.

[0087] Step S2 includes:

[0088] Step S21. On the standard image, the second derivative operator is used to enhance the contrast between the lines and the background at the junction of the wafer and the substrate. Based on the region with contrast higher than the threshold, morphological recognition is performed to extract the endpoints of the bonding wires. The endpoints of the bonding wires are then expanded to obtain the connected bonding wire regions.

[0089] Step S22. Divide the image into ROI blocks using the bonding wire area and the outer contour of the chip as boundaries. Within the ROI, use normalized cross-correlation to match with the standard wafer grayscale texture template. Binarize the regions with high matching confidence and perform connected component analysis to obtain a standardized wafer image.

[0090] Step S3. Apply the particle swarm optimization algorithm to adjust the brightness of each LED in the ring lamp. Use the visual parameters and grayscale images under monochromatic light illumination as the training set to train the neural network model, output the grayscale mapping equation under point light source radiation, solve the grayscale mapping equation set established by all point light sources, obtain the grayscale dispersion parameters of the image, and adjust the color of the chip image.

[0091] Step S3 includes:

[0092] Step S31. With minimizing the grayscale standard deviation of the wafer region as the optimization objective, construct a fitness function, simulate all LED brightness configuration schemes as particles, construct a particle swarm model, use the fitness function to evaluate the quality of wafer images captured under different particle configurations, output the configuration scheme with the highest quality and make adjustments.

[0093] Step S32. In a darkroom environment, illuminate the chip sample with a monochromatic point light source and record the visual parameters, including: incident angle, wavelength, intensity and wafer image grayscale, to form training data pairs, construct a neural network model, input the visual parameter vector of the point light source, output the wafer surface grayscale distribution equation, superimpose the grayscale distribution equations under all light sources to form a mapping equation set, solve the mapping equation set to obtain the wafer image surface grayscale dispersion parameters, and correct the color shift caused by uneven illumination spectrum.

[0094] Step S4. Obtain chip image features, compare them with the image features of standard chips, determine chip production quality, determine whether the chip is a good product, and for defective chips, screen out the defective areas of the wafer image and output the defective area image.

[0095] Step S4 includes:

[0096] Step S41. Extract the image feature vector from the known good chip image and compare it with the feature vector of the current chip wafer image item by item. By judging whether each feature is within the tolerance threshold, determine whether the chip is good.

[0097] Step S42. Based on the comparison results of image features, filter out areas that do not meet the standards and mark them as defective areas. Generate defective area images by annotating the filtered defective areas.

[0098] Step S5. Detect sub-pixel edges of the defect area, fit the edge gradient direction, detect the grayscale curve of the foreign object defect, fit the grayscale curve by ellipse, calculate the geometric centroid coordinates and deflection angle of the foreign object defect, generate a feature vector, input it into the SVM classifier for surface defect classification, and output the surface defect type.

[0099] Step S5 includes:

[0100] Step S51. Delineate the invalid area by trapezoidal region. Within the delineated effective trapezoidal region, delineate the edge of the foreign object region by Zernike moment or gray-level moment. Cluster the sub-pixel edge points based on Euclidean distance. For the clustered edge points, calculate the gray-level gradient direction. Extract a narrow band region along the gray-level gradient normal direction. Fit the gray-level value change curve in the normal direction using a fourth-order polynomial. Connect all edge points within the cluster to form the defect contour.

[0101] Step S52. Perform least-squares ellipse fitting on the extracted defect contours to construct structured data. Generate high-dimensional feature vectors for the structured data. Load the defect vector training set from the cloud to train the support vector machine classifier and output the chip defect type and confidence level.

[0102] Example: The process involves photographing the chip production area, calibrating the LED chip positions, extracting the image edge contours, performing constant-number decomposition on the edge contour points, establishing a transformation matrix from the camera image coordinate system to the reference coordinate system, projecting into the standard coordinate system, correcting calibration deviations, segmenting solder joints, dividing the region by the solder lines, locating the wafer, adjusting the brightness of each LED in the ring lamp, performing color space conversion, acquiring LED chip image features, detecting defect areas, calculating the geometric centroid coordinates and deflection angle of foreign matter defects, and outputting the surface defect type.

[0103] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0104] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring the production of automotive LED modules based on the Internet of Things, characterized in that, The method includes the following steps: Step S1. Photograph the chip production area, mark the LED chip position, extract the edge contour of the LED chip image, project all chip images onto the standard coordinate system, select and correct the image with the smallest standard deviation of the projection to obtain the standard chip image; Step S2. Identify solder joints by contrasting the solder lines and image edge lines. Solder joints expand to obtain solder lines. Divide regions using solder lines as boundaries, match grayscale features of each region, determine connected regions of the wafer, perform a closing operation on the connected regions, and locate the wafer using the minimum circumscribed moment of the connected regions to obtain the wafer image. Step S3. Apply the particle swarm optimization algorithm to adjust the brightness of each LED in the ring lamp. Use the visual parameters and grayscale images under monochromatic light illumination as the training set to train the neural network model, output the grayscale mapping equation under point light source radiation, solve the grayscale mapping equation set established by all point light sources, obtain the grayscale dispersion parameters of the image, and adjust the color of the chip image. Step S4. Obtain chip image features, compare them with the image features of standard chips, determine chip production quality, determine whether the chip is a good product, and for defective chips, screen out the defective areas of the wafer image and output the defective area image. Step S5. Detect sub-pixel edges of the defect area, fit the edge gradient direction, detect the grayscale curve of the foreign object defect, fit the grayscale curve by ellipse, calculate the geometric centroid coordinates and deflection angle of the foreign object defect, generate a feature vector, input it into the SVM classifier for surface defect classification, and output the surface defect type.

2. The method for monitoring the production of automotive LED modules based on the Internet of Things according to claim 1, characterized in that: Step S1 includes: Step S11. Set up the camera and calibration board so that the field of view covers the stage area, uniformly sample the outline points of the LED chip, select an auxiliary coordinate system, align the sample point set by iterating the nearest point and perform homography matrix transformation, and calculate the transformation matrix from the image coordinate system to the reference coordinate system. Step S12. Project all chip images to the standard coordinate system using the transformation matrix, calculate the standard deviation of each projected image in the standard coordinate system, select the image with the smallest standard deviation as the reference to correct the image, eliminate pose deviation, and obtain the standard image of the chip.

3. The method for monitoring the production of automotive LED modules based on the Internet of Things according to claim 2, characterized in that: Step S2 includes: Step S21. On the standard image, the second derivative operator is used to enhance the contrast between the lines and the background at the junction of the wafer and the substrate. Based on the region with contrast higher than the threshold, morphological recognition is performed to extract the endpoints of the bonding wires. The endpoints of the bonding wires are then expanded to obtain the connected bonding wire regions. Step S22. Divide the image into ROI blocks using the bonding wire area and the outer contour of the chip as boundaries. Within the ROI, use normalized cross-correlation to match with the standard wafer grayscale texture template. Binarize the regions with high matching confidence and perform connected component analysis to obtain a standardized wafer image.

4. The method for monitoring the production of automotive LED modules based on the Internet of Things according to claim 3, characterized in that: Step S3 includes: Step S31. With minimizing the grayscale standard deviation of the wafer region as the optimization objective, construct a fitness function, simulate all LED brightness configuration schemes as particles, construct a particle swarm model, use the fitness function to evaluate the quality of wafer images captured under different particle configurations, output the configuration scheme with the highest quality and make adjustments. Step S32. In a darkroom environment, illuminate the chip sample with a monochromatic point light source and record the visual parameters, including: incident angle, wavelength, intensity and wafer image grayscale, to form training data pairs, construct a neural network model, input the visual parameter vector of the point light source, output the wafer surface grayscale distribution equation, superimpose the grayscale distribution equations under all light sources to form a mapping equation set, solve the mapping equation set to obtain the wafer image surface grayscale dispersion parameters, and correct the color shift caused by uneven illumination spectrum. Step S4 includes: Step S41. Extract the image feature vector from the known good chip image and compare it with the feature vector of the current chip wafer image item by item. By judging whether each feature is within the tolerance threshold, determine whether the chip is good. Step S42. Based on the comparison results of image features, filter out areas that do not meet the standards and mark them as defective areas. Generate defective area images by annotating the filtered defective areas.

5. The method for monitoring the production of automotive LED modules based on the Internet of Things according to claim 4, characterized in that: Step S5 includes: Step S51. Delineate the invalid area by trapezoidal region. Within the delineated effective trapezoidal region, delineate the edge of the foreign object region by Zernike moment or gray-level moment. Cluster the sub-pixel edge points based on Euclidean distance. For the clustered edge points, calculate the gray-level gradient direction. Extract a narrow band region along the gray-level gradient normal direction. Fit the gray-level value change curve in the normal direction using a fourth-order polynomial. Connect all edge points within the cluster to form the defect contour. Step S52. Perform least-squares ellipse fitting on the extracted defect contours to construct structured data. Generate high-dimensional feature vectors for the structured data. Load the defect vector training set from the cloud to train the support vector machine classifier and output the chip defect type and confidence level.

6. An IoT-based automotive LED module production monitoring system, characterized in that, The system includes the following modules: chip calibration module, wafer identification module, light source training module, quality monitoring module, and defect monitoring module; The chip calibration module is used to deploy different cameras to capture the chip production area, calibrate the LED chip position and perform chain code contour tracking, extract the edge contour of the LED chip image, decompose the edge contour points into a fixed number, make the reference image and the target image point set correspond, obtain the transformation matrix between the camera's image coordinate system and the reference coordinate system, use the projection positioning algorithm to project all chip images onto the standard coordinate system, select the image with the smallest projection standard deviation, correct the calibration deviation based on geometric coordinate transformation, and straighten the chip to obtain the standard image. The wafer recognition module is used to set the contrast between the solder line and the image edge line by setting the second derivative threshold, segment the solder joint by adaptive threshold, obtain the solder line by dilation algorithm, divide the region by the solder line as the boundary, match the gray-scale features of the wafer region by normalized cross-correlation algorithm, determine the wafer connected region, perform a closing operation on the connected region, locate the wafer by the minimum circumscribed moment of the connected region, and obtain the chip wafer image. The light source training module is used to apply the particle swarm optimization algorithm to adjust the brightness of each LED in the ring lamp, so that the grayscale of the chip image is uniform. The visual parameters under monochromatic light illumination and the corresponding grayscale images are used as the training set. The training set is enhanced by a pseudo-anomaly simulation strategy. The training set is input into the neural network model and outputs the grayscale mapping equation under point light source radiation. A grayscale mapping equation system is established for all point light sources under three-color ring illumination conditions. The grayscale dispersion parameters of the chip image are obtained by solving the equation system. A color space conversion model is established according to the grayscale dispersion parameters and the chip image is color-corrected. The quality monitoring module is used to acquire the image features of LED chips, including the spacing between wafers, average grayscale value, length and width, area and center distance, and compare them with the image features of standard chips. The comparison results determine the chip production quality, and the chip is judged as a good product according to the production quality. At the same time, for defective chips, defective areas are screened out and defective area images are output. The defect monitoring module is used to detect sub-pixel edges of defect areas, perform Euclidean clustering on edge points, fit the gray-level gradient direction along the image edge using a fourth-order polynomial, detect the gray-level curve of foreign objects, fit the gray-level curve by an ellipse, calculate the geometric centroid coordinates and deflection angle of the foreign object defect, and generate feature vectors using the geometric centroid coordinates, deflection angle, gray level, and pixel area. All feature vectors in the defect area constitute a feature matrix, which is input into an SVM classifier of a support vector machine to classify surface defect traces and output the surface defect type.

7. The IoT-based automotive LED module production monitoring system according to claim 6, characterized in that: The chip calibration module includes: a contour tracking unit and a coordinate projection unit; The contour tracking unit is used to set up the camera and calibration plate so that the field of view covers the stage area, uniformly sample the contour points of the LED chip, select an auxiliary coordinate system, align the sample point set by iterating the nearest point and perform homography matrix transformation, and calculate the transformation matrix from the image coordinate system to the reference coordinate system. The coordinate projection unit is used to project all chip images to the standard coordinate system using the transformation matrix, calculate the projection standard deviation of each projected image in the standard coordinate system, select the image with the smallest standard deviation as the reference to correct the image, eliminate the pose deviation, and obtain the standard image of the chip. The wafer identification module includes: a wire bonding extraction unit and a connectivity segmentation unit; The wire bonding extraction unit is used to enhance the contrast between lines and background at the junction of the wafer and the substrate on a standard image using a second derivative operator, perform morphological recognition based on regions with contrast higher than a threshold, extract wire bonding endpoints, and dilate the wire bonding endpoints to obtain connected wire bonding regions. The connected segmentation unit is used to divide the image into ROI blocks with the bonding wire area and the outer contour of the chip as the boundary. Within the ROI, normalized cross-correlation is used to match with the standard wafer grayscale texture template. Regions with high matching confidence are binarized and connected component analysis is performed to obtain a standardized wafer image.

8. The IoT-based automotive LED module production monitoring system according to claim 7, characterized in that: The light source training module includes: an illumination unit, a grayscale mapping unit, and a feature extraction unit; The ensemble illumination unit is used to construct a fitness function with the goal of minimizing the grayscale standard deviation of the wafer region, simulate all LED brightness configuration schemes as particles, construct a particle swarm model, use the fitness function to evaluate the quality of wafer images captured under different particle configurations, output the configuration scheme with the highest quality and make adjustments. The grayscale mapping unit is used to illuminate the chip sample with a monochromatic point light source in a dark room environment, record visual parameters, including incident angle, wavelength, intensity and wafer image grayscale, form training data pairs, construct a neural network model, input the visual parameter vector of the point light source, and output the grayscale distribution equation of the wafer surface. The feature extraction unit is used to superimpose the gray-level distribution equations under all light sources to form a mapping equation set. Solving the mapping equation set yields the gray-level dispersion parameters of the wafer image surface, correcting the color shift caused by uneven illumination spectrum.

9. The IoT-based automotive LED module production monitoring system according to claim 8, characterized in that: The quality monitoring module includes: an image comparison unit and a good product classification unit; The image comparison unit is used to extract image feature vectors from known good chip images and compare them item by item with the feature vectors of the current chip wafer image. By judging whether each feature is within the tolerance threshold, it is determined whether the chip is good. The good product classification unit is used to filter out areas that do not meet the standards based on the comparison results of image features, and mark them as defective areas. The filtered defective areas are then labeled to generate defective area images.

10. The IoT-based automotive LED module production monitoring system according to claim 9, characterized in that: The defect monitoring module includes: a foreign object segmentation unit, a surface defect unit, and a classification and identification unit; The foreign object segmentation unit is used to delineate the invalid area by using a trapezoidal region. Within the delineated trapezoidal effective area, the edge of the foreign object area is delineated by Zernike moment or grayscale moment, and the sub-pixel edge points are clustered based on Euclidean distance. The surface defect unit is used to calculate the gray-level gradient direction for the clustered edge points, extract a narrow band region along the gray-level gradient normal direction, fit the gray-level value change curve in the normal direction using a fourth-order polynomial, connect all the edge points within the cluster to form a defect contour, and perform least-squares ellipse fitting on the extracted defect contour to construct structured data. The classification and recognition unit is used to generate high-dimensional feature vectors for structured data, load defect vector training sets from the cloud to train a support vector machine classifier, and output chip defect type and confidence level.

Images