Mini LED welding spot quality analysis method and system

Abstract

The invention discloses a Mini LED welding spot quality analysis method and system, and the method comprises the steps: obtaining a Mini LED welding spot infrared image and a visible light image, and carrying out the feature fusion, and obtaining a fused Mini LED welding spot image; acquiring a preset number of fused Mini LED welding spot defect images and fused Mini LED welding spot normal images, and establishing a training data set according to the fused Mini LED welding spot defect images and the fused Mini LED welding spot normal images; and inputting the training data set into a preset neural network for deep learning training to obtain a welding spot defect detection model, obtaining a to-be-detected fused Mini LED welding spot image, and inputting the to-be-detected fused Mini LED welding spot image into the welding spot defect detection model to detect whether the Mini LED welding spot has defects. The problem that welding spot defect detection is not accurate enough in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and particularly relates to a method and system for analyzing the quality of Mini LED solder joints. Background Art

[0002] The quality inspection of Mini LED solder joints is a key link to ensure the performance, reliability and lifespan of products. To improve the inspection efficiency and accuracy, machine learning and image recognition technologies have been widely applied to solder joint defect detection.

[0003] In the prior art, most solder joint defect detections are realized based on infrared images or visible light images. However, both of these two methods have their own disadvantages. Infrared images can accurately capture the internal details of solder joints, but are poor at identifying surface defects of solder joints. Visible light images can intuitively display surface defects, but cannot capture the internal details of solder joints. Therefore, there is a certain problem of low accuracy in the current solder joint defect detection methods. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method and system for analyzing the quality of Mini LED solder joints, aiming to solve the problem of insufficient accuracy in solder joint defect detection in the prior art.

[0005] The embodiments of the present invention are implemented as follows: On the one hand, a method for analyzing the quality of Mini LED solder joints is proposed, and the method includes: Respectively obtain the infrared image and visible light image of the Mini LED solder joint collected by the infrared image acquisition device and the visible light image acquisition device, and perform feature fusion on the infrared image and visible light image of the Mini LED solder joint to obtain a fused Mini LED solder joint image; Collect a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establish a training data set according to the fused Mini LED solder joint defect images and fused Mini LED solder joint normal images; Input the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and obtain the fused Mini LED solder joint image to be detected, and input the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there are defects in the Mini LED solder joint.

[0006] Further, in the above method for analyzing the quality of Mini LED solder joints, the step of performing feature fusion on the infrared image and visible light image of the Mini LED solder joint to obtain a fused Mini LED solder joint image includes: Grayscale the visible light image of the Mini LED solder joint to obtain a grayscale image of the Mini LED solder joint, and use a Gaussian filter or a median filter to remove the noise in the grayscale image of the Mini LED solder joint; Use an edge detection algorithm to perform edge detection on the grayscale image of the Mini LED solder joint to obtain the contour features of the solder joint, and use a preset extraction algorithm to extract the texture features of the solder joint; Analyze the infrared image of the Mini LED solder joint to obtain the thermal distribution area map and the corresponding thermal distribution characteristics of the solder joint, and fuse the contour features, texture features, and thermal distribution characteristics of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini LED solder joint image.

[0007] Further, in the above Mini LED solder joint quality analysis method, the step of fusing the contour features, texture features, and thermal distribution characteristics of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini LED solder joint image further includes: Create a blank canvas with the same size as the original image; Based on the position information, draw the extracted contour features on the blank canvas, generate a texture map according to the extracted texture features, then overlay the texture map as a rendering layer on the blank canvas, and finally overlay the thermal distribution features on the blank canvas to obtain a fused Mini LED solder joint image; Among them, the texture features are highlighted using different grayscale values, and the thermal distribution features are displayed using pseudo-color mapping to show the thermal distribution.

[0008] Further, in the above Mini LED solder joint quality analysis method, after the step of overlaying the thermal distribution features on the blank canvas to obtain a fused Mini LED solder joint image, it further includes: Perform data augmentation on the fused Mini LED solder joint defect image according to a preset rule, and the step of performing data augmentation on the fused Mini LED solder joint defect image according to a preset rule includes: Increase the brightness or color of the contour feature layer in the fused Mini LED solder joint defect image; and / or Apply random noise or variation to the texture feature layer; and / or Perform weighted overlay on the thermal distribution features to make them present different thermal distribution situations.

[0009] Further, in the above Mini LED solder joint quality analysis method, the step of performing data augmentation on the fused MiniLED solder joint image according to a preset rule further includes: Apply different degrees of compression and detail loss to each feature layer and adjust the contrast, hue, or saturation of each feature layer; and / or Add random perturbations in the normal direction of the texture feature layer; and / or Introduce local occlusion in the fused Mini LED solder joint defect image.

[0010] Furthermore, in the above Mini LED solder joint quality analysis method, the step of using the texture map as a rendering layer and overlaying it on a blank canvas, and finally overlaying the heat distribution feature on the blank canvas to obtain the fused Mini LED solder joint image includes: Use Fourier transform to convert the images of the texture map and the heat distribution feature from the spatial domain to the frequency domain respectively; Perform different weighted superpositions on the low-frequency components and high-frequency components of the images of the texture map and the heat distribution feature respectively to retain the low-frequency information of the texture and enhance the high-frequency part of the image of the heat distribution feature; Convert the superimposed frequency-domain image back to the spatial domain through inverse Fourier transform to obtain the fused Mini LED solder joint image.

[0011] Furthermore, in the above Mini LED solder joint quality analysis method, the step of using the texture map as a rendering layer and overlaying it on a blank canvas, and finally overlaying the heat distribution feature on the blank canvas to obtain the fused Mini LED solder joint image further includes: Control the fusion degree of the texture map and the heat distribution feature by using different transparency values to ensure that the texture map and the heat distribution feature do not cover each other when overlaid.

[0012] Another object of the present invention is to provide a Mini LED solder joint quality analysis system, and the system includes: An acquisition module, configured to respectively acquire the Mini LED solder joint infrared image and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device, and perform feature fusion on the Mini LED solder joint infrared image and the visible light image to obtain a fused Mini LED solder joint image; A fusion module, configured to collect a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establish a training data set according to the fused Mini LED solder joint defect images and the fused Mini LED solder joint normal images; A detection module, configured to input the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and acquire a fused Mini LED solder joint image to be detected, and input the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there is a defect in the Mini LED solder joint.

[0013] In another aspect, the present invention provides a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0014] In another aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0015] In the embodiment of the present invention, by respectively acquiring the Mini LED solder joint infrared image and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device, and performing feature fusion on the Mini LED solder joint infrared image and the visible light image to obtain a fused Mini LED solder joint image; collecting a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establishing a training data set according to the fused Mini LED solder joint defect images and the fused Mini LED solder joint normal images; inputting the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and acquiring a fused Mini LED solder joint image to be detected, and inputting the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there is a defect in the Mini LED solder joint. Through the information complementarity of the two images, after fusing the visible light and infrared images for image recognition, the advantages of the two can be effectively combined, the deficiencies of each can be supplemented, and the accuracy of defect detection can be improved. It solves the problem of inaccurate solder joint defect detection in the prior art. Description of the Drawings

[0016] Figure 1 It is a flowchart of the Mini LED solder joint quality analysis method proposed in the first embodiment of the present invention; Figure 2 It is a schematic structural diagram of the Mini LED solder joint quality analysis system in the third embodiment of the present invention.

[0017] The following specific embodiments will further illustrate the present invention in conjunction with the above drawings. Specific Embodiments

[0018] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0019] It should be noted that when an element is referred to as being "fixed on" another element, it can be directly on the other element or there can also be a middle element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be a middle element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are only for the purpose of illustration.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0021] The following will specifically describe in detail how to improve the accuracy of Mini LED solder joint defect detection in combination with specific embodiments and drawings.

[0022] Embodiment 1 Please refer to Figure 1 , which shows a method for analyzing the quality of Mini LED solder joints proposed in the first embodiment of the present invention. The method includes steps S10 to S12.

[0023] Step S10, respectively obtain the infrared image and visible light image of the Mini LED solder joint collected by the infrared image acquisition device and the visible light image acquisition device, and perform feature fusion on the infrared image and visible light image of the Mini LED solder joint to obtain a fused Mini LED solder joint image.

[0024] Among them, an infrared image acquisition device is a device that can capture the infrared radiation emitted by an object and convert it into a visible image. Exemplarily, for example, an infrared imager. Infrared images are usually used to detect the heat distribution of an object because all objects emit infrared radiation according to their temperature. In the detection of Mini LED solder joints, infrared images can help identify the temperature distribution of the solder joints, which is very useful for evaluating the welding quality and discovering potential overheated or undercooled areas; while a visible light image acquisition device is another image acquisition device, such as a camera, which can capture the reflected light of an object within the visible spectrum range and generate what is commonly called a photo. It can provide visual details of the solder joints, such as size, shape, and surface conditions. Through these visible light images, visible defects that may occur during the welding process, such as cracks and bubbles, can be clearly identified.

[0025] Specifically, these two types of images provide information on different aspects of the solder joints. Fusing Mini LED solder joint images refers to the image obtained through a feature fusion process, which combines the information of infrared images and visible light images, providing a more comprehensive and information-rich view, and can help capture the temperature distribution and appearance details of the solder joints simultaneously, so as to more accurately evaluate the welding quality.

[0026] Step S11, collect a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establish a training data set based on the fused Mini LED solder joint defect images and fused Mini LED solder joint normal images.

[0027] Among them, collecting normal samples and abnormal samples to suggest a training data set for subsequent model training. Specifically, collect a certain number of fused Mini LED solder joint defect images, where the solder joints in these images have known defects, and a certain number of fused Mini LED solder joint normal images, which show well-welded solder joints. In specific implementation, the number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images can be set according to the actual situation and is not limited here.

[0028] Specifically, after collecting a sufficient number of fused images of defective and normal solder joints, the next step is to organize these images into a training data set. In machine learning and deep learning, a training data set is used to train a model, and the model learns from these data to identify specific patterns or features. In this scenario, the training data set will contain solder joint images labeled as "defective" and "normal", and the goal of the model is to learn to distinguish between these two types of solder joints.

[0029] Step S12: Input the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and acquire a fused Mini LED solder joint image to be detected. Input the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there are defects in the Mini LED solder joints.

[0030] Among them, the preset neural network refers to a pre-designed neural network architecture, which can be a convolutional neural network (CNN), a recurrent neural network (RNN), or other types of deep learning models, specifically depending on the complexity of the task and the characteristics of the data. Deep learning training means using the training data set to train the neural network so that it can learn and identify the defect features in the solder joint image. During the training process, the neural network continuously adjusts its internal parameters (weights and biases) to minimize the prediction error. After deep learning training, the neural network converges to a stable state, and at this time it can be used as a solder joint defect detection model. This model can receive a new solder joint image as input and output a prediction result indicating whether there are defects in the solder joint. Specifically, in practical applications, the solder joint images to be detected are used to test the solder joint defect detection model. These images are fused Mini LED solder joint images obtained by feature fusion of the infrared images and visible light images of the solder joints collected actually to ensure that they contain sufficient information for the model to analyze. Finally, the solder joint defect detection model outputs a prediction result indicating whether there are defects in the input solder joint image. This prediction result can be used to guide subsequent quality control.

[0031] In summary, in the Mini LED solder joint quality analysis method in the above embodiments of the present invention, by respectively acquiring the Mini LED solder joint infrared image and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device, and performing feature fusion on the Mini LED solder joint infrared image and the visible light image to obtain a fused Mini LED solder joint image; collecting a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establishing a training data set according to the fused Mini LED solder joint defect images and the fused Mini LED solder joint normal images; inputting the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and acquiring a fused Mini LED solder joint image to be detected, and inputting the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there are defects in the Mini LED solder joints. Through the information complementarity of the two images, after fusing the visible light and infrared images for image recognition, the advantages of the two can be effectively combined, the deficiencies of each can be supplemented, and the accuracy of defect detection can be improved. It solves the problem of inaccurate solder joint defect detection in the prior art.

[0032] Example 2 This embodiment also proposes a method for analyzing the quality of Mini LED solder joints. The difference between the method for analyzing the quality of Mini LED solder joints proposed in this embodiment and the method for analyzing the quality of Mini LED solder joints proposed in Embodiment 1 is as follows: The step of performing feature fusion on the infrared image and visible light image of the Mini LED solder joint to obtain a fused Mini LED solder joint image includes: Perform grayscale processing on the visible light image of the Mini LED solder joint to obtain a grayscale image of the Mini LED solder joint, and use a Gaussian filter or a median filter to remove noise in the grayscale image of the Mini LED solder joint; Use an edge detection algorithm to perform edge detection on the grayscale image of the Mini LED solder joint to obtain the contour features of the solder joint, and use a preset extraction algorithm to extract the texture features of the solder joint; Analyze the infrared image of the Mini LED solder joint to obtain a thermal distribution area map of the solder joint and the corresponding thermal distribution characteristics, and fuse the contour features, texture features, and thermal distribution characteristics of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini LED solder joint image.

[0033] Among them, performing grayscale processing on the visible light image of the Mini LED solder joint simplifies the image processing process, and uses a Gaussian filter or a median filter to remove noise in the grayscale image. Use an edge detection algorithm to perform edge detection on the grayscale image, such as the Canny algorithm, to obtain the contour features of the solder joint. The edge detection algorithm can identify areas with drastic changes in brightness in the image, that is, the edges of objects. The contour features of the solder joint are an important basis for identifying the position and shape of the solder joint. At the same time, using a preset extraction algorithm to extract the texture features of the solder joint is very useful for identifying features such as the surface condition and material type of the solder joint. Specifically, the local binary pattern (LBP) can be used to extract texture features; Furthermore, analyze the infrared image of the Mini LED solder joint to obtain a thermal distribution area map of the solder joint and the corresponding thermal distribution characteristics. The infrared image can reflect the temperature distribution of the solder joint. By analyzing the infrared image, the thermal distribution area of the solder joint and the temperature difference in different areas can be determined. According to the position information of the thermal distribution area, the contour features, texture features, and thermal distribution characteristics of the solder joint are fused. This step is to integrate information from different image sources to generate a comprehensive image containing more useful information.

[0034] Exemplarily, create a blank canvas with the same size as the original image; Based on the location information, the extracted contour features are drawn on a blank canvas. A texture map is generated according to the extracted texture features, and then the texture map is superimposed on the blank canvas as a rendering layer. Finally, the heat distribution feature is superimposed on the blank canvas to obtain a fused Mini LED solder joint image; Among them, different gray values are used to highlight the texture features, and the heat distribution features are displayed using pseudo-color mapping to show the heat distribution.

[0035] Specifically, first, a blank canvas with the same size as the original image (i.e., the infrared image or visible light image of the Mini LED solder joint) is created. This canvas will serve as the base for the fused image and be used for subsequent feature drawing and superposition. Based on the location information, the extracted contour features are drawn on the blank canvas. Next, a texture map is generated according to the extracted texture features. The texture map is usually a grayscale image, where different gray values represent different texture features. Then, this texture map is superimposed on the blank canvas as a rendering layer. By superimposing the texture map, information about the surface condition of the solder joint can be added to the fused image. Finally, the heat distribution feature is superimposed on the blank canvas. The pseudo-color mapping technique can be used to map the temperature values to specific colors. In this way, different temperature regions of the solder joint can be identified by color differences in the fused image. After superimposing the contour features, texture features, and heat distribution features on the blank canvas, the fused Mini LED solder joint image is obtained. This fused image combines information from different image sources and provides a comprehensive view of the solder joint.

[0036] In addition, to avoid the heat distribution feature covering the texture feature during superposition, methods such as transparency and weight adjustment can be used to finely control the contribution of each layer. This can not only retain the texture features but also fully display the details of the heat distribution feature map, avoiding mutual occlusion or conflict between the two.

[0037] Specifically, the fusion degree of the texture map and the heat distribution feature is controlled by using different transparency values to ensure that the texture map and the heat distribution feature do not cover each other during superposition. Or the texture map and the heat distribution feature image are transformed from the spatial domain (i.e., the two-dimensional plane of image pixels) to the frequency domain using the Fourier transform. In the frequency domain, the image is represented as a set of different frequency components. Next, different weighted superpositions are performed on the low-frequency components and high-frequency components of the texture map and the heat distribution feature image respectively. The purpose of this step is to retain the low-frequency information of the texture during the fusion process while enhancing the high-frequency part of the heat distribution feature image. Finally, the superimposed frequency domain image is transformed back to the spatial domain through the inverse Fourier transform to obtain the fused Mini LED solder joint image.

[0038] In addition, in some alternative embodiments of the present invention, after the step of superimposing the thermal distribution characteristics on the blank canvas to obtain the fused Mini LED solder joint image, the following steps are further included: Data augmentation is performed on the fused Mini LED solder joint defect image according to a preset rule. The step of performing data augmentation on the fused Mini LED solder joint defect image according to the preset rule includes: Increasing the brightness or color of the contour feature layer in the fused Mini LED solder joint defect image; and / or Applying random noise or variation to the texture feature layer; and / or Performing weighted superposition on the thermal distribution characteristics to present different thermal distribution situations.

[0039] Among them, since the number of Mini LED solder joint defect image samples is small, in order to increase the richness of the dataset, data augmentation can be performed on the fused Mini LED solder joint image. For example, the brightness of the contour feature layer can be increased or its color can be changed. This can highlight the contour of the solder joint. Random noise or a certain form of variation is applied to the texture feature layer. This can increase the diversity of the texture, making the texture features in the fused image more abundant and complex. Finally, weighted superposition can be performed on the thermal distribution characteristics to change their manifestation in the fused image. By adjusting the weights, different thermal intensities or distribution patterns can be presented for the thermal distribution. This processing helps to simulate various thermal distribution situations that may occur during the actual welding process.

[0040] Furthermore, the step of performing data augmentation on the fused Mini LED solder joint image according to the preset rule further includes: Applying different degrees of compression and detail loss to each feature layer and adjusting the contrast, hue, or saturation of each feature layer; and / or Adding random perturbations in the normal direction of the texture feature layer; and / or Introducing local occlusion in the fused Mini LED solder joint defect image.

[0041] Among them, each feature layer in the fused image (such as the texture layer, the heat distribution layer, etc.) is processed separately. By applying different degrees of compression, the quality loss that may occur during image transmission or storage can be simulated. At the same time, the processing of detail loss can simulate the situation of image resolution reduction or blurring. In addition, adjusting the contrast, hue or saturation can change the overall visual effect of the image, making it more diverse. Special processing is carried out on the texture feature layer. By adding random perturbations in the normal direction, the slight undulations or irregularities of the texture surface can be simulated. This processing helps to increase the complexity and diversity of the texture, making the texture features in the fused image more difficult to predict and identify. And introducing local occlusion on the entire fused image can simulate the occlusion or occluder situation that may be encountered in practical applications. This processing helps the training model to learn to accurately identify and locate solder joint defects even when part of the information is missing.

[0042] In summary, the Mini LED solder joint quality analysis method proposed in the above embodiments of the present invention respectively obtains the Mini LED solder joint infrared image and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device, and performs feature fusion on the Mini LED solder joint infrared image and the visible light image to obtain a fused Mini LED solder joint image; collects a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establishes a training data set according to the fused Mini LED solder joint defect images and fused Mini LED solder joint normal images; inputs the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and obtains the fused Mini LED solder joint image to be detected, and inputs the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there are defects in the Mini LED solder joint. Through the information complementarity of the two images, the visible light and infrared images are fused and then image recognition is performed, which can effectively combine the advantages of both, make up for their respective deficiencies, and improve the accuracy of defect detection. It solves the problem of inaccurate solder joint defect detection in the prior art.

[0043] Embodiment 3 Please refer to Figure 2 , which shows the Mini LED solder joint quality analysis system proposed in the third embodiment of the present invention. The system includes: An acquisition module 100, configured to respectively obtain the Mini LED solder joint infrared image and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device, and perform feature fusion on the Mini LED solder joint infrared image and the visible light image to obtain a fused Mini LED solder joint image; A fusion module 200, configured to collect a preset number of defective images of fused Mini-LED solder joints and normal images of fused Mini-LED solder joints, and establish a training dataset based on the defective images of fused Mini-LED solder joints and the normal images of fused Mini-LED solder joints; A detection module 300, configured to input the training dataset into a preset neural network for deep learning training to obtain a solder joint defect detection model, and acquire an image of a fused Mini-LED solder joint to be detected, and input the image of the fused Mini-LED solder joint to be detected into the solder joint defect detection model to detect whether there are defects in the Mini-LED solder joint.

[0044] Furthermore, in the above Mini-LED solder joint quality analysis system, the step of performing feature fusion on the infrared image and the visible light image of the Mini-LED solder joint to obtain a fused Mini-LED solder joint image includes: Performing grayscale processing on the visible light image of the Mini-LED solder joint to obtain a grayscale image of the Mini-LED solder joint, and using a Gaussian filter or a median filter to remove noise in the grayscale image of the Mini-LED solder joint; Performing edge detection on the grayscale image of the Mini-LED solder joint by using an edge detection algorithm to obtain the contour features of the solder joint, and extracting the texture features of the solder joint by using a preset extraction algorithm; Analyzing the infrared image of the Mini-LED solder joint to obtain a thermal distribution area map of the solder joint and the corresponding thermal distribution features, and fusing the contour features, texture features, and thermal distribution features of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini-LED solder joint image.

[0045] Furthermore, in the above Mini-LED solder joint quality analysis system, the step of fusing the contour features, texture features, and thermal distribution features of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini-LED solder joint image further includes: Creating a blank canvas with the same size as the original image; Based on the position information, drawing the extracted contour features onto the blank canvas, generating a texture map according to the extracted texture features, then overlaying the texture map as a rendering layer onto the blank canvas, and finally overlaying the thermal distribution features onto the blank canvas to obtain a fused Mini-LED solder joint image; Among them, the texture features are highlighted using different grayscale values, and the thermal distribution features are displayed using pseudo-color mapping to show the thermal distribution.

[0046] Further, in the above Mini LED solder joint quality analysis system, after the step of superimposing the thermal distribution feature onto the blank canvas to obtain the fused Mini LED solder joint image, the following steps are also included: Data augmentation is performed on the fused Mini LED solder joint defect image according to a preset rule. The step of performing data augmentation on the fused Mini LED solder joint defect image according to the preset rule includes: Increasing the brightness or color of the contour feature layer in the fused Mini LED solder joint defect image; and / or Applying random noise or variations to the texture feature layer; and / or Performing weighted superposition on the thermal distribution feature to present different thermal distribution situations.

[0047] Further, in the above Mini LED solder joint quality analysis system, the step of performing data augmentation on the fused Mini LED solder joint image according to the preset rule further includes: Applying different degrees of compression and detail loss to each feature layer and adjusting the contrast, hue, or saturation of each feature layer; and / or Adding random perturbations in the normal direction of the texture feature layer; and / or Introducing local occlusions in the fused Mini LED solder joint defect image.

[0048] Further, in the above Mini LED solder joint quality analysis system, the step of using the texture map as a rendering layer to be superimposed onto the blank canvas and finally superimposing the thermal distribution feature onto the blank canvas to obtain the fused Mini LED solder joint image includes: Using Fourier transform to convert the images of the texture map and the thermal distribution feature from the spatial domain to the frequency domain respectively; Performing different weighted superpositions on the low-frequency components and high-frequency components of the images of the texture map and the thermal distribution feature respectively to retain the low-frequency information of the texture while enhancing the high-frequency part of the image of the thermal distribution feature; Converting the superimposed frequency-domain image back to the spatial domain through inverse Fourier transform to obtain the fused Mini LED solder joint image.

[0049] Further, in the above Mini LED solder joint quality analysis system, the step of using the texture map as a rendering layer to be superimposed onto the blank canvas and finally superimposing the thermal distribution feature onto the blank canvas to obtain the fused Mini LED solder joint image further includes: Controlling the fusion degree of the texture map and the thermal distribution feature by using different transparency values to ensure that the texture map and the thermal distribution feature do not cover each other when superimposed.

[0050] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments, and will not be elaborated here.

[0051] Embodiment 4 On the other hand, the present invention also provides a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method according to any one of the above Embodiments 1 to 2 are implemented.

[0052] Embodiment 5 On the other hand, the present invention also provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method according to any one of the above Embodiments 1 to 2 are implemented.

[0053] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0054] Those skilled in the art can understand that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device.

[0055] More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection part with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable storage medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or processing it in other suitable ways if necessary, and then storing it in a computer memory.

[0056] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0057] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0058] The above-described embodiments merely represent several implementation manners of the present invention, and the description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. A method for analyzing the quality of Mini LED solder joints, characterized in that, The method includes: Obtain the infrared image of the Mini LED solder joint and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device respectively, and perform feature fusion on the infrared image and the visible light image of the Mini LED solder joint to obtain a fused Mini LED solder joint image; Collect a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establish a training data set according to the fused Mini LED solder joint defect images and the fused Mini LED solder joint normal images; Input the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, obtain the fused Mini LED solder joint image to be detected, and input the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there are defects in the Mini LED solder joint.

2. The Mini LED solder joint quality analysis method according to claim 1, wherein, The step of performing feature fusion on the infrared image and the visible light image of the Mini LED solder joint to obtain a fused Mini LED solder joint image includes: Perform grayscale processing on the visible light image of the Mini LED solder joint to obtain a grayscale image of the Mini LED solder joint, and use a Gaussian filter or a median filter to remove the noise in the grayscale image of the Mini LED solder joint; Use an edge detection algorithm to perform edge detection on the grayscale image of the Mini LED solder joint to obtain the contour features of the solder joint, and use a preset extraction algorithm to extract the texture features of the solder joint; Analyze the infrared image of the Mini LED solder joint to obtain a thermal distribution area map of the solder joint and the corresponding thermal distribution features, and fuse the contour features, texture features, and thermal distribution features of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini LED solder joint image.

3. The Mini LED solder joint quality analysis method according to claim 2, wherein The step of fusing the contour features, texture features, and thermal distribution features of the solder joint according to the position information of the thermal distribution area to obtain a fused Mini LED solder joint image further includes: Create a blank canvas with the same size as the original image; Based on the position information, draw the extracted contour features on the blank canvas, generate a texture map according to the extracted texture features, then overlay the texture map as a rendering layer on the blank canvas, and finally overlay the thermal distribution features on the blank canvas to obtain a fused Mini LED solder joint image; Among them, the texture features are highlighted using different grayscale values, and the thermal distribution features are displayed using pseudo-color mapping to show the thermal distribution.

4. The Mini LED solder joint quality analysis method according to claim 3, wherein After the step of overlaying the thermal distribution features on the blank canvas to obtain a fused Mini LED solder joint image, it further includes: Perform data augmentation on the fused Mini LED solder joint defect images according to a preset rule. The step of performing data augmentation on the fused Mini LED solder joint defect images according to a preset rule includes: Increase the brightness or color of the contour feature layer in the fused Mini LED solder joint defect image; and / or Apply random noise or variation to the texture feature layer; and / or Perform weighted overlay on the thermal distribution features to make them present different thermal distribution situations.

5. The Mini LED solder joint quality analysis method according to claim 4, characterized in that The step of data augmentation for the fused Mini LED solder joint image according to the preset rules further includes: Applying different degrees of compression and detail loss to each feature layer and adjusting the contrast, hue, or saturation of each feature layer; and / or Adding random perturbations in the normal direction of the texture feature layer; and / or Introducing local occlusions in the fused Mini LED solder joint defect image.

6. The Mini LED solder joint quality analysis method according to claim 3, wherein The step of using the texture map as a rendering layer and overlaying it on a blank canvas, and finally overlaying the heat distribution feature on the blank canvas to obtain the fused Mini LED solder joint image includes: Using Fourier transform to convert the images of the texture map and the heat distribution feature from the spatial domain to the frequency domain respectively; Performing different weighted superpositions on the low-frequency components and high-frequency components of the images of the texture map and the heat distribution feature respectively to retain the low-frequency information of the texture and enhance the high-frequency part of the image of the heat distribution feature; Converting the superimposed frequency-domain image back to the spatial domain through inverse Fourier transform to obtain the fused Mini LED solder joint image.

7. The Mini LED solder joint quality analysis method according to claim 3, wherein The step of using the texture map as a rendering layer and overlaying it on a blank canvas, and finally overlaying the heat distribution feature on the blank canvas to obtain the fused Mini LED solder joint image further includes: Controlling the fusion degree of the texture map and the heat distribution feature by using different transparency values to ensure that the texture map and the heat distribution feature do not cover each other when overlaid.

8. A Mini LED solder joint quality analysis system, characterized in that, The system includes: An acquisition module for respectively acquiring the Mini LED solder joint infrared image and the visible light image collected by the infrared image acquisition device and the visible light image acquisition device, and performing feature fusion on the Mini LED solder joint infrared image and the visible light image to obtain a fused Mini LED solder joint image; A fusion module for collecting a preset number of fused Mini LED solder joint defect images and fused Mini LED solder joint normal images, and establishing a training data set according to the fused Mini LED solder joint defect images and the fused Mini LED solder joint normal images; A detection module for inputting the training data set into a preset neural network for deep learning training to obtain a solder joint defect detection model, and acquiring a fused Mini LED solder joint image to be detected, and inputting the fused Mini LED solder joint image to be detected into the solder joint defect detection model to detect whether there are defects in the Mini LED solder joint.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

Images