Visual inspection method and device for reflow soldering quality and computer equipment

Abstract

The invention discloses a visual inspection method and device for reflow soldering quality and computer equipment. The method comprises the steps of obtaining a to-be-detected product image entering a detection area; preprocessing the to-be-detected product image and extracting a welding spot ROI region; the welding spot ROI is put into a defect classification model for welding defect identification and classification, and a classification result is obtained; and determining an equipment parameter adjustment scheme according to the classification result in combination with a preset adjustment rule, and adjusting the reflow soldering equipment according to the equipment parameter adjustment scheme. By implementing the method, the welding spot quality can be automatically recognized in the welding process, the production parameters are automatically adjusted according to the detection result, the welding quality is effectively improved, potential quality problems are avoided, the production efficiency can be improved, manual intervention is reduced, and the labor cost is saved.

Description

Technical Field

[0001] The present invention relates to a method for detecting welding quality, and more specifically to a visual inspection method, device and computer equipment for the welding quality of reflow soldering. Background Art

[0002] Reflow soldering is a common welding technology in modern electronic assembly processes, and its welding quality has a crucial impact on the performance and reliability of electronic products. Currently, most reflow soldering equipment mainly relies on traditional temperature control and chain speed adjustment mechanisms to ensure welding quality. These traditional methods compensate for possible problems during the welding process by adjusting the heating temperature and conveyor speed of the equipment. However, the limitation of these methods is that they usually cannot monitor and identify specific defects in real time during the welding process. That is to say, when welding defects occur, the equipment can only correct them by manually adjusting the temperature and chain speed, rather than directly diagnosing the specific causes of the problems.

[0003] In this case, traditional corrective measures are often based on empirical speculation rather than precise analysis based on real-time data. This lack of precise feedback in the corrective method not only affects production efficiency but also may affect the quality of the final product because welding defects may not be detected and corrected in a timely manner. Therefore, there is a need for more precise control of the reflow soldering process, especially for the real-time monitoring and automatic adjustment of solder joint quality.

[0004] Therefore, it is necessary to design a new method to automatically identify the quality of solder joints during the welding process, automatically adjust production parameters according to the detection results, effectively improve welding quality, avoid potential quality problems, improve production efficiency, reduce manual intervention and save labor costs. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a visual inspection method, device and computer equipment for the welding quality of reflow soldering.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions: A visual inspection method for the welding quality of reflow soldering, including:

[0007] Obtain an image of the product to be detected entering the detection area;

[0008] Preprocess the image of the product to be detected and extract the solder joint ROI area;

[0009] Input the solder joint ROI area into a defect classification model for welding defect identification and classification to obtain a classification result;

[0010] Determine a device parameter adjustment plan according to the classification result in combination with a preset adjustment rule, and adjust the reflow soldering equipment according to the device parameter adjustment plan.

[0011] 2. The visual inspection method for the reflow soldering quality according to claim 1, wherein the image of the product to be inspected is obtained by taking pictures with an industrial camera installed in the outlet cooling area of the reflow soldering equipment. Among them, the industrial camera and the annular LED light source are installed in the outlet cooling area of the reflow soldering equipment in a bilaterally symmetric layout.

[0012] Its further technical solution is: the preprocessing of the image of the product to be inspected and the extraction of the solder joint ROI area include:

[0013] Performing fiducial point positioning on the image of the product to be inspected, correcting the offset of the image of the product to be inspected to obtain a corrected image;

[0014] Extracting the solder joint ROI area from the corrected image through threshold segmentation and morphological processing.

[0015] Its further technical solution is: the defect classification model is obtained by training a deep learning model with several solder joint ROI areas with labels of different defect types and the positions of the defects as a sample set.

[0016] Its further technical solution is: the defect classification model is obtained by training a deep learning model with several solder joint ROI areas with labels of different defect types and the positions of the defects as a sample set, including:

[0017] Collecting a data set of several labeled images, where each image includes a solder joint ROI area with labeled defect types and positions;

[0018] Processing the data set to obtain a sample set;

[0019] Constructing a deep learning model;

[0020] Using the sample set to optimize the deep learning model through transfer learning, focal loss, and the AdamW optimizer, and performing FP16 quantization acceleration using TensorRT to obtain a defect classification model.

[0021] Its further technical solution is: the use of the sample set to optimize the deep learning model through transfer learning, focal loss, and the AdamW optimizer, and performing FP16 quantization acceleration using TensorRT to obtain a defect classification model, including:

[0022] Using the sample set, perform transfer learning, fine-tune based on the weights pre-trained on ImageNet, use focal loss to solve the class imbalance problem, adopt AdamW as the optimizer, with a learning rate of 3e-4, adopt the Cosine decay strategy, learn the deep learning model, and use TensorRT to quantize the deep learning model to obtain the trained model, and convert the trained model into the ONNX format to obtain the defect classification model.

[0023] Its further technical solution is: The device adjustment plan includes adjusting the temperature and chain speed of the reflow soldering temperature control system.

[0024] Its further technical solution is: Determining the device parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjusting the reflow soldering equipment according to the device parameter adjustment plan, including:

[0025] Determine the device parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjust the reflow soldering equipment according to the device parameter adjustment plan. When adjusting, follow the safety boundary constraints, where the safety boundary constraints include: the adjustment range of the temperature in the temperature zone does not exceed ±3°C / min, the change range of the chain speed does not exceed ±10%, and upper and lower limits are set to avoid too fast or stopped chain speed.

[0026] The present invention also provides a visual inspection device for reflow soldering quality, including:

[0027] An image acquisition unit for acquiring an image of the product to be detected entering the detection area;

[0028] An ROI region extraction unit for preprocessing the image of the product to be detected and extracting the solder joint ROI region;

[0029] A classification unit for performing welding defect identification and classification on the solder joint ROI region into the defect classification model to obtain a classification result;

[0030] An adjustment unit for determining the device parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjusting the reflow soldering equipment according to the device parameter adjustment plan.

[0031] The present invention also provides a computer device, which includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program, the above method is implemented.

[0032] The beneficial effects of the present invention compared with the prior art are as follows: By acquiring the image of the product to be detected and preprocessing it, the solder joint area is extracted; then, the solder joint ROI is input into the defect classification model for welding defect identification and classification to generate a classification result; according to the classification result and in combination with the preset adjustment rules, an equipment parameter adjustment plan is automatically formulated; then, according to the adjustment plan, the production parameters of the reflow soldering equipment are automatically adjusted. This process realizes the automatic detection and adjustment of welding quality, effectively improves the welding quality, reduces potential problems, improves production efficiency, and reduces manual intervention and labor costs.

[0033] The following further describes the present invention with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 It is a flowchart of the visual inspection method for reflow soldering welding quality provided by an embodiment of the present invention;

[0036] Figure 2 It is a sub - flowchart of the visual inspection method for reflow soldering welding quality provided by an embodiment of the present invention;

[0037] Figure 3 It is a sub - flowchart of the visual inspection method for reflow soldering welding quality provided by an embodiment of the present invention;

[0038] Figure 4 It is a schematic block diagram of the visual inspection device for reflow soldering welding quality provided by an embodiment of the present invention;

[0039] Figure 5 It is a schematic block diagram of the **** unit of the visual inspection device for reflow soldering welding quality provided by an embodiment of the present invention;

[0040] Figure 6 It is a schematic block diagram of the computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0042] It should be understood that when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0043] It should also be understood that the terms used in this specification of the present invention are merely for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in this specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0044] It should be further understood that the term "and / or" used in this specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0045] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of the visual inspection method for the reflow soldering quality provided by the embodiment of the present invention. This visual inspection method for the reflow soldering quality is applied to a server. The server can be a controller of a reflow soldering device or an independent server. The server interacts with the reflow soldering device and an industrial camera, takes an image of the product to be detected through the industrial camera, preprocesses the image, extracts the solder joint area for defect identification; uses a deep learning model to classify the solder joint defects, and automatically adjusts the temperature and chain speed of the reflow soldering device according to the classification results in combination with preset rules; the model is trained through transfer learning, focal loss and an optimizer to optimize the defect classification effect; the device adjustment scheme follows safety boundary constraints to ensure the soldering quality and device safety. This method realizes automatic identification and adjustment during the soldering process, improves production efficiency and quality, and reduces manual intervention.

[0046] Figure 1 is a schematic flowchart of the visual inspection method for the reflow soldering quality provided by the embodiment of the present invention. As Figure 1 shown, this method includes the following steps S110 to S140.

[0047] S110. Obtain an image of the product to be detected entering the detection area.

[0048] In this embodiment, the image of the product to be detected is taken by an industrial camera installed in the outlet cooling area of the reflow soldering device. Among them, the industrial camera and the annular LED light source are installed in the outlet cooling area of the reflow soldering device in a bilateral symmetric layout manner.

[0049] The product images to be detected are obtained by an industrial camera. Specifically, these images are sourced from the outlet cooling zone of the reflow soldering equipment. The temperature in the cooling zone is relatively low (usually less than 60°C) to avoid interference from high temperatures to the imaging equipment. The industrial camera is installed at the position of the cooling zone and works in conjunction with a ring-shaped LED light source to form a bilaterally symmetric layout. This design can ensure the uniformity of the light source during the imaging process and effectively avoid interference caused by the high temperatures around the equipment.

[0050] In this embodiment, the product here is a PCB board.

[0051] Specifically, the industrial camera and the ring-shaped LED light source are installed in the outlet cooling zone of the reflow soldering equipment to ensure sufficient and uniform light during image capture. The ring-shaped LED light source provides stable illumination and can select red or blue light according to needs to adapt to the visual inspection requirements of different solder joints.

[0052] The bilaterally symmetric layout of the camera and the light source helps reduce the influence of shadows or uneven illumination in the image, ensuring that the captured product images are clear and of high quality.

[0053] The timing of triggering the photo taking is achieved by the linkage of a photoelectric sensor and a conveyor belt encoder. The photoelectric sensor is used to sense the current position of the PCB, while the conveyor belt encoder provides accurate data on the moving speed and position of the PCB. The two work together to ensure that the camera takes pictures at the appropriate position and time, thus ensuring the accuracy of the image data.

[0054] The obtained images are called product images to be detected. These images are mainly used for subsequent deep learning inference and defect classification processing. The product images to be detected usually include an overall view of the PCB, covering the solder joint area, ensuring that each solder joint can be analyzed in detail.

[0055] After the reflow soldering process is completed, the product will pass through the cooling zone. Due to the relatively low temperature in the cooling zone, the industrial camera can capture high-quality images in an environment without interference from heat sources. In addition, the temperature in the cooling zone also helps reduce the thermal expansion and contraction phenomenon on the surface of the PCB after image acquisition, thus ensuring that the images of the solder joints are clear and distinguishable.

[0056] Through the above design and process, the product images to be detected can provide sufficient information for subsequent defect identification and classification. The entire image acquisition process lays the foundation for subsequent deep learning analysis, making the subsequent defect classification more accurate and reliable.

[0057] S120. Preprocess the product images to be detected and extract the solder joint ROI area.

[0058] In this embodiment, the welding ROI area refers to the image formed by the area where the solder joints are located in the product images to be detected.

[0059] During the visual inspection process in the reflow soldering equipment, it is first necessary to preprocess the PCB image to be inspected in order to extract the solder joint area related to the soldering quality. This process is called "ROI (Region of Interest) extraction" in machine vision, which is a key step in the inspection.

[0060] In one embodiment, please refer to Figure 2 , the above step S120 may include steps S121 to S122.

[0061] S121. Locate the fiducial points of the product image to be inspected, correct the offset of the product image to be inspected, so as to obtain a corrected image.

[0062] In this embodiment, the corrected image refers to the image obtained after the offset correction of the product image to be inspected.

[0063] Specifically, on the surface of the PCB board, there are usually some calibration points or feature points, usually called fiducial points or Fiducial Marks. These fiducial points are used to ensure that the actual position of the PCB can be aligned when taking images. Locating these fiducial points is the first step in image processing. The accurate location of the fiducial points can provide a key reference for subsequent image correction. Generally, this step will use an image processing library (such as OpenCV) to perform image matching.

[0064] Due to factors such as the camera shooting angle, lens distortion, and conveyor belt displacement, the PCB image to be inspected may be offset. In order to ensure the accuracy of the image, it is necessary to perform geometric correction on the image to correct this offset. This usually includes operations such as rotation, translation, and scaling, so that the image is aligned with the fiducial points. In OpenCV, affine transformation or perspective transformation can be used to correct the image, so that the image offset problem is solved, and thus the subsequent solder joint extraction and analysis are more accurate.

[0065] By completing step S121, the obtained image will eliminate the position offset, and all subsequent solder joint extraction and defect analysis will be based on this corrected image.

[0066] S122. Extract the solder joint ROI area from the corrected image through threshold segmentation and morphological processing.

[0067] In this embodiment, in the corrected image, the color and brightness of the solder joints are usually significantly different from the surrounding background. To extract the solder joints, threshold segmentation technology in image processing can be used. By setting an appropriate grayscale value or color threshold, the solder joints in the image can be distinguished from other regions. For example, the regions in the image with a grayscale value lower than a certain value are regarded as the background, while the regions higher than this value are regarded as the solder joints. This process helps to extract the solder joint regions, thereby reducing the interference of the background.

[0068] After the solder joint regions are extracted, there may be some noise or irregular shapes (for example, the edges of the solder joint regions may not be smooth, or there may be small noise points). To remove these interferences and extract a more accurate solder joint ROI (region of interest), morphological processing is usually required. Morphological processing is an image operation based on geometric shapes. Common operations include dilation, erosion, opening operation, and closing operation, etc. These operations can effectively eliminate noise, smooth the edges of the solder joints, and further extract representative solder joint regions.

[0069] In other embodiments, the following methods can also be used to extract the solder joint ROI regions:

[0070] ROI extraction based on edge detection. Canny edge detection is a commonly used edge detection algorithm. It identifies edges by looking for intensity changes in the image. In the solder joint image, there is usually an obvious edge between the solder joints and the background. The edge information of the solder joints can be accurately extracted through Canny edge detection. Then, the contour extraction algorithm or morphological processing can be combined to extract the solder joint ROI. The Sobel operator is also a common edge detection method, which can be used to detect the gradient changes in the image and thus help to identify the edges of the solder joints.

[0071] ROI extraction based on color space. The color of the solder joints usually has a relatively significant change, especially in terms of color, which may be significantly different from the surrounding background. In the HSV color space, the hue (H) and saturation (S) are more sensitive to the color of the solder joints. The color regions of the solder joints can be extracted by setting appropriate thresholds. The Lab color space is defined based on the perception of the human visual system and is commonly used for image color analysis. It is more adaptable to light changes than the RGB color space. Therefore, it may be more robust in extracting the solder joint regions under some conditions.

[0072] ROI extraction based on region growing. Region growing is another commonly used image segmentation method. It usually starts from a seed pixel and gradually expands to the surrounding pixels until a certain stopping criterion (such as similarity, color, or grayscale value) is reached. This method can effectively extract relatively consistent regions from the image, such as solder joints. Especially when the grayscale value or color change in the solder joint region is relatively uniform, region growing can accurately extract this region.

[0073] For ROI extraction based on deep learning, if there is sufficient labeled data, a deep learning model can be trained to automatically detect solder joints and extract the ROI region. Object detection models based on CNN (such as YOLO, Faster R-CNN, RetinaNet, etc.) can automatically identify the positions and bounding boxes of solder joints in an image, and further extract the regions of interest. The advantage of this method is that it can handle more complex image features and has higher robustness. Using semantic segmentation networks such as U-Net and DeepLab, the entire image can be divided into different regions, and each region can be marked as belonging to a solder joint or the background. In this way, the solder joint region can be extracted more accurately, especially in the case of complex backgrounds and multiple solder joints.

[0074] For template matching based on machine vision, if the shapes and sizes of the solder joints are relatively consistent, the template matching method can be used to extract the ROI region. Template matching slides a fixed template across the entire image to find the region that best matches the template. This method can be used when the solder joints have relatively regular shapes. Although this method may be sensitive to noise and scale changes, it has good effects in some fixed scenarios.

[0075] For segmentation based on adaptive thresholding, compared with global thresholding segmentation, adaptive thresholding segmentation can dynamically adjust the threshold for different regions in the image. For example, when the brightness of the image changes greatly, a lower threshold may be set in some regions, while a higher threshold may be set in other regions. Common adaptive thresholding methods include the mean method, weighted mean method, and Gaussian weighted method. Through adaptive thresholding, the situation where the contrast between the solder joints and the background varies greatly can be better handled.

[0076] For the image watershed algorithm, the watershed algorithm is a topology-based image segmentation method that identifies different regions in the image by simulating the process of water flowing from low valleys to highlands. Solder joints usually appear as isolated regions in the image, and the watershed algorithm can effectively separate these isolated solder joint regions, which is suitable for some solder joints with irregular shapes.

[0077] That is, in addition to thresholding segmentation and morphological processing, methods based on edge detection, color space, region growing, deep learning, template matching, etc. can also be used to extract the ROI region of solder joints. The specific ROI extraction method should be determined according to the characteristics of the image, background complexity, and requirements for computing resources. Combining multiple methods (for example, first using edge detection to extract the approximate region, and then using morphological processing to refine the result) can often obtain better results.

[0078] S121 performs positioning and image correction through reference points, aligning the image of the product to be detected and eliminating offsets, ensuring the accuracy of solder joint extraction.

[0079] S122 extracts the ROI region of the solder joints through threshold segmentation and morphological processing, thereby providing clear input data for the defect classification of the deep learning model.

[0080] These steps provide accurate data input for subsequent deep learning inference (defect classification) and closed-loop quality control, ensuring the effective operation of the automatic detection and real-time feedback mechanism for welding quality.

[0081] S130: Input the solder joint ROI region into the defect classification model to identify and classify welding defects, so as to obtain a classification result.

[0082] In this embodiment, the classification result refers to whether there are defects in the image of the product to be detected and the corresponding defect types when there are defects.

[0083] Among them, the defect classification model is obtained by training a deep learning model with several solder joint ROI regions with labels of different defect types and the positions where the defects are located as a sample set.

[0084] In one embodiment, please refer to Figure 3 , the above-mentioned defect classification model is obtained by training a deep learning model with several solder joint ROI regions with labels of different defect types and the positions where the defects are located as a sample set, and may include steps S131 to S134.

[0085] S131: Collect a dataset of several labeled images, where each image includes a solder joint ROI region with labeled defect types and positions.

[0086] In this embodiment, a large number of labeled image datasets are collected. These images need to contain solder joint ROI regions, and each region should be clearly labeled with its defect type (such as insufficient soldering, bridging, offset, solder ball, etc.) and the specific position of the defect. Through such a dataset, the model can learn the characteristics of different defects and their locations.

[0087] Specifically, each image contains the ROI of one or more solder joints and labels the defect type and the position where the defect is located for each solder joint. The label information includes but is not limited to:

[0088] Defect type: such as insufficient soldering, bridging, offset, solder ball, etc. These defect types represent different failure modes of the welding process.

[0089] Defect position: that is, the specific position where the defect appears within the solder joint area.

[0090] Through this data collection method, the deep learning model can learn different types of defects and their specific distributions in the images, providing sufficient samples to train and validate the model.

[0091] S132. Process the dataset to obtain a sample set.

[0092] In this embodiment, for the collected image dataset, certain processing is required to ensure its suitability for the training of the deep learning model. This typically includes preprocessing of the images, such as operations like normalization and data augmentation, to enhance the generalization ability of the model. For example, the images can be rotated, noise can be added, the brightness can be adjusted, etc., to increase the diversity of the data.

[0093] Specifically, for normalization: Scale the pixel values of the images to a certain range (e.g., 0 to 1) to improve the stability of model training.

[0094] For data augmentation: Use some techniques to expand the diversity of the dataset and increase the robustness of the model. Common techniques include:

[0095] Rotation: Rotate the images by a certain angle to simulate solder joint images under different shooting angles.

[0096] Noise addition: Add random noise to the images to make the model more robust when processing noisy images.

[0097] Brightness adjustment: Simulate images under different lighting conditions.

[0098] Data augmentation helps prevent overfitting and enables the model to better generalize to new, unseen images.

[0099] S133. Build a deep learning model.

[0100] In this case, after the dataset is processed, the next step is to build a deep learning model. This is usually based on technologies such as convolutional neural networks, and through continuous iterative optimization, it extracts the features of the solder joint images to help the model identify different types of welding defects.

[0101] Specifically, the model architecture design generally uses a convolutional neural network (CNN) architecture. The convolutional layer can effectively extract local features in the images (such as the shape, texture, etc. of the solder joints). Through multiple convolutional layers, different levels of features in the images are gradually extracted, and these features are very important for the classification task. After the convolutional layer, fully connected layers are usually added to map the extracted features to the final categories for defect classification.

[0102] The model architecture can be fine-tuned according to the requirements of the task. For example, select appropriate convolutional kernel sizes, activation functions, pooling layers, etc.

[0103] S134. Optimize the deep learning model using the sample set through transfer learning, focal loss, and the AdamW optimizer, and perform FP16 quantization acceleration using TensorRT to obtain a defect classification model.

[0104] In this embodiment, use the sample set to perform transfer learning, fine-tune based on the weights pre-trained on ImageNet, adopt focal loss to solve the class imbalance problem, use AdamW as the optimizer with a learning rate of 3e-4, adopt the Cosine decay strategy to learn the deep learning model, and use TensorRT to quantize the deep learning model to obtain a trained model, and convert the trained model to the ONNX format to obtain a defect classification model.

[0105] Specifically, in the way of transfer learning, use the pre-trained ImageNet model weights to fine-tune the deep learning model to reduce the training time and improve the performance of the model. To solve the class imbalance problem, use focal loss to optimize the loss function. The AdamW optimizer is used, and the learning rate is adjusted through the Cosine decay strategy to optimize the training process of the model.

[0106] In addition, after training is completed, use TensorRT to quantize the model, convert it to the FP16 format, thereby accelerating the inference process. Convert the trained model to the ONNX format for cross-platform deployment and use.

[0107] Since training a deep learning model requires a large amount of sample data, directly training from scratch may be very time-consuming and require a large amount of data. To save training time and improve the performance of the model, transfer learning is usually adopted, that is, use the model weights that have been trained on a large dataset (such as ImageNet) for initialization.

[0108] Then, for the specific solder joint defect classification task, fine-tune these pre-trained weights to make the model adapt to the new task. Usually, only the last few layers are adjusted, or all layers are adjusted to adapt to the specific task.

[0109] In the case of class imbalance (such as some types of welding defects occurring less frequently), the conventional cross-entropy loss function may not be able to effectively train the model, resulting in poor recognition ability of the model for minority classes. Focal loss is a modified loss function that enhances the model's attention to minority classes by assigning greater weights to difficult-to-classify samples.

[0110] The AdamW optimizer is an improved version of the Adam optimizer, specifically optimized for weight decay (L2 regularization). It can better control the overfitting phenomenon of the model while maintaining efficient computation.

[0111] During the training process, the learning rate has an important impact on the convergence speed and effect of model training. The Cosine decay strategy can gradually reduce the learning rate as the training progresses, enabling the model to converge more stably to the optimal solution in the later stage of training.

[0112] After training is completed, quantization is usually performed. The TensorRT tool is used to reduce the precision of the model to FP16 (16-bit floating point) to accelerate the inference process. FP16 quantization can significantly reduce the resource consumption of model computation while maintaining relatively high precision, thereby improving the running efficiency of the model in practical applications.

[0113] ONNX (Open Neural Network Exchange) is an open intermediate representation format for deep learning frameworks, which enables trained models to be interoperable between different platforms and frameworks. After training is completed, the model is converted to the ONNX format for easy deployment on different hardware platforms or inference engines.

[0114] Optimize the performance of the deep learning model through transfer learning, use Focal Loss to solve the class imbalance problem, and adopt the AdamW optimizer to improve the training effect. At the same time, accelerate the inference process by quantifying the model to FP16 to ensure meeting the real-time requirements in the production environment. The goal of performance optimization is to achieve an inference speed of less than 50 milliseconds for each solder joint to ensure high throughput.

[0115] Through the above steps, the constructed solder joint defect classification model can not only efficiently identify different types of welding defects but also perform fast inference on multiple platforms. Through technical means such as transfer learning, data augmentation, optimizer selection, focal loss, and quantization, the efficiency and accuracy of the model are ensured. In addition, the export in ONNX format also ensures the portability of the model, facilitating its application in the actual production environment.

[0116] Regarding the problem of insufficient data, defect data can be synthesized through a generative adversarial network to supplement the training dataset and improve the generalization ability of the model. Specifically, the training of a deep learning model requires a large amount of high-quality labeled data. However, in practical applications, obtaining sufficient defect data can be a challenge, especially for some rare defect types. To overcome this problem, generative adversarial network technology can be adopted.

[0117] A generative adversarial network is a model composed of two neural networks: a generator and a discriminator. The goal of the generator is to generate data similar to the real data distribution, while the goal of the discriminator is to distinguish between the generated data and the real data. Through adversarial training, the generator gradually generates more realistic data until the discriminator can no longer distinguish between the true and the false. In the scenario of defect detection, generative adversarial networks can be used to generate synthetic data similar to real defects to supplement the training dataset.

[0118] Through generative adversarial networks, various types of welding defect images (such as solder joint voids, offsets, bridges, etc.) can be generated, and these synthetic data are mixed with real defect data during training. This method effectively alleviates the problem of insufficient data, especially for some difficult-to-obtain defect types. By enhancing the dataset, the deep learning model can better learn different types of welding defects.

[0119] Since the generated synthetic defect images have diversity in samples and can simulate different welding environments and conditions, combining these synthetic data during model training can make the model more generalization-capable and avoid overfitting. In this way, the model can not only identify the defects seen in the experimental data but also have good recognition ability for various new types of defects that appear in the actual production environment.

[0120] The advantage of this method is that it does not require a large amount of additional manually labeled data. It can generate rich training samples through a computer, improving the accuracy and robustness of the model.

[0121] S140. Determine the equipment parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjust the reflow soldering equipment according to the equipment parameter adjustment plan.

[0122] In this embodiment, the equipment adjustment plan includes adjusting the temperature and chain speed of the reflow soldering temperature control system.

[0123] Specifically, determine the equipment parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjust the reflow soldering equipment according to the equipment parameter adjustment plan. When adjusting, follow the safety boundary constraints, where the safety boundary constraints include: the adjustment range of the temperature in the temperature zone does not exceed ±3°C / min, the change range of the chain speed does not exceed ±10%, and set upper and lower limits to avoid too fast or stopped chain speed.

[0124] Specifically, the detection system automatically feeds back to the reflow soldering equipment and makes corresponding parameter adjustments through the classification of welding defects in combination with the preset adjustment rules. Specifically, the equipment adjustment includes:

[0125] Temperature control system adjustment: According to the correlation between the defect type and the temperature, adjust the temperature control system of the reflow soldering to ensure that the temperature during the welding process is within a reasonable range and prevent defects caused by too high or too low temperature.

[0126] Chain speed adjustment: Adjust the chain speed of the reflow soldering according to the defect type and welding quality feedback. For some defects, such as solder joints being loose or bridged, it may be necessary to slow down the chain speed, while for other types of defects, it may be necessary to speed up the chain speed.

[0127] When adjusting the equipment parameters, certain safety boundary constraints must be observed to ensure the normal operation of the equipment and the stability of the welding quality:

[0128] Temperature adjustment range: The temperature adjustment range of the temperature zone is limited to ±3°C / min to avoid negative impacts on the solder joints caused by thermal shock.

[0129] Chain speed adjustment range: The adjustment range of the chain speed is limited to ±10%, and upper and lower limits must be set to avoid quality problems of the solder joints caused by the chain running too fast or PCB board stacking caused by the chain running too slow.

[0130] During the operation of the equipment, historical production data (such as temperature curves, chain speeds) is continuously collected and correlated with the detected defect types for analysis. Based on the IPC standard and the experience of process engineers, the expert rule base establishes a mapping relationship between defect types and parameter adjustments and is optimized in a data-driven manner. Among them, the expert rule base is a database integrating preset adjustment rules.

[0131] In this embodiment, the above-mentioned expert rule base is shown in Table 1.

[0132] Table 1. Expert rule base

[0133]

[0134]

[0135]

[0136]

[0137]

[0138] In addition, for the detection of micro solder joints, a 5X telecentric lens (with a resolution of 5μm / pixel) can be used to ensure that even micro solder joints can be clearly detected. Specifically, during the reflow soldering process, the size of the solder joints may be small, especially in high-density electronic products. The detection of micro solder joints becomes particularly important. To effectively capture these micro solder joints, using a telecentric lens is a common solution.

[0139] A telecentric lens is a special lens design that has a "telecentric optical axis", meaning that changes in the distance between the object and the lens do not significantly affect the image magnification. In a conventional lens, the closer the object is to the lens, the larger and more distorted the image becomes. In contrast, a telecentric lens can maintain the same magnification ratio, making it particularly suitable for high-precision measurements and the detection of small objects.

[0140] By using a 5X telecentric lens in combination with a high-resolution camera (such as a camera sensor with a resolution of 5μm / pixel), it is possible to ensure that even very small solder joints (such as those less than 0.2mm) can be clearly imaged. The high-resolution sensor can capture details, providing sufficient information to identify defects in small solder joints, such as poor soldering and misalignment.

[0141] This method ensures that even in the case of small-sized solder joints, the system can clearly identify and classify each solder joint, thereby avoiding missed detections due to unclear inspections.

[0142] For the problem of highly reflective surfaces, a polarizer and low-angle illumination can be used to reduce the interference of solder surface reflection on image quality. Specifically, during the reflow soldering process, especially for components soldered with solder paste, their surfaces may have strong reflective properties. The reflected light can interfere with the image acquisition of the camera, resulting in a decline in image quality and thus affecting the accuracy of defect detection. To solve this problem, the following techniques are usually adopted:

[0143] Polarizer: A polarizer can effectively reduce the interference of surface reflected light. Reflected light often has a strong polarization direction in a certain part of the light wave, and the polarizer reduces the interference caused by reflected light by selectively filtering out light with a specific polarization direction. By using the polarizer in combination with the lens, strong reflected light from the solder paste surface or other highly reflective surfaces can be prevented from affecting the image quality.

[0144] Low-angle illumination: By changing the angle of the light source so that the light irradiates the solder joint surface at a lower angle, the interference of reflected light can be further reduced. Low-angle illumination can ensure that the surface of the solder joint is evenly illuminated while preventing the light from being directly reflected into the lens. This can ensure that the details in the image are clearly visible, especially when the solder joint surface is smooth or has metallic reflectivity.

[0145] By these two methods, it is possible to effectively improve the image quality, reduce the reflection interference, and ensure the stable detection of the quality of solder joints on highly reflective surfaces.

[0146] The above visual inspection method for reflow soldering quality obtains an image of the product to be inspected and preprocesses it to extract the solder joint area. Then, the solder joint ROI is input into the defect classification model for welding defect identification and classification to generate a classification result. According to the classification result and in combination with the preset adjustment rules, an equipment parameter adjustment plan is automatically formulated. Then, the production parameters of the reflow soldering equipment are automatically adjusted according to the adjustment plan. This process realizes the automatic detection and adjustment of welding quality, effectively improves welding quality, reduces potential problems, improves production efficiency, and reduces manual intervention and labor costs.

[0147] Figure 4 It is a schematic block diagram of a visual inspection device 300 for reflow soldering quality provided by an embodiment of the present invention. As Figure 4 shown, corresponding to the above visual inspection method for reflow soldering quality, the present invention also provides a visual inspection device 300 for reflow soldering quality. The visual inspection device 300 for reflow soldering quality includes units for executing the above visual inspection method for reflow soldering quality, and this device can be configured in terminals such as desktop computers, tablet computers, laptops, etc. Specifically, please refer to Figure 4 , the visual inspection device 300 for reflow soldering quality includes an image acquisition unit 301, an ROI area extraction unit 302, a classification unit 303, and an adjustment unit 304.

[0148] The image acquisition unit 301 is used to acquire an image of the product to be inspected entering the detection area; the ROI area extraction unit 302 is used to preprocess the image of the product to be inspected and extract the solder joint ROI area; the classification unit 303 is used to perform welding defect identification and classification on the solder joint ROI area in the defect classification model to obtain a classification result; the adjustment unit 304 is used to determine an equipment parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjust the reflow soldering equipment according to the equipment parameter adjustment plan.

[0149] In one embodiment, as Figure 5 shown, the ROI area extraction unit 302 includes a correction subunit 3021 and an area extraction subunit 3022.

[0150] The correction subunit 3021 is used to perform fiducial point positioning on the image of the product to be inspected and correct the offset of the image of the product to be inspected to obtain a corrected image; the area extraction subunit 3022 is used to extract the solder joint ROI area from the corrected image through threshold segmentation and morphological processing.

[0151] In one embodiment, the visual inspection device 300 for reflow soldering quality further includes a model training unit, which is used to train a deep learning model with several solder joint ROI regions with tags of different defect types and defect locations as a sample set to obtain a defect classification model.

[0152] In one embodiment, the model training unit includes:

[0153] An acquisition subunit, which is used to acquire a data set of several labeled images, where each image includes a solder joint ROI region labeled with defect types and locations; a processing subunit, which is used to process the data set to obtain a sample set; a construction subunit, which is used to construct a deep learning model; a training subunit, which is used to optimize the deep learning model with the sample set through transfer learning, focal loss, and an AdamW optimizer, and perform FP16 quantization acceleration using TensorRT to obtain a defect classification model.

[0154] In one embodiment, the training subunit is used to use transfer learning with the sample set, fine-tune based on the weights pre-trained on ImageNet, adopt focal loss to solve the class imbalance problem, use AdamW as the optimizer, with a learning rate of 3e-4, adopt the Cosine decay strategy to learn the deep learning model, and use TensorRT to quantize the deep learning model to obtain a trained model, and convert the trained model into the ONNX format to obtain a defect classification model.

[0155] In one embodiment, the adjustment unit 304 is used to determine a device parameter adjustment plan according to the classification result in combination with a preset adjustment rule, and adjust the reflow soldering equipment according to the device parameter adjustment plan, and follow safety boundary constraints during adjustment, where the safety boundary constraints include: the adjustment range of the temperature in the temperature zone does not exceed ±3°C / min, the change range of the chain speed does not exceed ±10%, and upper and lower limits are set to avoid the chain speed being too fast or stopping.

[0156] It should be noted that those skilled in the art can clearly understand that the specific implementation processes of the above-mentioned visual inspection device 300 for reflow soldering quality and each unit can refer to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and conciseness of description, they will not be elaborated here.

[0157] The above-mentioned visual inspection device 300 for reflow soldering quality can be implemented in the form of a computer program, and this computer program can run on a computer device as shown in Figure 6 shown.

[0158] Please refer to Figure 6 , Figure 6It is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 may be a server. Among them, the server may be an independent server or a server cluster composed of multiple servers.

[0159] Refer to Figure 6 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501. Among them, the memory may include a non-volatile storage medium 503 and an internal memory 504.

[0160] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions. When the program instructions are executed, the processor 502 can be made to execute a visual inspection method for reflow soldering quality.

[0161] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0162] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can be made to execute a visual inspection method for reflow soldering quality.

[0163] The network interface 505 is used for network communication with other devices. Those skilled in the art can understand that Figure 6 the structure shown in

[0164] is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] Among them, the processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0165] Obtain an image of the product to be detected entering the detection area; preprocess the image of the product to be detected and extract the solder joint ROI area; input the solder joint ROI area into a defect classification model for welding defect identification and classification to obtain a classification result; determine a device parameter adjustment plan according to the classification result in combination with a preset adjustment rule, and adjust the reflow soldering equipment according to the device parameter adjustment plan.

[0166] Among them, the image of the product to be detected is obtained by an industrial camera installed in the outlet cooling area of the reflow soldering equipment. Among them, the industrial camera and the annular LED light source are installed in the outlet cooling area of the reflow soldering equipment in a bilateral symmetric layout manner.

[0167] Among them, the defect classification model is obtained by training a deep learning model with several solder joint ROI regions with labels of different defect types and the positions where the defects are located as a sample set.

[0168] The device adjustment scheme includes adjusting the temperature and chain speed of the reflow soldering temperature control system.

[0169] In one embodiment, when the processor 502 implements the step of preprocessing the image of the product to be detected and extracting the solder joint ROI region, the following steps are specifically implemented:

[0170] Locate the fiducial points of the image of the product to be detected, correct the offset of the image of the product to be detected, so as to obtain a corrected image; extract the solder joint ROI region from the corrected image through threshold segmentation and morphological processing.

[0171] In one embodiment, when the processor 502 implements the step that the defect classification model is obtained by training a deep learning model with several solder joint ROI regions with labels of different defect types and the positions where the defects are located as a sample set, the following steps are specifically implemented:

[0172] Collect a data set of several labeled images, where each image includes a solder joint ROI region with labeled defect types and positions; process the data set to obtain a sample set; construct a deep learning model; use the sample set to optimize the deep learning model through transfer learning, focal loss, and AdamW optimizer, and use TensorRT for FP16 quantization acceleration to obtain a defect classification model.

[0173] In one embodiment, when the processor 502 implements the step of using the sample set to optimize the deep learning model through transfer learning, focal loss, and AdamW optimizer, and using TensorRT for FP16 quantization acceleration to obtain a defect classification model, the following steps are specifically implemented:

[0174] Use the sample set to perform transfer learning, fine-tune based on the weights pre-trained on ImageNet, use focal loss to solve the class imbalance problem, use AdamW as the optimizer, the learning rate is 3e-4, adopt the Cosine decay strategy, learn the deep learning model, and use TensorRT to quantize the deep learning model to obtain a trained model, and convert the trained model into the ONNX format to obtain a defect classification model.

[0175] In one embodiment, when the processor 502 implements the step of determining a device parameter adjustment scheme according to the classification result in combination with a preset adjustment rule, and adjusting the reflow soldering device according to the device parameter adjustment scheme, the following steps are specifically implemented:

[0176] Determine the equipment parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjust the reflow soldering equipment according to the equipment parameter adjustment plan, and follow the safety boundary constraints during the adjustment. Among them, the safety boundary constraints include: the adjustment range of the temperature in the temperature zone does not exceed ±3°C / min, the change range of the chain speed does not exceed ±10%, and upper and lower limits are set to avoid the chain speed being too fast or stopped.

[0177] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0178] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, and the storage medium is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0179] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the following steps:

[0180] Obtain an image of the product to be detected entering the detection area; preprocess the image of the product to be detected and extract the solder joint ROI area; input the solder joint ROI area into a defect classification model for welding defect identification and classification to obtain a classification result; determine an equipment parameter adjustment plan according to the classification result in combination with the preset adjustment rules, and adjust the reflow soldering equipment according to the equipment parameter adjustment plan.

[0181] Among them, the image of the product to be detected is obtained by shooting with an industrial camera installed in the outlet cooling area of the reflow soldering equipment. Among them, the industrial camera and the annular LED light source are installed in the outlet cooling area of the reflow soldering equipment in a bilateral symmetric layout.

[0182] Among them, the defect classification model is obtained by training a deep learning model with a number of solder joint ROI regions with labels of different defect types and the positions where the defects are located as a sample set.

[0183] The device adjustment scheme includes adjusting the temperature and chain speed of the reflow soldering temperature control system.

[0184] In one embodiment, when the processor executes the computer program to implement the step of preprocessing the image of the product to be detected and extracting the solder joint ROI region, the following steps are specifically implemented:

[0185] Locate the fiducial points of the image of the product to be detected, correct the offset of the image of the product to be detected to obtain a corrected image; extract the solder joint ROI region from the corrected image through threshold segmentation and morphological processing.

[0186] In one embodiment, when the processor executes the computer program to implement the step that the defect classification model is obtained by training a deep learning model with a number of solder joint ROI regions with labels of different defect types and the positions where the defects are located as a sample set, the following steps are specifically implemented:

[0187] Collect a data set of a number of labeled images, where each image includes a solder joint ROI region with a labeled defect type and position; process the data set to obtain a sample set; construct a deep learning model; use the sample set to optimize the deep learning model through transfer learning, focal loss, and AdamW optimizer, and perform FP16 quantization acceleration using TensorRT to obtain a defect classification model.

[0188] In one embodiment, when the processor executes the computer program to implement the step of using the sample set to optimize the deep learning model through transfer learning, focal loss, and AdamW optimizer, and performing FP16 quantization acceleration using TensorRT to obtain a defect classification model, the following steps are specifically implemented:

[0189] Use the sample set for transfer learning, fine-tune based on the weights pre-trained on ImageNet, adopt focal loss to solve the class imbalance problem, use AdamW as the optimizer, the learning rate is 3e-4, adopt the Cosine decay strategy to learn the deep learning model, and use TensorRT to quantize the deep learning model to obtain a trained model, and convert the trained model to the ONNX format to obtain a defect classification model.

[0190] In one embodiment, when the processor executes the computer program to implement the step of determining a device parameter adjustment plan according to the classification result in combination with a preset adjustment rule and adjusting the reflow soldering equipment according to the device parameter adjustment plan, the following steps are specifically implemented:

[0191] Determine a device parameter adjustment plan according to the classification result in combination with a preset adjustment rule, and adjust the reflow soldering equipment according to the device parameter adjustment plan. When adjusting, follow the safety boundary constraints, where the safety boundary constraints include: the adjustment range of the temperature in the temperature zone does not exceed ±3°C / min, the change range of the chain speed does not exceed ±10%, and upper and lower limits are set to avoid the chain speed being too fast or stopped.

[0192] The storage medium can be a variety of computer-readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disc, etc., which can store program codes.

[0193] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0194] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0195] The steps in the method embodiments of the present invention can be adjusted, combined, and deleted according to actual needs. The units in the device embodiments of the present invention can be combined, divided, and deleted according to actual needs. In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0196] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0197] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A visual inspection method for reflow soldering quality, characterized in that: include: Acquire an image of the product to be inspected that enters the inspection area; Preprocessing the image of the product to be inspected and extracting the soldering point ROI area; The welding spot ROI area is placed in a defect classification model to identify and classify welding defects to obtain a classification result; An equipment parameter adjustment plan is determined according to the classification result and preset adjustment rules, and the reflow soldering equipment is adjusted according to the equipment parameter adjustment plan.

2. The reflow soldering quality visual inspection method according to claim 1, characterized in that: The image of the product to be inspected is captured by an industrial camera installed in the outlet cooling zone of the reflow soldering equipment, wherein the industrial camera and the annular LED light source are installed in the outlet cooling zone of the reflow soldering equipment in a bilaterally symmetrical layout.

3. The reflow soldering quality visual inspection method according to claim 1, characterized in that: The preprocessing of the image of the product to be inspected and extracting the solder joint ROI area includes: Positioning the reference points of the image of the product to be inspected, and correcting the deviation of the image of the product to be inspected to obtain a corrected image; The solder joint ROI area is extracted from the rectified image through threshold segmentation and morphological processing.

4. The reflow soldering quality visual inspection method according to claim 1, characterized in that: The defect classification model is obtained by training a deep learning model using a number of solder joint ROI areas with labels of different defect types and defect locations as a sample set.

5. The reflow soldering quality visual inspection method according to claim 4, characterized in that: The defect classification model is obtained by training a deep learning model using a number of solder joint ROI areas with labels of different defect types and defect locations as sample sets, including: Collect a data set of several annotated images, where each image includes a solder joint ROI region annotated with defect type and location; Processing the data set to obtain a sample set; Build deep learning models; The deep learning model is optimized using the sample set through transfer learning, focus loss and AdamW optimizer, and FP16 quantization acceleration is performed using TensorRT to obtain a defect classification model.

6. The reflow soldering quality visual inspection method according to claim 5, characterized in that: The method of optimizing the deep learning model by using the sample set through transfer learning, focus loss and AdamW optimizer, and using TensorRT for FP16 quantization acceleration to obtain a defect classification model includes: Transfer learning is used with the sample set, and fine-tuning is performed based on the ImageNet pre-trained weights. Focus loss is used to solve the category imbalance problem. AdamW is used as the optimizer, the learning rate is 3e-4, and the Cosine attenuation strategy is used to learn the deep learning model. The deep learning model is quantized using TensorRT to obtain a trained model, and the trained model is converted into ONNX format to obtain a defect classification model.

7. The reflow soldering quality visual inspection method according to claim 1, characterized in that: The equipment adjustment plan includes adjusting the temperature and chain speed of the reflow soldering temperature control system.

8. The reflow soldering quality visual inspection method according to claim 1, characterized in that: The method of determining a device parameter adjustment scheme according to the classification result and a preset adjustment rule, and adjusting the reflow soldering equipment according to the device parameter adjustment scheme, includes: According to the classification result and preset adjustment rules, an equipment parameter adjustment plan is determined, and the reflow soldering equipment is adjusted according to the equipment parameter adjustment plan, and safety boundary constraints are followed during the adjustment. The safety boundary constraints include: the adjustment range of the temperature in the temperature zone does not exceed ±3°C / min, the chain speed change range does not exceed ±10%, and upper and lower limits are set to avoid excessive chain speed or stoppage.

9. Reflow soldering quality visual inspection device, characterized in that, include: An image acquisition unit, used to acquire an image of a product to be inspected that enters the inspection area; A ROI region extraction unit, used to pre-process the image of the product to be inspected and extract the soldering point ROI region; A classification unit, used for identifying and classifying welding defects by placing the welding spot ROI area into a defect classification model to obtain a classification result; The adjustment unit is used to determine a device parameter adjustment plan according to the classification result and a preset adjustment rule, and adjust the reflow soldering equipment according to the device parameter adjustment plan.

10. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images