A PCB welding defect recognition method and storage medium based on machine vision

Abstract

The present invention discloses a PCB welding defect recognition method and storage medium based on machine vision. The PCB to be tested is detected in a complex dynamic scene to improve the welding defect recognition accuracy, and then the welding defect data of each welding point in the welding area image is obtained through an improved YOLOv5 network, and the size relationship between each welding defect evaluation value obtained based on the welding defect data and a preset welding defect evaluation threshold is judged, so as to classify all welding areas into one of three categories of welding areas. Since the severity of the welding defect influence corresponding to the three categories of welding areas is different in size order, the welding area that should be processed with defect first can be determined, that is, different welding area images are used as base points for analysis, and the PCB does not need to be checked one by one. Not only the PCB welding defect recognition efficiency and accuracy are improved, but also the PCB welding area that should be processed with defect first is prompted, thereby improving the welding defect processing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of oilfield production management, and in particular to a PCB welding defect recognition method based on machine vision, an electronic device, and a computer-readable storage medium. Background Art

[0002] PCB, or printed circuit board, is an important electronic component in the electronics field. In order to ensure the quality compliance of PCB, it is usually necessary to perform welding defect detection on PCB during the production process. One of the current mainstream detection methods is to identify the welding defects of PCB by comparing the PCB welding images. Although it can achieve the purpose of defect identification, the entire process requires the PCB to be checked one by one. Considering that the subsequent processing methods for different defect situations in production may be different, this is not conducive to the effective grasp of the overall defect situation of PCB, which can easily lead to a decrease in the efficiency of welding defect identification, and is likely to affect the efficiency of subsequent processing of identified defects. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art to at least a certain extent. To this end, the present invention proposes a PCB welding defect recognition method and storage medium based on machine vision, which can improve the efficiency of PCB welding defect recognition.

[0004] In a first aspect, an embodiment of the present invention provides a PCB welding defect recognition method based on machine vision, comprising the following steps:

[0005] Step S1, using a plurality of pre-configured camera units to perform real-time multi-directional shooting of the PCB to be tested, so as to obtain video streams of the PCB to be tested at different angles, thereby obtaining target image data of the PCB to be tested according to the video streams, and obtaining a plurality of welding area images of the PCB to be tested according to the target image data;

[0006] Step S2: for each welding area image, input the welding area image into a pre-trained improved YOLOv5 network, so as to obtain welding defect data corresponding to each welding point in the welding area image; for each welding point in the welding area image, determine a welding point evaluation value of the welding point according to the welding defect data corresponding to the welding point;

[0007] Step S3, for each of the welding area images, determining a first welding defect assessment value corresponding to the welding area image according to the welding point evaluations of all the welding points; judging whether the first welding defect assessment value does not exceed a preset welding defect assessment threshold, if so, determining that the welding area corresponding to the welding area image is a first type welding area, otherwise executing step S4;

[0008] Step S4, assigning a test weight to the welding area image, and determining a second welding defect assessment value corresponding to the welding area image according to the first welding defect assessment value corresponding to the welding area image, the test weight, and the acquired real-time associated data of the welding environment of the PCB to be tested;

[0009] Step S5, judging whether the second welding defect assessment value does not exceed a preset welding defect assessment threshold value, if so, determining that the welding area corresponding to the welding area image is a second type welding area, otherwise determining that the welding area corresponding to the welding area image is a third type welding area, wherein the severity of the welding defect impact of each welding area is sorted from high to low, in the following order: third type welding area, second type welding area, first type welding area.

[0010] Optionally, in one embodiment of the present invention, the step in step S1, obtaining multiple welding area images of the PCB to be tested according to the target image data, includes the following steps:

[0011] Step S11, pre-cleaning the target image data to obtain first image data; performing pixel processing on the first image data to obtain second image data; performing threshold segmentation processing on the second image data to obtain third image data;

[0012] Step S12: performing image segmentation processing on the third image data according to preset welding influencing parameters to obtain multiple welding area images of the PCB to be tested, wherein the preset welding influencing parameters include at least one of the following: welding temperature parameters, welding vibration parameters and electromagnetic operation parameters of the PCB to be tested.

[0013] Optionally, in one embodiment of the present invention, the improved YOLOv5 network in step S2 is determined by the following steps:

[0014] The GSconv convolution is improved to obtain the FGSD convolution; the conv convolution of the YOLOv5 network is replaced by the FGSD convolution to obtain the improved YOLOv5 network.

[0015] Optionally, in one embodiment of the present invention, when the welding defect data includes a solder joint redundant quantity value, a solder joint position difference value, and a solder joint solder difference value, the step in step S2, determining the solder joint evaluation value of the solder joint according to the welding defect data corresponding to the solder joint, includes the following steps:

[0016] Step S21, substituting the solder point redundant quantity value, the solder point position difference value and the solder point solder difference value corresponding to the solder point into a first estimation calculation formula for calculation, to obtain the solder point estimation value of the solder point, wherein the first estimation calculation formula is as follows:

[0017] ;

[0018] Estimate the weld point of the weld point, is the reference value of the preset number of welding points. is the redundant number value of the solder joints, is the preset welding point position reference value, is the solder joint position difference value, To preset the solder reference value of the solder joint, is the solder difference value of the solder joint, is the first preset solder joint conversion factor, is the second preset solder joint conversion factor, is the third preset solder joint conversion factor, .

[0019] Optionally, in one embodiment of the present invention, the step in step S3, determining the first welding defect assessment value corresponding to the welding area image according to the weld point evaluation values ​​of all the welding points, comprises the following steps:

[0020] Step S31, calculating the average value of the weld point evaluation values ​​of all the weld points to obtain a weld point mean parameter; for each weld point, calculating the absolute value of the difference between the corresponding weld point evaluation value and the weld point mean parameter;

[0021] Step S32: Screen out all the weld point estimation values ​​whose corresponding absolute values ​​are less than or equal to a preset weld point error threshold, and calculate the sum of the remaining weld point estimation values ​​to obtain a first welding defect assessment value corresponding to the welding area image.

[0022] Optionally, in one embodiment of the present invention, the step in step S4, assigning a test weight to the welding area image, comprises the following steps:

[0023] Step S41, calculating the sum of the first welding defect assessment values ​​corresponding to all target welding area images, thereby obtaining a total defect assessment amount, wherein the target welding area image is the welding area image whose first welding defect assessment value is greater than the preset welding defect assessment threshold;

[0024] Step S42, calculating the quotient of the first welding defect assessment value corresponding to the welding area image and the defect assessment total amount, thereby obtaining a test weight corresponding to the welding area image.

[0025] Optionally, in one embodiment of the present invention, when the welding environment real-time associated data includes regional electromagnetic interference amount, regional temperature change amount and regional pressure change amount, the step in step S4, according to the first welding defect assessment value corresponding to the welding area image, the test weight and the acquired welding environment real-time associated data of the PCB to be tested, determines the second welding defect assessment value corresponding to the welding area image, including the following steps:

[0026] Step S43, substituting the first welding defect assessment value corresponding to the welding area image, the test weight, the regional electromagnetic interference amount, the regional temperature change amount, and the regional pressure change amount into a second estimation calculation formula for calculation, to obtain a second welding defect assessment value corresponding to the welding area image, wherein the second estimation calculation formula is as follows:

[0027] ;

[0028] is the second welding defect evaluation value corresponding to the welding area image, is the test weight corresponding to the welding area image, is the first welding defect evaluation value corresponding to the welding area image, is the preset welding defect assessment threshold, is the temperature change in the region, is the pressure change in the region, is the electromagnetic interference amount in the area, It is the electromagnetic environment control threshold of the preset area.

[0029] In a second aspect, an embodiment of the present invention provides an electronic device, including:

[0030] at least one processor;

[0031] at least one memory for storing at least one program;

[0032] When at least one of the programs is executed by at least one of the processors, the method for identifying PCB welding defects based on machine vision as described in the first aspect is implemented.

[0033] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the PCB welding defect recognition method based on machine vision as described in the first aspect.

[0034] The present invention proposes a PCB welding defect recognition method and storage medium based on machine vision. The PCB to be tested is detected in a complex dynamic scene to improve the PCB welding defect recognition accuracy, thereby effectively obtaining multiple welding area images of the PCB to be tested, and then obtaining welding defect data corresponding to each welding point in the welding area image through a pre-trained improved YOLOv5 network. On this basis, by judging the size relationship between the first welding defect evaluation value and the second welding defect evaluation value calculated based on the welding defect data and the preset welding defect evaluation threshold, each welding area corresponding to the welding area image is divided into a first type of welding area, a second type of welding area or a third type of welding area. Since the severity of the welding defects corresponding to the three types of welding areas is different in size order, it can be determined which welding areas should be given priority for defect processing. That is to say, the entire welding defect recognition process is analyzed with different welding area images as the base point, and there is no need to check the PCB one by one. This can not only effectively improve the efficiency and accuracy of PCB welding defect recognition, but also can prompt the staff to the PCB welding area that should be given priority for defect processing, which is conducive to improving the efficiency of processing PCB welding defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a PCB welding defect recognition method based on machine vision provided by an embodiment of the present invention;

[0036] Figure 2 1 is a schematic diagram of the structure of an improved YOLOv5 network provided by an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of the FGSD convolution structure provided by an embodiment of the present invention;

[0038] Figure 4 yes Figure 1 A partial flow chart of the step S1 of "obtaining multiple welding area images of the PCB to be tested according to the target image data";

[0039] Figure 5 yes Figure 1 A partial flow chart of the step "determining the weld point evaluation value of the weld point according to the welding defect data corresponding to the weld point" in step S2;

[0040] Figure 6 yes Figure 1 A partial flow chart of the step "determining a first welding defect evaluation value corresponding to the welding area image according to the weld evaluation evaluation values ​​of all welding points" in step S3 of FIG.

[0041] Figure 7 yes Figure 1 A partial flow chart of the step "assigning test weights to welding area images" in step S4;

[0042] Figure 8 yes Figure 1 A partial flow chart of the step S4 of “determining a second welding defect assessment value corresponding to the welding area image according to the first welding defect assessment value corresponding to the welding area image, the test weight, and the acquired real-time associated data of the welding environment of the PCB to be tested”;

[0043] Fig. 9 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] like Figure 1 As shown, an embodiment of the present invention provides a PCB welding defect recognition method based on machine vision, which may specifically include but is not limited to steps S1 to S5.

[0045] Step S1, real-time multi-directional shooting of the PCB to be tested is performed by multiple pre-configured camera units to obtain video streams of different angles of the PCB to be tested, thereby obtaining target image data of the PCB to be tested according to the video stream, and obtaining multiple welding area images of the PCB to be tested according to the target image data. That is to say, by testing the PCB to be tested in a complex dynamic scene, it is possible to solve the problem that single-angle shooting often cannot fully capture PCB surface information, especially at complex welding points or PCB areas that are difficult to directly observe; wherein the camera unit can be of various types, which are not limited here, for example, a high-resolution camera or a webcam with a self-configurable resolution, and the multi-directional shooting can be achieved by, but not limited to, a rotatable pan-tilt platform, and multiple pre-configured camera units are installed on the rotatable pan-tilt platform to obtain video streams of different angles of the PCB to be tested;

[0046] Step S2: for each welding area image, input the welding area image into the pre-trained improved YOLOv5 network, so as to obtain welding defect data corresponding to each welding point in the welding area image; for each welding point in the welding area image, determine the welding point evaluation value of the welding point according to the welding defect data corresponding to the welding point;

[0047] Step S3, for each welding area image, determine a first welding defect evaluation value corresponding to the welding area image according to the welding point evaluation values ​​of all welding points; determine whether the first welding defect evaluation value does not exceed a preset welding defect evaluation threshold value, if so, determine that the welding area corresponding to the welding area image is a first type of welding area, otherwise execute step S4, wherein the preset welding defect evaluation threshold value can be set accordingly according to the actual application scenario, and is not limited here;

[0048] Step S4, assigning a test weight to the welding area image, and determining a second welding defect assessment value corresponding to the welding area image according to the first welding defect assessment value corresponding to the welding area image, the test weight, and the acquired real-time associated data of the welding environment of the PCB to be tested;

[0049] Step S5, judging whether the second welding defect evaluation value does not exceed the preset welding defect evaluation threshold, if so, determining that the welding area corresponding to the welding area image is the second type welding area, otherwise determining that the welding area corresponding to the welding area image is the third type welding area, wherein the severity of the welding defect impact of each welding area is sorted from high to low, in the following order: the third type welding area, the second type welding area, the first type welding area, it can be understood that the first type welding area, the second type welding area and the third type welding area can specifically present different welding defect impact situations in different application scenarios, which is related to the values ​​of the above-mentioned related calculation parameters specifically obtained, and is not limited here. For example, in some cases, the welding defect impact situation presented by the third type welding area is mainly an open circuit defect, and at the same time there are a small number of short circuit defects; the welding defect impact situation presented by the second type welding area is mainly a missing hole defect, and at the same time there are a small number of stray defects; the welding defect impact situation presented by the third type welding area is mainly a mouse bite defect, and at the same time there are a small number of pseudo copper defects, etc.

[0050] In this step, the PCB to be tested is detected in a complex dynamic scene to improve the PCB welding defect recognition accuracy, so as to effectively obtain multiple welding area images of the PCB to be tested, and then the welding defect data corresponding to each welding point in the welding area image is obtained through the pre-trained improved YOLOv5 network. On this basis, by judging the size relationship between the first welding defect evaluation value and the second welding defect evaluation value calculated based on the welding defect data and the preset welding defect evaluation threshold, each welding area corresponding to the welding area image is divided into a first type of welding area, a second type of welding area or a third type of welding area. Since the severity of the welding defects corresponding to these three types of welding areas is different in size order, it can be determined which welding areas should be given priority for defect processing. That is to say, the entire welding defect recognition process is analyzed with different welding area images as the base point, and there is no need to check the PCB one by one. This can not only effectively improve the PCB welding defect recognition, but also effectively improve the PCB welding defect recognition accuracy. It can improve the efficiency and accuracy of identification, and can also prompt the staff of the PCB welding area that should be given priority for defect treatment, which is conducive to improving the efficiency of processing PCB welding defects. It should be noted that when the first welding defect evaluation value has exceeded the preset welding defect evaluation threshold, not only the first welding defect evaluation value corresponding to the welding area image and the test weight are considered for the evaluation of welding defects, but also the real-time associated data of the welding environment of the PCB to be tested is considered for the evaluation of welding defects. This is also to match the situation where the first welding defect evaluation value has exceeded the preset welding defect evaluation threshold. That is to say, in this case, the severity of the welding defect in the welding area is more obvious, so a more refined welding defect evaluation is required, so the real-time associated data of the welding environment of the PCB to be tested is introduced for further calculation, which is also another important invention point of this embodiment, which ensures that the obtained welding defect evaluation result will be more accurate, which is conducive to further improving the PCB welding defect identification accuracy.

[0051] In one embodiment, there are multiple specific ways to obtain the target image data of the PCB to be tested according to the video stream in step S1, which can be selected and obtained according to the actual application scenario, and there is no limitation here. For example, it can be but not limited to capturing pixels of the acquired video stream to obtain an associated image space, and then performing feature representation on the pixels in the image space by a feature space clustering method or a clustering neural network to obtain feature space points, and then mapping the feature space points to the image space to obtain associated data segments, and finally performing image information extraction, threshold division, image content screening and other operations on the associated data segments in turn to obtain the target image data of the PCB to be tested.

[0052] In one embodiment, the welding area image is input into the pre-trained improved YOLOv5 network in step S2 to obtain welding defect data corresponding to each welding point in the welding area image. There may be multiple ways, wherein the improved YOLOv5 network in step S2 may be determined by, but not limited to, the following steps: improving the GSconv convolution to obtain the FGSD convolution; replacing the conv convolution of the YOLOv5 network by the FGSD convolution to obtain the improved YOLOv5 network; there may be multiple specific ways to improve the GSconv convolution, for example, a FGSD convolution structure may be, but not limited to, as follows Figure 3 As shown, specifically, Figure 2 As shown in the figure, the conv convolution of the YOLOv5 network (including the input end, Backbone part, Neck part and Prediction part from left to right) is replaced by FGSD convolution. It is mainly reflected in the convolution structure related to the input end, that is, the FGSD convolution is used to replace some convolutions in the Backbone part of the YOLOv5 network, thereby realizing the replacement and update of the conv convolution of the YOLOv5 network. Figure 2 The specific structure of the C3 module in the conv convolution is also shown in the lower left corner. It can be seen that compared with the traditional YOLOv5 network, the improved YOLOv5 network has better accuracy and robustness. Based on efficient feature extraction and target recognition strategies, the processing speed of the recognition algorithm is further optimized while ensuring high accuracy. The training method of the improved YOLOv5 network can be, but is not limited to: first, the multiple welding area images corresponding to the acquired historical welding detection time period are standardized, and then each of the standardized welding area images is respectively input into the input end of the constructed improved YOLOv5 network, and finally the Prediction part of the improved YOLOv5 network outputs the trained output value, which is the welding defect data corresponding to each welding point. In the specific application process, the improved YOLOv5 network identifies each welding point in the welding area image from the welding area image by feature recognition, and then identifies the welding defect data corresponding to each welding point by the updated conv convolution, and finally obtains the welding defect data corresponding to each welding point in the welding area image.

[0053] In one embodiment of the present invention, in step S1, multiple welding area images of the PCB to be tested are obtained according to the target image data, such as Figure 4 As shown, the steps may specifically include but are not limited to:

[0054] Step S11, pre-cleaning the target image data to obtain first image data; performing pixel processing on the first image data to obtain second image data; performing threshold segmentation processing on the second image data to obtain third image data;

[0055] Step S12: performing image segmentation processing on the third image data according to preset welding influencing parameters to obtain multiple welding area images of the PCB to be tested, wherein the preset welding influencing parameters include at least one of the following: welding temperature parameters, welding vibration parameters and electromagnetic operation parameters of the PCB to be tested.

[0056] In this step, by performing pre-cleaning processing, pixel processing and threshold segmentation processing on the target image data, the required third image data can be obtained, and then the third image data is subjected to image segmentation processing according to the preset welding influence parameters, so that multiple welding area images of the PCB to be tested can be obtained. In the whole process, the pre-cleaning processing can be but not limited to normalization processing, brightness contrast processing, etc., the pixel processing can be but not limited to grayscale processing, pixel point extraction, etc., and the purpose of the threshold segmentation processing is to separate the welding area from the second image data; the welding temperature parameter represents the temperature influence of the surrounding environment or / and the welding tool of the PCB to be tested under the test condition, the welding vibration parameter represents the PCB vibration of the PCB to be tested under the test condition, and the electromagnetic operation parameter represents the electromagnetic influence of the PCB to be tested under the test condition.

[0057] It should be noted that there may be more preset welding influencing parameters, which need to be determined accordingly according to the actual application scenario, and are not limited here; the means of performing image segmentation processing on the third image data according to the preset welding influencing parameters, taking the welding temperature parameters as an example, may be but not limited to: real-time acquisition of different welding temperature values ​​of the PCB to be tested under the test conditions, the welding temperature value may be affected by the ambient temperature, and may also be affected by the welding device, and the third image data may be segmented according to the image areas corresponding to the different welding temperature values ​​obtained, so that multiple welding area images of the PCB to be tested can be obtained, and so on, the welding vibration parameters and the electromagnetic operation parameters can be similarly processed, which will not be elaborated here.

[0058] like Figure 5 As shown, in one embodiment of the present invention, when the welding defect data includes a redundant number value of a welding point, a position difference value of a welding point, and a solder difference value of a welding point, the steps in step S2, determining the welding point evaluation value of the welding point according to the welding defect data corresponding to the welding point, may include but are not limited to the following steps:

[0059] Step S21, substitute the solder point redundant quantity value, solder point position difference value and solder point solder difference value corresponding to the solder point into the first estimation calculation formula for calculation to obtain the solder point evaluation value of the solder point. The first estimation calculation formula is as follows:

[0060] ;

[0061] in, To estimate the weld point of the weld, is the reference value of the preset number of welding points. is the redundant number of solder joints, is the preset welding point position reference value, is the solder joint position difference value, To preset the solder reference value of the solder joint, is the solder difference value of the solder joint, is the first preset solder joint conversion factor, is the second preset solder joint conversion factor, is the third preset solder joint conversion factor, .

[0062] In this step, Characterize the redundant solder joints that appear during welding. Characterizes the welding position misalignment of the welding point during welding. Characterize the solder redundancy or shortage of the welding point during welding. It can be seen that the solder redundancy quantity value, the solder position difference value and the solder difference value of the solder point can well characterize the real-time welding situation of the welding point. Therefore, it is more appropriate to use the solder redundancy quantity value, the solder position difference value and the solder difference value of the solder point for welding evaluation, so that the solder redundancy quantity value, the solder position difference value and the solder difference value of the solder point corresponding to the welding point can be substituted into the first estimation calculation formula for calculation to obtain the solder point evaluation value of the welding point.

[0063] It should be noted that in addition to the three related parameters of welding defect data given above, there may be more types of welding defect data, which need to be configured accordingly according to the actual application scenario. For example, it may also include but is not limited to weld spot area difference values, weld spot depth difference values, etc., which are not limited here.

[0064] like Figure 6 As shown, in one embodiment of the present invention, the step in step S3, determining the first welding defect evaluation value corresponding to the welding area image according to the weld point evaluation of all welding points, may include but is not limited to the following steps:

[0065] Step S31, calculating the average value of the weld point evaluations of all weld points to obtain the weld point mean parameter; for each weld point, calculating the absolute value of the difference between the corresponding weld point evaluation and the weld point mean parameter;

[0066] Step S32: Screen out all weld point estimation values ​​whose corresponding absolute values ​​are less than or equal to a preset weld point error threshold, and calculate the sum of the remaining weld point estimation values ​​to obtain a first welding defect assessment value corresponding to the welding area image.

[0067] In this step, the weld point mean parameter is obtained by calculating the average value of the weld point evaluations of all weld points, and the weld point mean parameter reflects the average degree of the weld point evaluations of all weld points, and then the absolute value of the difference between the corresponding weld point evaluation and the weld point mean parameter is calculated for each weld point, so as to reflect the degree of difference between each weld point evaluation and the weld point mean parameter, and then evaluate whether the degree of difference is allowed, that is, if the absolute value corresponding to the weld point evaluation is less than or equal to the preset weld point error threshold, it means that the degree of difference corresponding to the weld point evaluation is not allowed, so it needs to be screened out, otherwise it can be retained, so that all weld point evaluations that meet the requirements can be obtained, and finally by calculating the sum of the remaining weld point evaluations, the first welding defect evaluation value corresponding to the required welding area image can be obtained.

[0068] like Figure 7 As shown, in one embodiment of the present invention, the step in step S4, assigning a test weight to the welding area image, may include but is not limited to the following steps:

[0069] Step S41, calculating the sum of the first welding defect assessment values ​​corresponding to all target welding area images, thereby obtaining a total defect assessment amount, wherein the target welding area image is a welding area image whose first welding defect assessment value is greater than a preset welding defect assessment threshold;

[0070] Step S42, calculating the quotient of the first welding defect assessment value corresponding to the welding area image and the defect assessment total amount, thereby obtaining a test weight corresponding to the welding area image.

[0071] In this step, the total defect assessment amount is obtained by calculating the sum of the first welding defect assessment values ​​corresponding to all target welding area images. Since the target welding area image is a welding area image in which the first welding defect assessment value is greater than the preset welding defect assessment threshold, that is to say, the calculated total defect assessment amount corresponds to the application scenario in which the first welding defect assessment value is greater than the preset welding defect assessment threshold, which reflects the defect assessment situation in this scenario. Then, by calculating the quotient of the first welding defect assessment value corresponding to the welding area image and the total defect assessment amount, the test weight corresponding to the welding area image can be obtained.

[0072] like Figure 8As shown, in one embodiment of the present invention, when the real-time associated data of the welding environment includes the regional electromagnetic interference amount, the regional temperature change amount and the regional pressure change amount, the step in step S4, according to the first welding defect assessment value corresponding to the welding area image, the test weight and the acquired real-time associated data of the welding environment of the PCB to be tested, determines the second welding defect assessment value corresponding to the welding area image, which may include but is not limited to the following steps:

[0073] Step S43: Substitute the first welding defect evaluation value, test weight, regional electromagnetic interference amount, regional temperature change amount, and regional pressure change amount corresponding to the welding area image into the second estimation calculation formula for calculation to obtain the second welding defect evaluation value corresponding to the welding area image, wherein the second estimation calculation formula is as follows:

[0074] ;

[0075] is the second welding defect evaluation value corresponding to the welding area image, is the test weight corresponding to the welding area image, is the first welding defect evaluation value corresponding to the welding area image, To preset the welding defect assessment threshold, is the regional temperature change, is the regional pressure change, is the regional electromagnetic interference, It is the electromagnetic environment control threshold of the preset area.

[0076] In this step, the regional electromagnetic interference amount reflects the real-time electromagnetic interference influence of the PCB to be tested under the welding condition, the regional temperature change amount reflects the influence of the uncertain temperature change of the PCB to be tested under the welding condition, and the regional pressure change amount reflects the influence of the uncertain pressure change of the PCB to be tested under the welding condition. It can be seen that the regional electromagnetic interference amount, the regional temperature change amount and the regional pressure change amount can well characterize the real-time situation of the welding environment of the PCB to be tested, so as to take the real-time situation of this part of the welding environment into consideration, and substitute the first welding defect assessment value corresponding to the welding area image, the test weight, and the regional electromagnetic interference amount, the regional temperature change amount and the regional pressure change amount into the second estimation calculation formula for calculation, so as to accurately and reliably calculate the second welding defect assessment value corresponding to the welding area image.

[0077] It should be noted that the real-time associated data of the welding environment may also include more types of data, and its specific parameters and corresponding values ​​need to be set and obtained according to different application scenarios, which are not limited here.

[0078] Fig. 9FIG. 1 is a schematic diagram of the structure of an electronic device 1000 provided by an embodiment of the present invention. Fig. 9 As shown, the electronic device 1000 includes a memory 1100 and a processor 1200. The number of the memory 1100 and the processor 1200 may be one or more. Fig. 9 In the embodiment, a memory 1100 and a processor 1200 are taken as an example; the memory 1100 and the processor 1200 in the device may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0079] The memory 1100, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the PCB welding defect recognition method based on machine vision provided in any embodiment of the present invention. The processor 1200 implements the above-mentioned PCB welding defect recognition method based on machine vision by running the software programs, instructions and modules stored in the memory 1100.

[0080] The memory 1100 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function. In addition, the memory 1100 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1100 may further include a memory remotely arranged relative to the processor 1200, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0081] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the machine vision-based PCB welding defect recognition method provided in any embodiment of the present invention.

[0082] An embodiment of the present invention further provides a computer program product, including a computer program or computer instructions, wherein the computer program or computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device executes a PCB welding defect identification method based on machine vision as provided in any embodiment of the present invention.

[0083] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0084] In hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0085] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process or an execution thread, and a component may be located on a computer or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may communicate, for example, through local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems through signals).

Claims

1. A PCB welding defect recognition method based on machine vision, characterized in that: The steps include: Step S1, using a plurality of pre-configured camera units to perform real-time multi-directional shooting of the PCB to be tested, so as to obtain video streams of the PCB to be tested at different angles, thereby obtaining target image data of the PCB to be tested according to the video streams, and obtaining a plurality of welding area images of the PCB to be tested according to the target image data; Step S2: for each welding area image, input the welding area image into a pre-trained improved YOLOv5 network, so as to obtain welding defect data corresponding to each welding point in the welding area image; for each welding point in the welding area image, determine a welding point evaluation value of the welding point according to the welding defect data corresponding to the welding point; Step S3, for each of the welding area images, determining a first welding defect assessment value corresponding to the welding area image according to the welding point evaluations of all the welding points; judging whether the first welding defect assessment value does not exceed a preset welding defect assessment threshold, if so, determining that the welding area corresponding to the welding area image is a first type welding area, otherwise executing step S4; Step S4, assigning a test weight to the welding area image, and determining a second welding defect assessment value corresponding to the welding area image according to the first welding defect assessment value corresponding to the welding area image, the test weight, and the acquired real-time associated data of the welding environment of the PCB to be tested; Step S5, judging whether the second welding defect assessment value does not exceed a preset welding defect assessment threshold value, if so, determining that the welding area corresponding to the welding area image is a second type welding area, otherwise determining that the welding area corresponding to the welding area image is a third type welding area, wherein the severity of the welding defect impact of each welding area is sorted from high to low, in the order of: third type welding area, second type welding area, first type welding area; Wherein, when the welding defect data includes a redundant number value of a welding point, a position difference value of a welding point, and a solder difference value of a welding point, the step in step S2, determining the welding point evaluation value of the welding point according to the welding defect data corresponding to the welding point, includes the following steps: Step S21, substituting the solder point redundant quantity value, the solder point position difference value and the solder point solder difference value corresponding to the solder point into a first estimation calculation formula for calculation, to obtain the solder point estimation value of the solder point, wherein the first estimation calculation formula is as follows: W=α|S0-S|+β|T0-T|+γ|Q0-Q|; W is the solder joint evaluation value of the solder joint, S0 is the preset solder joint quantity reference value, S is the solder joint redundant quantity value, T0 is the preset solder joint position reference value, T is the solder joint position difference value, Q0 is the preset solder joint solder reference value, Q is the solder joint solder difference value, α is the first preset solder joint conversion factor, β is the second preset solder joint conversion factor, γ is the third preset solder joint conversion factor, α+β+γ=1.

2. The PCB welding defect recognition method based on machine vision according to claim 1 is characterized in that: The step in step S1, obtaining multiple welding area images of the PCB to be tested according to the target image data, includes the following steps: Step S11, pre-cleaning the target image data to obtain first image data; performing pixel processing on the first image data to obtain second image data; Performing threshold segmentation processing on the second image data to obtain third image data; Step S12: performing image segmentation processing on the third image data according to preset welding influencing parameters to obtain multiple welding area images of the PCB to be tested, wherein the preset welding influencing parameters include at least one of the following: welding temperature parameters, welding vibration parameters and electromagnetic operation parameters of the PCB to be tested.

3. The PCB welding defect recognition method based on machine vision according to claim 1 is characterized in that: The improved YOLOv5 network in step S2 is determined by the following steps: The GSconv convolution is improved to obtain the FGSD convolution; the conv convolution of the YOLOv5 network is replaced by the FGSD convolution to obtain the improved YOLOv5 network.

4. The PCB welding defect recognition method based on machine vision according to claim 1 is characterized in that: The step in step S3, determining the first welding defect evaluation value corresponding to the welding area image according to the weld evaluation values ​​of all the welding points, comprises the following steps: Step S31, calculating the average value of the weld point evaluation values ​​of all the weld points to obtain a weld point mean parameter; for each weld point, calculating the absolute value of the difference between the corresponding weld point evaluation value and the weld point mean parameter; Step S32: Screen out all the weld point estimation values ​​whose corresponding absolute values ​​are less than or equal to a preset weld point error threshold, and calculate the sum of the remaining weld point estimation values ​​to obtain a first welding defect assessment value corresponding to the welding area image.

5. The PCB welding defect recognition method based on machine vision according to claim 1 is characterized in that: The step in step S4, assigning a test weight to the welding area image, comprises the following steps: Step S41, calculating the sum of the first welding defect assessment values ​​corresponding to all target welding area images, thereby obtaining a total defect assessment amount, wherein the target welding area image is the welding area image whose first welding defect assessment value is greater than the preset welding defect assessment threshold; Step S42, calculating the quotient of the first welding defect assessment value corresponding to the welding area image and the defect assessment total amount, thereby obtaining a test weight corresponding to the welding area image.

6. The method for identifying PCB welding defects based on machine vision according to claim 1 or 5, characterized in that: When the welding environment real-time associated data includes regional electromagnetic interference amount, regional temperature change amount and regional pressure change amount, the step in step S4, according to the first welding defect assessment value corresponding to the welding area image, the test weight and the acquired welding environment real-time associated data of the PCB to be tested, determines the second welding defect assessment value corresponding to the welding area image, including the following steps: Step S43, substituting the first welding defect assessment value corresponding to the welding area image, the test weight, the regional electromagnetic interference amount, the regional temperature change amount, and the regional pressure change amount into a second estimation calculation formula for calculation, to obtain a second welding defect assessment value corresponding to the welding area image, wherein the second estimation calculation formula is as follows: M is the second welding defect assessment value corresponding to the welding area image, μ is the test weight corresponding to the welding area image, P is the first welding defect assessment value corresponding to the welding area image, P0 is the preset welding defect assessment threshold, I0 is the temperature change in the area, F0 is the pressure change in the area, H0 is the electromagnetic interference amount in the area, and H is the preset area electromagnetic environment control threshold.

7. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the machine vision-based PCB welding defect recognition method as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that: A program executable by a processor is stored therein, and when the program executable by the processor is executed by the processor, it is used to implement the PCB welding defect recognition method based on machine vision as described in any one of claims 1 to 6.

Images