Component placement quality monitoring system and method based on SMT placement machine

Abstract

The present disclosure is about a component mounting quality monitoring system and method based on an SMT placement machine. The method includes: obtaining a component mounting area image of a circuit board to be tested, obtaining the area position coordinates of each solder joint in the component mounting area image, generating a bounding box corresponding to the area position coordinate according to a preset rule for each area position coordinate, dividing the component mounting area image according to the bounding box to obtain a plurality of sub-images, obtaining the grayscale value of each pixel point in the sub-image for each of the plurality of sub-images, identifying the reflective points in the sub-image using the grayscale value, removing the reflective points to obtain a sub-image after removing the reflection, inputting the sub-image after removing the reflection into a pre-trained solder joint defect recognition model, and obtaining a defect recognition result output by the solder joint defect recognition model after identifying the obtained sub-image. This solution avoids the interference of reflection on the component mounting quality monitoring.

Description

Technical Field

[0001] The present disclosure relates to the technical field of component mounting, and in particular to a component mounting quality monitoring system and method based on an SMT mounting machine. Background Art

[0002] In the related technology, the SMT (Surface Mounted Technology) placement machine accurately places the surface mount components on the PCB pads by moving the placement head, and then performs soldering through reflow soldering. Circuit soldering defects may occur during the soldering process, so it is necessary to detect whether there are soldering defects by acquiring component mounting images. However, since soldering is usually performed using solder paste, its metal properties cause reflections at the solder joints, resulting in reflections in the acquired component mounting images, which interferes with the identification of soldering defects and affects the accuracy of soldering defect identification. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a component mounting quality monitoring system and method based on an SMT placement machine.

[0004] According to a first aspect of an embodiment of the present disclosure, a component mounting quality monitoring system based on an SMT placement machine is provided, comprising:

[0005] A first acquisition module is used to acquire an image of a component mounting area of ​​a circuit board to be tested;

[0006] A second acquisition module is used to acquire the regional position coordinates of each solder point in the component mounting area image;

[0007] A segmentation module, for generating a bounding box corresponding to each area position coordinate according to a preset rule, and segmenting the component mounting area image according to the bounding box to obtain a plurality of sub-images;

[0008] A first recognition module is used to obtain, for each sub-image of the multiple sub-images, a grayscale value of each pixel in the sub-image, and identify a reflective point in the sub-image using the grayscale value; the reflective point is a reflective point on a welding point in the sub-image;

[0009] A removal module, used for removing the reflective points to obtain a sub-image after removing the reflective points;

[0010] The second recognition module is used to input the sub-image after removing the reflection into the pre-trained solder joint defect recognition model to obtain the defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image.

[0011] In some embodiments of the present disclosure, the segmentation module includes:

[0012] The first acquisition submodule is used to acquire the size information of the standard bounding box; the standard bounding box is generated by using the solder joint image of the standard solder joint;

[0013] The first generating submodule is used to generate a bounding box corresponding to the region position coordinates according to the size information; the region position coordinates fall into the bounding box; the size of the bounding box corresponds to the size information.

[0014] In some embodiments of the present disclosure, the first identification module includes:

[0015] A second acquisition submodule is used to acquire the gray value of each pixel point in the position coordinates of the welding point area in the sub-image to obtain a gray value set;

[0016] A selection submodule is used to select, for each column of pixels in the sub-image, a target pixel whose grayscale value is a sudden grayscale value, and the position coordinates of the target pixel;

[0017] A second generating submodule, used for generating a feature map to be identified based on the target pixel point and the position coordinates of the target pixel point;

[0018] The recognition submodule is used to input the feature map to be recognized into a pre-trained convolutional neural network model to obtain a boundary line of reflective points output by the convolutional neural network model based on the feature map to be recognized;

[0019] The determination submodule is used to determine the regional position coordinates of the reflective point based on the reflective point boundary line.

[0020] In some embodiments of the present disclosure, the system further includes:

[0021] A calculation module, configured to calculate an average value of all gray values ​​within the position coordinates of the solder joint area when it is determined that there is no sudden gray value in the gray value set;

[0022] An adding module, used for adding the average value and a preset grayscale adjustment value to obtain a first threshold value;

[0023] A selection module, configured to select a target grayscale value greater than or equal to the first threshold from the grayscale value set;

[0024] The first determination module is used to determine the regional position coordinates of the reflective point based on the pixel points corresponding to the target grayscale value.

[0025] In some embodiments of the present disclosure, the removal module includes:

[0026] A third acquisition submodule is used to acquire other grayscale values ​​in the grayscale value set except the sudden grayscale value;

[0027] A calculation submodule, used for calculating the average value of the other grayscale values;

[0028] The assignment submodule is used to assign the grayscale values ​​of all pixels within the regional position coordinates of the reflective point to the average value to obtain the sub-image after removing the reflection.

[0029] In some embodiments of the present disclosure, there are multiple component mounting area images of the circuit board to be tested, and each component mounting area image is shot at a different angle; the second acquisition module includes:

[0030] A fourth acquisition submodule is used to acquire the regional position coordinates of each solder point in each component mounting area image of the plurality of component mounting area images;

[0031] The system further comprises:

[0032] A second determination module is used to determine, for each solder joint in the circuit board to be tested, a plurality of sub-images corresponding to the solder joint by using a relative position relationship between a sub-image and an image of a corresponding component mounting area; the plurality of sub-images includes an image of the solder joint;

[0033] The third determination module is used to determine whether the welding spot has a welding defect when the defect recognition result of at least one sub-image among the multiple sub-images corresponding to the welding spot is that there is a defect.

[0034] According to a second aspect of an embodiment of the present disclosure, a component mounting quality monitoring method based on an SMT placement machine is provided, comprising:

[0035] Acquire a component mounting area image of the circuit board to be tested;

[0036] Obtaining the regional position coordinates of each solder point in the component mounting area image;

[0037] For each area position coordinate, generating a bounding box corresponding to the area position coordinate according to a preset rule, and dividing the component mounting area image according to the bounding box to obtain a plurality of sub-images;

[0038] For each sub-image of the multiple sub-images, a grayscale value of each pixel in the sub-image is obtained, and a reflective point in the sub-image is identified by using the grayscale value; the reflective point is a reflective point on a welding point in the sub-image;

[0039] Removing the reflection points to obtain a sub-image after the reflections are removed;

[0040] The sub-image after removing the reflection is input into a pre-trained solder joint defect recognition model to obtain a defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image.

[0041] In some embodiments of the present disclosure, generating a bounding box corresponding to the region position coordinates according to a preset rule includes:

[0042] Acquire size information of a standard bounding box; the standard bounding box is generated using a solder joint image of a standard solder joint;

[0043] According to the size information, a bounding box corresponding to the region position coordinates is generated; the region position coordinates fall into the bounding box; and the size of the bounding box corresponds to the size information.

[0044] In some embodiments of the present disclosure, obtaining the grayscale value of each pixel in the sub-image and identifying the reflective point in the sub-image by using the grayscale value includes:

[0045] Obtaining the grayscale value of each pixel point in the position coordinates of the welding point area in the sub-image to obtain a grayscale value set;

[0046] For each column of pixels in the sub-image, a target pixel whose grayscale value is a sudden grayscale value and the position coordinates of the target pixel are selected;

[0047] Generate a feature map to be identified based on the target pixel point and the position coordinates of the target pixel point;

[0048] Inputting the feature map to be identified into a pre-trained convolutional neural network model to obtain a boundary line of reflective points output by the convolutional neural network model based on the feature map to be identified;

[0049] The regional position coordinates of the reflective point are determined based on the reflective point boundary line.

[0050] In some embodiments of the present disclosure, after obtaining the grayscale value of each pixel point in the welding spot area position coordinates in the sub-image to obtain the grayscale value set, the method further includes:

[0051] When it is determined that there is no sudden grayscale value in the grayscale value set, calculating the average value of all grayscale values ​​in the position coordinates of the soldering point area;

[0052] Adding the average value to a preset grayscale adjustment value to obtain a first threshold value;

[0053] Selecting a target grayscale value greater than or equal to the first threshold from the grayscale value set;

[0054] The regional position coordinates of the reflective point are determined based on the pixel points corresponding to the target grayscale value.

[0055] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by acquiring a component mounting area image of a circuit board to be tested, acquiring the area position coordinates of each solder joint in the component mounting area image, for each area position coordinate, generating a bounding box corresponding to the area position coordinate according to a preset rule, dividing the component mounting area image according to the bounding box to obtain a plurality of sub-images, for each of the plurality of sub-images, acquiring the grayscale value of each pixel in the sub-image, identifying the reflective points in the sub-image using the grayscale value, removing the reflective points to obtain a sub-image after removing the reflection, inputting the sub-image after removing the reflection into a pre-trained solder joint defect recognition model, and obtaining a defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image. By using the bounding box method to segment the solder joint image, the situation where the sub-image segmentation error is caused by reflection or defects in the solder joint itself is effectively avoided, that is, the segmented sub-image does not contain the complete solder joint image to be identified, thereby improving the accuracy of identifying solder joint defects. In addition, the reflective points in the sub-image are accurately identified by grayscale value, and the reflective points are removed before defect identification, avoiding the interference of reflection on the monitoring results and improving the accuracy of component mounting quality monitoring.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0058] Figure 1 It is a structural schematic diagram of a component mounting quality monitoring system based on an SMT placement machine according to an exemplary embodiment.

[0059] Figure 2 The present invention is a flow chart of a component mounting quality monitoring method based on an SMT placement machine according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0061] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present disclosure. The singular forms "a", "an" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0062] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the disclosed embodiments, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the disclosed embodiments, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at" or "when" or "in response to determination".

[0063] In addition, various forms of processes shown in the embodiments of the present disclosure may be used to reorder, add or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and this document does not limit this.

[0064] In the related art, the SMT placement machine accurately places the surface mount components on the PCB pads by moving the placement head, and then performs soldering through reflow soldering. Circuit soldering defects may occur during the soldering process, so it is necessary to detect whether there are soldering defects by acquiring component placement images. However, since solder paste is usually used for soldering, its metal properties cause reflections in the solder joints, resulting in reflections in the acquired component placement images, which interferes with the identification of soldering defects and affects the accuracy of soldering defect identification.

[0065] In order to solve the above problems, the present disclosure provides a component mounting quality monitoring system and method based on an SMT placement machine, by acquiring a component mounting area image of a circuit board to be tested, acquiring the area position coordinates of each solder joint in the component mounting area image, for each area position coordinate, generating a bounding box corresponding to the area position coordinate according to a preset rule, dividing the component mounting area image according to the bounding box to obtain a plurality of sub-images, for each of the plurality of sub-images, acquiring the grayscale value of each pixel in the sub-image, identifying the reflective points in the sub-image using the grayscale value, removing the reflective points to obtain a sub-image after removing the reflection, inputting the sub-image after removing the reflection into a pre-trained solder joint defect recognition model, and obtaining a defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image. By using the bounding box method to segment the solder joint image, the situation where the sub-image segmentation error is caused by reflection or defects in the solder joint itself is effectively avoided, that is, the segmented sub-image does not contain the complete solder joint image to be identified, thereby improving the accuracy of identifying solder joint defects. In addition, the reflective points in the sub-image are accurately identified by grayscale value, and the reflective points are removed before defect identification, avoiding the interference of reflection on the monitoring results and improving the accuracy of component mounting quality monitoring.

[0066] Figure 1 is a structural block diagram of a component placement quality monitoring system based on an SMT placement machine according to an exemplary embodiment. Figure 1 As shown, it should be noted that the component mounting quality monitoring system based on the SMT placement machine of the embodiment of the present disclosure is applied to the component mounting quality monitoring method based on the SMT placement machine. Figure 1 As shown, the system may include a first acquisition module 101 , a second acquisition module 102 , a segmentation module 103 , a first recognition module 104 , a removal module 105 and a second recognition module 106 .

[0067] Wherein, the first acquisition module 101 is used to acquire the component mounting area image of the circuit board to be tested;

[0068] The second acquisition module 102 is used to acquire the regional position coordinates of each solder point in the component mounting area image;

[0069] The segmentation module 103 is used to generate a bounding box corresponding to each area position coordinate according to a preset rule, and segment the component mounting area image according to the bounding box to obtain multiple sub-images;

[0070] The first recognition module 104 is used to obtain the gray value of each pixel in each sub-image of the plurality of sub-images, and recognize the reflective point in the sub-image by using the gray value; the reflective point is the reflective point on the welding point in the sub-image;

[0071] A removal module 105 is used to remove the reflection points to obtain a sub-image after the reflections are removed;

[0072] The second recognition module 106 is used to input the sub-image after removing the reflection into the pre-trained solder joint defect recognition model to obtain the defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image.

[0073] As an example, the component mounting area image may be obtained by photographing the component mounting area with a camera, or may be obtained by using a visual sensor.

[0074] As an example, the solder joint defect recognition model can be trained using an existing image recognition model. For example, the image recognition model can be a convolutional neural network model.

[0075] As an example, the defect recognition result may include whether a solder joint corresponding to the sub-image has a defect, the defect type, and the location information of the defect.

[0076] In one embodiment, an edge detection algorithm may be used to preliminarily identify the regional position coordinates of each solder joint in the component mounting area image.

[0077] It is understandable that due to the existence of reflection and solder joint defects, the above-mentioned area position coordinates are only rough positioning of the solder joints.

[0078] In the disclosed embodiment, after obtaining the regional position coordinates of each solder joint, in order to solve the above-mentioned problem of inaccurate solder joint positioning, a bounding box corresponding to each regional position coordinate is generated according to a preset rule, and the component mounting area image is divided according to the bounding box to obtain multiple sub-images, thereby ensuring that each sub-image can include a complete solder joint image.

[0079] It is understandable that the grayscale value of the reflective point is different from the grayscale value of other areas of the solder joint. Therefore, the grayscale value of each pixel in the sub-image can be used to identify the reflective point on the solder joint in the sub-image.

[0080] In one embodiment, in order to more accurately identify solder joint defects in the component mounting area, the identified reflective points may be removed, thereby avoiding interference of reflections on defect identification.

[0081] In the disclosed embodiment, the sub-image after the reflection is removed may be input into a pre-trained solder joint defect recognition model, and the solder joint defect recognition model may output a defect recognition result after recognizing the sub-image after the reflection is removed.

[0082] In some embodiments, the defect identification result may be any one or more of the following items: false soldering, bridging soldering, insufficient soldering, half soldering, and pulled tip.

[0083] In some embodiments of the present disclosure, the segmentation module 103 may specifically include:

[0084] The first acquisition submodule is used to acquire the size information of the standard bounding box; the standard bounding box is generated using the solder joint image of the standard solder joint;

[0085] The first generating submodule is used to generate a bounding box corresponding to the region position coordinates according to the size information; the region position coordinates fall into the bounding box; the size of the bounding box corresponds to the size information.

[0086] It is understandable that since solder joints may have problems such as reflection, false soldering, bridging, insufficient soldering, half soldering, and pointed ends, the solder joint size identified in the component mounting area image may be different from the actual solder joint size. If the solder joint area position coordinates are directly identified based on the component mounting area image, inaccurate recognition will occur. Segmentation errors will occur when the image is segmented according to the area position coordinates, resulting in the segmented sub-image being unable to cover the complete solder joint image, which in turn leads to inaccurate recognition results.

[0087] Therefore, a solder joint image of a standard solder joint may be acquired in advance, a standard bounding box of the solder joint in the solder joint image may be generated, size information of the standard bounding box may be obtained, and a bounding box corresponding to the above-mentioned area position coordinates may be generated according to the size information.

[0088] In one embodiment, due to the presence of defects such as false solder joints or pulled tips at the lower solder joints, the area that needs to be identified may be larger than the size of the above-mentioned standard bounding box. Therefore, a size adjustment value can be set according to actual conditions, and the size information can be adaptively adjusted based on the size adjustment value to ensure that the bounding box generated for the above-mentioned area position coordinates according to the adjusted size information can cover the complete solder joint image.

[0089] As an example, a bounding box corresponding to the region position coordinates may be generated according to the above-adjusted size information, and the region position coordinates fall within the bounding box.

[0090] In some embodiments of the present disclosure, the first identification module 104 may specifically include:

[0091] The second acquisition submodule is used to acquire the gray value of each pixel point in the position coordinates of the welding point area in the sub-image to obtain a gray value set;

[0092] A selection submodule is used to select, for each column of pixels in the sub-image, a target pixel whose grayscale value is a sudden grayscale value, and the position coordinates of the target pixel;

[0093] A second generation submodule is used to generate a feature map to be identified based on the target pixel point and the position coordinates of the target pixel point;

[0094] The recognition submodule is used to input the feature map to be recognized into the pre-trained convolutional neural network model to obtain the boundary line of the reflective point output by the convolutional neural network model based on the feature map to be recognized;

[0095] The determination submodule is used to determine the regional position coordinates of the reflective point based on the boundary line of the reflective point.

[0096] It can be understood that when there is reflection at the welding point in the sub-image, there is a large difference between the grayscale value of the reflective area and the grayscale value of the non-reflective area. Therefore, the grayscale value corresponding to the pixel points in each column of the sub-image can be analyzed, and the grayscale mutation value among the pixels in the same column can be selected. The pixel point corresponding to the grayscale mutation value is likely to be the edge pixel point of the reflective area. Therefore, the pixel point corresponding to the grayscale mutation value can be determined as the target pixel point.

[0097] In one embodiment, a feature map to be identified including all target pixels in a sub-image may be generated according to the position coordinates of the target pixels, so as to be used to identify reflective points using a pre-trained convolutional neural network model.

[0098] It should be noted that if the sub-image is directly input into the pre-trained convolutional neural network model for recognition, it may be difficult for the convolutional neural network model to accurately identify the reflective points in the sub-image directly due to problems such as limited training samples during the training process of the convolutional neural network. Generating the feature map to be identified through the target pixel points plays a role in enhancing the edge features of the reflective points and effectively reduces the amount of data, so that the convolutional neural network model can quickly and accurately identify the exact location of the reflective points.

[0099] In addition, since the sub-image is generated based on the bounding box, the sub-image includes not only the solder joint image, but also the image of part of the PCB substrate. The image of the PCB substrate does not only have a single color. Therefore, there may be some interfering pixels in the above-mentioned feature map to be identified. However, there are still certain differences between the image grayscale value and pattern shape of the PCB substrate and the solder joints and reflective points. Therefore, by identifying reflective points through the pre-trained convolutional neural network model, the boundary line of the reflective point can be accurately identified from the feature map to be identified with interfering pixels.

[0100] In some embodiments of the present disclosure, the system may further include:

[0101] A calculation module, used for calculating the average value of all gray values ​​in the position coordinates of the solder joint area when it is determined that there is no sudden gray value in the gray value set;

[0102] An adding module, used for adding the average value and a preset grayscale adjustment value to obtain a first threshold value;

[0103] A selection module, used for selecting a target grayscale value greater than or equal to a first threshold value from a grayscale value set;

[0104] The first determination module is used to determine the regional position coordinates of the reflective point based on the pixel points corresponding to the target grayscale value.

[0105] It is understandable that the grayscale value of the boundary line of some reflective points is greatly different from the grayscale value of the adjacent non-reflective area, but the boundaries of some reflective points are unclear and there is a gradual transition area.

[0106] Therefore, in the embodiment of the present disclosure, when it is determined that there is no sudden change in grayscale value in the grayscale value set, the average value of all grayscale values ​​in the position coordinates of the welding point area is calculated, and the average value is used to screen the pixel points corresponding to the reflective points.

[0107] It can be understood that the grayscale value of the reflective point is greater than the grayscale value of the non-reflective area of ​​the welding point, and the same is true for the reflective pixels in the above-mentioned transition area. Therefore, a certain value can be added to the average value, that is, the average value is added to the preset grayscale adjustment value to obtain the first threshold value. Using the first threshold value to screen the reflective pixels can improve the accuracy of the screening results.

[0108] In one embodiment, a target grayscale value greater than or equal to a first threshold is selected from a grayscale value set, and a pixel point corresponding to the target grayscale value is determined as a reflective pixel point. The set of coordinates of all reflective pixel points in the sub-image is the regional position coordinates of the reflective point.

[0109] In some embodiments of the present disclosure, the removal module 105 may specifically include:

[0110] The third acquisition submodule is used to acquire other grayscale values ​​in the grayscale value set except the mutation grayscale value;

[0111] The calculation submodule is used to calculate the average value of other gray values;

[0112] The assignment submodule is used to assign the grayscale values ​​of all pixels within the regional position coordinates of the reflective point to an average value to obtain a sub-image after removing the reflection.

[0113] In one embodiment, after identifying the regional position coordinates of the reflective points, other grayscale values ​​in the grayscale value set except the sudden grayscale value can be obtained, and the average value of other grayscale values ​​can be used to cover the reflective point area, that is, the grayscale value of the reflective point area is replaced with the above average value, thereby removing the reflective points in the sub-image.

[0114] In some embodiments of the present disclosure, there are multiple component mounting area images of the circuit board to be tested, and each component mounting area image is shot at a different angle. The second acquisition module 102 may specifically include:

[0115] A fourth acquisition submodule is used to acquire the regional position coordinates of each solder point in each component mounting area image of the plurality of component mounting area images;

[0116] The system also includes:

[0117] The second determination module is used to determine, for each solder joint in the circuit board to be tested, a plurality of sub-images corresponding to the solder joint by using the relative position relationship between the sub-image and the corresponding component mounting area image; the plurality of sub-images include an image of the solder joint;

[0118] The third determination module is used to determine whether a welding defect exists in a welding point when a defect recognition result of at least one sub-image among a plurality of sub-images corresponding to the welding point is that a defect exists.

[0119] It should be noted that in order to better identify defects, the same component mounting area can be photographed from multiple angles to prevent the inability to identify defects in the blind area due to failure to capture the complete solder joint image due to the shooting angle.

[0120] In one embodiment, the regional position coordinates of each solder point in each component mounting area image may be acquired using the acquisition method in any one of the embodiments of the present disclosure.

[0121] As an example, an identification can be generated for each solder joint on the circuit board to be tested, and each identification is bound to the relative position relationship between the corresponding solder joint and the component mounting area of ​​the circuit board to be tested. After obtaining multiple sub-images, the relative position relationship between the solder joint image and the component mounting area image in each sub-image is also determined, and the above identification is matched with the sub-image according to the relative position relationship, so as to determine the sub-images corresponding to each solder joint and belonging to multiple different component mounting area images, that is, to obtain sub-images of each solder joint at different shooting angles.

[0122] In the disclosed embodiment, when the defect recognition result of at least one sub-image among the multiple sub-images corresponding to the weld is a defect, it means that the weld has a defect at the angle corresponding to the at least one sub-image, thereby achieving multi-angle defect recognition of the weld and avoiding the problem of inability to fully identify defects due to shooting blind spots.

[0123] In addition, by using the defective sub-image with the defective result mentioned above, the specific location of the defect in the solder joint can be further determined, that is, it can be determined on which side of the solder joint the defect is located, thereby providing more information for further analysis of the cause of the defect and helping to optimize the component placement process.

[0124] According to the component mounting quality monitoring system based on the SMT placement machine proposed in the embodiment of the present disclosure, by acquiring the component mounting area image of the circuit board to be tested, the area position coordinates of each solder joint in the component mounting area image are acquired, and for each area position coordinate, a bounding box corresponding to the area position coordinate is generated according to a preset rule, and the component mounting area image is divided according to the bounding box to obtain a plurality of sub-images, and for each of the plurality of sub-images, the grayscale value of each pixel point in the sub-image is acquired, and the grayscale value is used to identify the reflective points in the sub-image, and the reflective points are removed to obtain a sub-image after the reflection is removed, and the sub-image after the reflection is removed is input into a pre-trained solder joint defect recognition model to obtain a defect recognition result output by the solder joint defect recognition model after the obtained sub-image is recognized. By using the bounding box method to segment the solder joint image, the situation where the sub-image segmentation error is caused by reflection or defects in the solder joint itself is effectively avoided, that is, the segmented sub-image does not contain the complete solder joint image to be identified, thereby improving the accuracy of identifying solder joint defects. In addition, the reflective points in the sub-image are accurately identified by grayscale value, and the reflective points are removed before defect identification, avoiding the interference of reflection on the monitoring results and improving the accuracy of component mounting quality monitoring.

[0125] Figure 2 FIG. 1 is a flow chart of a component placement quality monitoring method based on an SMT placement machine according to an exemplary embodiment. Figure 2 , the method comprising:

[0126] Step 201, obtaining a component mounting area image of a circuit board to be tested;

[0127] Step 202, obtaining the regional position coordinates of each solder joint in the component mounting area image;

[0128] Step 203, for each area position coordinate, generate a bounding box corresponding to the area position coordinate according to a preset rule, and divide the component mounting area image according to the bounding box to obtain multiple sub-images;

[0129] Step 204, for each sub-image in the plurality of sub-images, obtain the gray value of each pixel in the sub-image, and use the gray value to identify the reflective point in the sub-image; the reflective point is the reflective point on the welding point in the sub-image;

[0130] Step 205, removing the reflection points to obtain a sub-image after removing the reflections;

[0131] Step 206 , input the sub-image after removing the reflection into the pre-trained solder joint defect recognition model, and obtain the defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image.

[0132] In some embodiments of the present disclosure, generating a bounding box corresponding to the region position coordinates according to a preset rule in step 203 includes:

[0133] Step a1, obtaining size information of a standard bounding box; the standard bounding box is generated using a solder joint image of a standard solder joint;

[0134] Step a2: Generate a bounding box corresponding to the region position coordinates according to the size information; the region position coordinates fall into the bounding box; the size of the bounding box corresponds to the size information.

[0135] In some embodiments of the present disclosure, obtaining the grayscale value of each pixel in the sub-image in step 204 and identifying the reflective points in the sub-image using the grayscale value includes:

[0136] Step b1, obtaining the gray value of each pixel point in the position coordinates of the welding point area in the sub-image to obtain a gray value set;

[0137] Step b2, for each column of pixels in the sub-image, selecting a target pixel whose grayscale value is a sudden grayscale value, and the position coordinates of the target pixel;

[0138] Step b3, generating a feature map to be identified based on the target pixel and the position coordinates of the target pixel;

[0139] Step b4, inputting the feature map to be identified into a pre-trained convolutional neural network model to obtain a boundary line of reflective points output by the convolutional neural network model based on the feature map to be identified;

[0140] Step b5, determining the regional position coordinates of the reflective point based on the boundary line of the reflective point.

[0141] In some embodiments of the present disclosure, after obtaining the grayscale value of each pixel point in the weld area position coordinates in the sub-image in step b1 to obtain a grayscale value set, the following further includes:

[0142] Step c1, when it is determined that there is no sudden change in the gray value set, the average value of all gray values ​​in the position coordinates of the solder joint area is calculated;

[0143] Step c2, adding the average value to the preset grayscale adjustment value to obtain a first threshold value;

[0144] Step c3, selecting a target grayscale value greater than or equal to a first threshold from the grayscale value set;

[0145] Step c4, determining the regional position coordinates of the reflective point based on the pixel points corresponding to the target grayscale value.

[0146] In some embodiments of the present disclosure, the reflective points are removed to obtain a sub-image after the reflections are removed, including:

[0147] Step d1, obtaining other grayscale values ​​in the grayscale value set except the mutation grayscale value;

[0148] Step d2, calculating the average values ​​of other gray values;

[0149] Step d3, assigning the grayscale values ​​of all pixels within the regional position coordinates of the reflective point to an average value to obtain a sub-image after removing the reflection.

[0150] In some embodiments of the present disclosure, there are multiple component mounting area images of the circuit board to be tested, and each component mounting area image is shot at a different angle; obtaining the area position coordinates of each solder joint in the component mounting area image includes:

[0151] For each component mounting area image in the plurality of component mounting area images, obtaining the area position coordinates of each solder point in the component mounting area image;

[0152] The method also includes:

[0153] For each solder joint in the circuit board to be tested, a plurality of sub-images corresponding to the solder joint are determined by using a relative position relationship between the sub-image and the image of the component mounting area to which it belongs; the plurality of sub-images include an image of the solder joint;

[0154] When the defect recognition result of at least one sub-image among the multiple sub-images corresponding to the welding point is that there is a defect, it is determined that there is a welding defect in the welding point.

[0155] Regarding the method in the above embodiment, the specific manner of performing the operation in each step has been described in detail in the embodiment of the device, and will not be elaborated here.

[0156] According to the component mounting quality monitoring method based on the SMT placement machine proposed in the embodiment of the present disclosure, by acquiring the component mounting area image of the circuit board to be tested, the area position coordinates of each solder joint in the component mounting area image are acquired, and for each area position coordinate, a bounding box corresponding to the area position coordinate is generated according to a preset rule, and the component mounting area image is divided according to the bounding box to obtain a plurality of sub-images, and for each sub-image in the plurality of sub-images, the grayscale value of each pixel point in the sub-image is acquired, and the grayscale value is used to identify the reflective points in the sub-image, and the reflective points are removed to obtain a sub-image after the reflection is removed, and the sub-image after the reflection is removed is input into a pre-trained solder joint defect recognition model to obtain a defect recognition result output by the solder joint defect recognition model after the obtained sub-image is recognized. By using the bounding box method to segment the solder joint image, the situation where the sub-image segmentation error is caused by reflection or defects in the solder joint itself is effectively avoided, that is, the segmented sub-image does not contain the complete solder joint image to be identified, thereby improving the accuracy of identifying solder joint defects. In addition, the reflective points in the sub-image are accurately identified by grayscale value, and the reflective points are removed before defect identification, avoiding the interference of reflection on the monitoring results and improving the accuracy of component mounting quality monitoring.

[0157] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method and system according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0158] The above is only a specific implementation of the present disclosure. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present disclosure is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure.

Claims

1. A component placement quality monitoring system based on an SMT placement machine, characterized in that: include: A first acquisition module is used to acquire an image of a component mounting area of ​​a circuit board to be tested; A second acquisition module is used to acquire the regional position coordinates of each solder point in the component mounting area image; A segmentation module, for generating a bounding box corresponding to each area position coordinate according to a preset rule, and segmenting the component mounting area image according to the bounding box to obtain a plurality of sub-images; A first recognition module, configured to obtain, for each sub-image of the plurality of sub-images, a grayscale value of each pixel in the sub-image, and recognize a reflective point in the sub-image by using the grayscale value; The reflective point is a reflective point on a welding point in the sub-image; A removal module, used for removing the reflective points to obtain a sub-image after removing the reflective points; A second recognition module is used to input the sub-image after removing the reflection into a pre-trained solder joint defect recognition model to obtain a defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image; Wherein, the segmentation module includes: The first acquisition submodule is used to acquire the size information of the standard bounding box; the standard bounding box is generated by using the solder joint image of the standard solder joint; A first generating submodule is used to generate a bounding box corresponding to the region position coordinates according to the size information; the region position coordinates fall inside the bounding box; the size of the bounding box corresponds to the size information; Wherein, the first identification module includes: A second acquisition submodule is used to acquire the gray value of each pixel point in the welding point area position coordinates in the sub-image to obtain a gray value set; A selection submodule is used to select, for each column of pixels in the sub-image, a target pixel whose grayscale value is a sudden grayscale value, and the position coordinates of the target pixel; A second generating submodule, used for generating a feature map to be identified based on the target pixel point and the position coordinates of the target pixel point; The recognition submodule is used to input the feature map to be recognized into a pre-trained convolutional neural network model to obtain a boundary line of reflective points output by the convolutional neural network model based on the feature map to be recognized; A determination submodule, used to determine the regional position coordinates of the reflective point based on the reflective point boundary line; The system also includes: A calculation module, configured to calculate an average value of all gray values ​​within the position coordinates of the solder joint area when it is determined that there is no sudden gray value in the gray value set; An adding module, used for adding the average value and a preset grayscale adjustment value to obtain a first threshold value; A selection module, configured to select a target grayscale value greater than or equal to the first threshold from the grayscale value set; A first determination module, used to determine the regional position coordinates of the reflective point based on the pixel points corresponding to the target grayscale value; Wherein, the removal module comprises: A third acquisition submodule is used to acquire other grayscale values ​​in the grayscale value set except the sudden grayscale value; A calculation submodule, used for calculating the average value of the other grayscale values; The assignment submodule is used to assign the grayscale values ​​of all pixels within the regional position coordinates of the reflective point to the average value to obtain the sub-image after removing the reflection.

2. The component placement quality monitoring system based on the SMT placement machine according to claim 1 is characterized in that: There are multiple component mounting area images of the circuit board to be tested, and each component mounting area image is shot at a different angle; the second acquisition module includes: A fourth acquisition submodule is used to acquire the regional position coordinates of each solder point in each component mounting area image of the plurality of component mounting area images; The system further comprises: A second determination module is used to determine, for each solder joint in the circuit board to be tested, a plurality of sub-images corresponding to the solder joint by using a relative position relationship between a sub-image and an image of a corresponding component mounting area; the plurality of sub-images includes an image of the solder joint; The third determination module is used to determine whether the welding spot has a welding defect when the defect recognition result of at least one sub-image among the multiple sub-images corresponding to the welding spot is that there is a defect.

3. A component placement quality monitoring method based on an SMT placement machine, characterized in that: include: Acquire a component mounting area image of the circuit board to be tested; Obtaining the regional position coordinates of each solder point in the component mounting area image; For each area position coordinate, generating a bounding box corresponding to the area position coordinate according to a preset rule, and dividing the component mounting area image according to the bounding box to obtain a plurality of sub-images; For each sub-image of the multiple sub-images, obtaining a grayscale value of each pixel in the sub-image, and using the grayscale value to identify a reflective point in the sub-image; The reflective point is a reflective point on a welding point in the sub-image; Removing the reflection points to obtain a sub-image after the reflections are removed; Inputting the sub-image after removing the reflection into a pre-trained solder joint defect recognition model to obtain a defect recognition result output by the solder joint defect recognition model after recognizing the obtained sub-image; The step of generating a bounding box corresponding to the region position coordinates according to a preset rule includes: Acquire size information of a standard bounding box; the standard bounding box is generated using a solder joint image of a standard solder joint; According to the size information, a bounding box corresponding to the region position coordinates is generated; the region position coordinates fall inside the bounding box; the size of the bounding box corresponds to the size information; The step of obtaining the grayscale value of each pixel in the sub-image and identifying the reflective points in the sub-image by using the grayscale value includes: Obtaining the grayscale value of each pixel point in the position coordinates of the welding point area in the sub-image to obtain a grayscale value set; For each column of pixels in the sub-image, a target pixel whose grayscale value is a sudden grayscale value and the position coordinates of the target pixel are selected; Generate a feature map to be identified based on the target pixel point and the position coordinates of the target pixel point; Inputting the feature map to be identified into a pre-trained convolutional neural network model to obtain a boundary line of reflective points output by the convolutional neural network model based on the feature map to be identified; Determine the regional position coordinates of the reflective point based on the reflective point boundary line; Wherein, after obtaining the grayscale value of each pixel point in the welding spot area position coordinates in the sub-image to obtain the grayscale value set, it also includes: When it is determined that there is no sudden grayscale value in the grayscale value set, calculating the average value of all grayscale values ​​in the position coordinates of the soldering point area; Adding the average value to a preset grayscale adjustment value to obtain a first threshold value; Selecting a target grayscale value greater than or equal to the first threshold from the grayscale value set; Determine the regional position coordinates of the reflective point based on the pixel points corresponding to the target grayscale value; The step of removing the reflective points to obtain a sub-image after removing the reflective points includes: Obtaining other grayscale values ​​in the grayscale value set except the sudden grayscale value; Calculate the average value of the other gray values; The grayscale values ​​of all pixels within the regional position coordinates of the reflective point are assigned as the average value to obtain the sub-image after removing the reflection.

Images