PCB component detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a PCB component detection method and device, electronic equipment and a storage medium, and the method comprises the steps: calculating the matching degree between a first visible light image of a to-be-detected PCB and each template image, the template image being a visible light image of a template PCB and including a component in which a user is interested in the template PCB; selecting a target template image based on the obtained matching degree; and determining a component included in an area matched with the area where the component in the target template image is located in the first visible light image as a target component. As the area where the component is located in the pre-generated template image is relatively accurate, the accuracy of component detection can be remarkably improved by determining the target component based on the template image. In addition, since the template image comprises the components which the user is interested in, the components which the user is not interested in can be automatically filtered out, and the detection accuracy is further improved.

Description

Technical Field

[0001] The present application relates to the technical field of PCB detection, and in particular to a PCB component detection method, device, electronic equipment and storage medium. Background Art

[0002] With the continuous improvement of the integration of electronic products, the density and complexity of components included in PCB (Printed Circuit Board) boards have also increased significantly, which has put higher requirements on the detection of components.

[0003] The existing PCB component detection method is to collect images of PCB boards, determine the edge information of components in the image through models such as CNN (Convolutional Neural Network), and thus determine the area of ​​components in the image to complete component detection. However, due to factors such as edge detection accuracy, when using the above method for PCB component detection, there will be false detections, resulting in low detection accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a PCB component detection method, device, electronic device and storage medium to improve the accuracy of component detection. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a PCB component detection method, the method comprising:

[0006] Calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board;

[0007] Selecting a target template image from each template image based on the obtained matching degree;

[0008] Components included in a target area in the first visible light image are determined as target components, wherein the target area is an area that matches an area where components in the target template image are located.

[0009] Optionally, the calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image includes:

[0010] Perform component detection on the first visible light image of the PCB board to be detected to obtain a first region of components in the first visible light image;

[0011] The obtained first regions are matched with the second regions of the components in each template image, and the matching degrees between the first visible light image and each template image are obtained based on the matching results.

[0012] Optionally, matching the obtained first region with the second region of the component in each template image, and obtaining the matching degree between the first visible light image and each template image based on the matching result, includes:

[0013] For each template image, the matching degree between the first visible light image and the template image is obtained in the following manner:

[0014] Matching the obtained first region with the second region of the component in the template image to obtain a first number of successfully matched first regions;

[0015] A ratio of the first number to the total number of second regions in the template image is calculated, and a matching degree between the first visible light image and the template image is obtained based on the ratio.

[0016] Optionally, matching the obtained first area with the second area of ​​the component in the template image to obtain a first number of successfully matched first areas includes:

[0017] For each first region, respectively calculate the matching degree between the first region and the second region of the component in the template image, and if the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, determine that the first region is a successfully matched first region;

[0018] A first number of first regions that are matched successfully is obtained.

[0019] Optionally, the matching degree between the first region and the second region is calculated in the following manner:

[0020] Obtaining a first area where the first area intersects the second area;

[0021] The difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area:

[0022] A ratio of the first area to the second area is calculated as a matching degree between the first region and the second region.

[0023] Optionally, determining the components included in the target area in the first visible light image as target components includes:

[0024] In the region where the components in the first visible light image are located, a region whose matching degree with the region where the components in the target template image are located is greater than a second preset matching degree threshold is selected as the target region;

[0025] The components included in the target area are determined as target components.

[0026] Optionally, after determining the components included in the target area in the first visible light image as target components, the method further includes:

[0027] From the determined target components, remove the target components whose area size is smaller than a preset size threshold;

[0028] and / or,

[0029] Taking the center point of the area where the target component is located as the base point, the area where the target component is located is enlarged according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

[0030] Optionally, the method further includes:

[0031] Acquire a thermal imaging image of the PCB board to be inspected that is acquired synchronously with the first visible light image;

[0032] In the thermal imaging image, determining a fourth area corresponding to the position of the area where the target component is located;

[0033] determining the temperature of the target component according to the image data of the fourth area;

[0034] Detect whether the temperature of the target component is abnormal.

[0035] Optionally, the image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes:

[0036] Acquire temperature measurement parameters of components in a third region, wherein the third region is a region where components in the target template image are located and has the highest matching degree with the region where the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature;

[0037] According to the temperature measurement parameter, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

[0038] Optionally, the image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes:

[0039] In the thermal imaging image, determining a maximum grayscale value in the fourth area;

[0040] The maximum grayscale value is converted into a temperature as the temperature of the target component.

[0041] Optionally, the detecting whether the temperature of the target component is abnormal includes:

[0042] Acquire a preset alarm temperature of a component in a third area, wherein the third area is an area in the target template image where the component is located and has the highest matching degree with the area where the target component is located;

[0043] It is determined whether the temperature of the target component is greater than the preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

[0044] Optionally, the template image is generated by:

[0045] For the second visible light image of the template PCB board, perform the following operations:

[0046] Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction;

[0047] and / or,

[0048] Based on parameter information set for an initial region where components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

[0049] In a second aspect, an embodiment of the present application provides a PCB component detection device, the device comprising:

[0050] A matching module, used for calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board;

[0051] An image selection module, used for selecting a target template image from each template image based on the obtained matching degree;

[0052] The component determination module is used to determine the components included in the target area in the first visible light image as target components, wherein the target area is an area that matches the area where the components in the target template image are located.

[0053] Optionally, the matching module includes:

[0054] A component detection unit, configured to perform component detection on the first visible light image of the PCB board to be detected, and obtain a first region of components in the first visible light image;

[0055] The region matching unit is used to match the obtained first region with the second region of the component in each template image, and obtain the matching degree between the first visible light image and each template image based on the matching result.

[0056] Optionally, the region matching unit includes:

[0057] A matching number determination subunit is used to match the obtained first area with the second area of ​​the component in the template image for each template image, and obtain a first number of first areas that are successfully matched;

[0058] The matching degree calculation subunit is used to calculate, for each template image, the ratio of the first number corresponding to the template image output by the matching number determination subunit to the total number of second areas in the template image, and obtain the matching degree between the first visible light image and the template image based on the ratio.

[0059] Optionally, the matching quantity determination subunit is specifically used to:

[0060] For each first area, the matching degree between the first area and the second area of ​​the component in the template image is calculated respectively. If the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, the first area is determined to be a successfully matched first area; and a first number of successfully matched first areas is obtained.

[0061] Optionally, the matching degree between the first region and the second region is calculated in the following manner:

[0062] Obtaining a first area where the first area intersects the second area;

[0063] The difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area:

[0064] A ratio of the first area to the second area is calculated as a matching degree between the first region and the second region.

[0065] Optionally, the component determination module includes:

[0066] A region screening unit, configured to screen out, from the region where the component in the first visible light image is located, a region whose matching degree with the region where the component in the target template image is located is greater than a second preset matching degree threshold value as a target region;

[0067] The target component determination unit is used to determine the components included in the target area as target components.

[0068] Optionally, the device further comprises:

[0069] The component adjustment module is used to remove target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as a base point, enlarge the area where the target component is located according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

[0070] Optionally, the device further comprises:

[0071] A thermal imaging image acquisition module, used to acquire a thermal imaging image of the PCB board to be inspected which is acquired synchronously with the first visible light image;

[0072] A region determination module, used to determine, in the thermal imaging image, a fourth region corresponding to the position of the region where the target component is located;

[0073] A temperature determination module, used for determining the temperature of the target component according to the image data of the fourth area;

[0074] The temperature detection module is used to detect whether the temperature of the target component is abnormal.

[0075] Optionally, the image data of the fourth area includes a grayscale value, and the temperature determination module is specifically configured to:

[0076] Obtain temperature measurement parameters of components in a third area, wherein the third area is an area in which components in the target template image are located, and has the greatest matching degree with the area in which the target components are located, and the temperature measurement parameters are parameters for converting grayscale values ​​into temperature; according to the temperature measurement parameters, convert the grayscale values ​​in the fourth area into temperature as the temperature of the target component.

[0077] Optionally, the image data of the fourth area includes a grayscale value, and the temperature determination module is specifically configured to:

[0078] In the thermal imaging image, a maximum grayscale value in the fourth area is determined; and the maximum grayscale value is converted into a temperature as the temperature of the target component.

[0079] Optionally, the temperature detection module includes:

[0080] An alarm temperature acquisition unit, used to acquire a preset alarm temperature of a component in a third area, wherein the third area is an area in the target template image where the component is located and has the highest matching degree with the area where the target component is located;

[0081] The temperature detection unit is used to determine whether the temperature of the target component is greater than the preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

[0082] Optionally, the template image is generated by:

[0083] For the second visible light image of the template PCB board, perform the following operations:

[0084] Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction;

[0085] and / or,

[0086] Based on parameter information set for an initial region where components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

[0087] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0088] Memory, used to store computer programs;

[0089] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.

[0090] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented.

[0091] Beneficial effects of the embodiments of the present application:

[0092] In the scheme provided by the embodiment of the present application, the electronic device can calculate the matching degree between the first visible light image of the PCB board to be detected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, the template image that is closer to the first visible light image can be selected as a reference through the matching degree between the first visible light image of the PCB board to be detected and each template image. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection.

[0093] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0095] Figure 1 A schematic diagram of a flow chart of a PCB component detection method provided in an embodiment of the present application;

[0096] Figure 2 Based on Figure 1 A schematic diagram of a flow chart of a method for obtaining a matching degree between images in the illustrated embodiment;

[0097] Figure 3 Based on Figure 2 A schematic diagram of the intersection-over-union ratio between regions in the illustrated embodiment;

[0098] Figure 4 Based on Figure 1 A schematic diagram of a specific process of the PCB component detection method of the illustrated embodiment;

[0099] Figure 5 Based on Figure 1 A schematic diagram of a flow chart of a temperature detection method of the illustrated embodiment;

[0100] Figure 6 Based on Figure 5 A schematic diagram of a flow chart of a temperature determination method in the illustrated embodiment;

[0101] Figure 7 Based on Figure 5 A schematic flow chart of a method of converting a gray value into a temperature in the illustrated embodiment;

[0102] Figure 8 Based on Figure 5 A schematic diagram of a flow chart of a method for detecting whether the temperature is abnormal in the illustrated embodiment;

[0103] Fig. 9 A schematic diagram of the structure of a PCB component detection device provided in an embodiment of the present application;

[0104] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0105] Fig.11 Based on Fig.10 A schematic diagram of a specific structure of an electronic device of the illustrated embodiment. DETAILED DESCRIPTION

[0106] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0107] In order to improve the accuracy of component detection, the embodiments of the present application provide a PCB component detection method, device, electronic device, computer-readable storage medium and computer program product. The following first introduces a PCB component detection method provided in the embodiments of the present application.

[0108] A PCB component detection method provided in an embodiment of the present application can be applied to any electronic device that needs to perform component detection on a PCB board. For example, the electronic device can be a fixed stand detection device, a handheld detection device, or a device with data processing capabilities connected to an image acquisition device, etc. No specific limitation is made here. For the sake of clear description, it is referred to as an electronic device below.

[0109] like Figure 1As shown, a PCB component detection method includes:

[0110] S101, calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image.

[0111] The template image is a visible light image of the template PCB board and includes components of interest to the user in the template PCB board.

[0112] S102: Select a target template image from each template image based on the obtained matching degree.

[0113] S103: Determine components included in the target area in the first visible light image as target components.

[0114] The target area is an area that matches the area where the components in the target template image are located.

[0115] In the scheme provided by the embodiment of the present application, the electronic device can calculate the matching degree between the first visible light image of the PCB board to be detected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, the template image that is closer to the first visible light image can be selected as a reference through the matching degree between the first visible light image of the PCB board to be detected and each template image. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection.

[0116] Here, the method for generating the template image in the embodiment of the present application is first introduced, wherein the electronic device for generating the template image and the electronic device for executing the PCB component detection method provided in the embodiment of the present application can be the same device or different devices, and no specific limitation is made here.

[0117] First, in the actual scenario of inspecting PCB components, the types of PCB boards to be inspected are foreseeable, so these types of PCB boards can be determined as template PCB boards, and then the visible light image of the template PCB board is obtained, that is, the second visible light image is obtained.

[0118] For example, for a certain factory, the PCB boards it generates include: Class A PCB boards, Class B PCB boards and Class C PCB boards. It can be foreseen that when the factory detects PCB components, the PCB boards to be detected are likely to include at least one of Class A PCB boards, Class B PCB boards and Class C PCB boards. Therefore, Class A PCB boards, Class B PCB boards and Class C PCB boards can all be determined as template PCB boards, and then visible light images of these three types of PCB boards can be obtained.

[0119] Then, the electronic device can determine the initial area where the components in the second visible light image are located.

[0120] In one embodiment, the electronic device can perform component detection on the second visible light image to obtain a second detection result including an initial region where the component in the second visible light image is located (i.e., a second region of the component in the second visible light image), the type of the component, and / or the confidence of the detection result of the component. The second region of the component can be specifically characterized by the coordinates of the second region, and the component detection can be achieved by a pre-trained CNN or other detection model, or by an edge detection algorithm such as a Sobel operator or a Canny operator, which is not specifically limited here.

[0121] In another embodiment, the electronic device may also obtain a second detection result corresponding to the second visible light image manually determined by the user. Specifically, the second region of the component in the second visible light image included in the second detection result may be manually drawn by the user, the type of the component may be manually marked by the user, and the confidence of the component detection result may be the confidence of the drawing and marking result of the component manually input by the user.

[0122] Finally, after obtaining the second detection result of the second visible light image, the electronic device may perform at least one of the following operations on the second visible light image to obtain a template image:

[0123] The first operation: the electronic device adjusts the second area of ​​the component in the second visible light image based on the adjustment instruction for the second area of ​​the component in the second visible light image, wherein the adjustment instruction may include: a deletion instruction, an addition instruction and / or a size adjustment instruction, specifically:

[0124] For the deletion instruction, in one embodiment, the deletion instruction may be an instruction preset by the user, that is, the user may preset the type of components (e.g., resistors and / or capacitors, etc.), size thresholds, and / or confidence thresholds of the detection results that the user is interested in. On this basis, after obtaining the second detection result of the second visible light image, the electronic device may delete, in the second visible light image, components whose corresponding type is not the component type preset by the user, whose initial area where the component is located has a size smaller than the size threshold, and / or whose corresponding confidence is not greater than the preset confidence threshold.

[0125] For example, the user pre-sets the type of component he is interested in as a capacitor, the size threshold is "3 pixels long and 3 pixels wide", and the confidence threshold is 80%. Assume that after the second visible light image is detected for components, the detection results obtained include: the first component (type is a capacitor, the confidence is 88%, the length of the second area is 4 pixels and the width is 5 pixels), the second component (type is a resistor, the confidence is 90%, the length of the second area is 3 pixels and the width is 5 pixels), the third component (type is a capacitor, the confidence is 60%, the length of the second area is 4 pixels and the width is 4 pixels), and the fourth component is (type is a capacitor, the confidence is 81%, the length of the second area is 2 pixels and the width is 1 pixel). Then, according to the above-mentioned preset component type, size threshold and confidence threshold, the electronic device can delete the second component, the third component and the fourth component from the second visible light image. Among them, deleting the component can specifically be the detection result of deleting the component.

[0126] For the deletion instruction, in another embodiment, the deletion instruction can also be an instruction issued by the user in real time, that is, after the electronic device obtains the second area of ​​the component in the second visible light image, it can mark the second area in the second visible light image by means of a target frame, and display the marking result to the user. The user can select the second area to be deleted according to actual usage requirements, thereby generating a deletion instruction. Then the electronic device can delete the second area in the second visible light image based on the deletion instruction.

[0127] For example, after viewing the second regions displayed in the second visible light image, the user finds that one of the second regions does not contain any components, that is, the second region is a falsely detected region. Then, the user can manually identify the second region as a second region to be deleted, thereby generating a deletion instruction. When the electronic device receives the deletion instruction, it can delete the falsely detected second region.

[0128] For the adding instruction, when the electronic device displays the second area to the user, the user can add a new second area according to actual usage requirements, thereby generating an adding instruction. Then, the electronic device can add a new second area to the second visible light image based on the adding instruction.

[0129] For example, after viewing the second regions displayed in the second visible light image, the user finds that a component of interest has been missed, so the user can manually add a target frame at the component to identify the second region of the component. The electronic device can then add a new second region to the second visible light image based on the addition instruction, that is, record information such as the location of the newly added second region in the detection result in the second visible light image.

[0130] For the size adjustment instruction, when the electronic device displays the second area to the user, the user can adjust the size of the displayed second area according to actual usage requirements, thereby generating a size adjustment instruction. The electronic device can then adjust the size information of the second area in the second visible light image based on the size adjustment instruction.

[0131] For example, after viewing each second area displayed in the second visible light image, the user finds that a second area does not completely cover the components in the second area, so the user can manually enlarge the size of the second area so that the second area completely covers the components in the second area. Then, the electronic device can update the size information of the second area in the detection result in the second visible light image based on the size adjustment instruction.

[0132] In the second operation, after obtaining the second region of the component in the second visible light image, the electronic device can identify the second region in the second visible light image by means of a target frame, and display the identification result to the user. Then, the user can set the temperature measurement parameters and / or alarm temperature and other parameter information of the second region according to the type of the component in the second region. After obtaining the parameter information set by the user for the second region in the second detection result, the electronic device can establish a corresponding relationship between the second region and the parameter information.

[0133] For example, after obtaining the second area of ​​each component in the second visible light image, the electronic device can identify each second area in the second visible light image by means of a target box, and display the identification result to the user. Assuming that the component in the first second area is a resistor and the component in the second second area is a capacitor, the user can set the temperature measurement parameters and alarm temperature corresponding to the first second area according to the characteristics of the resistor, and set the temperature measurement parameters and alarm temperature corresponding to the second second area according to the characteristics of the capacitor. After obtaining the above parameter information, the electronic device can establish a correspondence between the second area and the parameter information.

[0134] After obtaining the template images of each template PCB board in the above manner, each template image can be stored in an electronic device that executes the PCB component detection method provided in the embodiment of the present application.

[0135] The following will introduce Figure 1 PCB component detection method shown.

[0136] When it is necessary to inspect the PCB components included in the PCB board to be inspected, the electronic device can obtain a first visible light image of the PCB board to be inspected.

[0137] For example, when the electronic device is a fixed pedestal detection device, a handheld detection device or other detection device, the electronic device itself may include a camera, on this basis, the electronic device can collect the first visible light image of the PCB board to be detected through its own camera. For another example, when the electronic device is a device with data processing capabilities such as a computer, it can collect the first visible light image of the PCB board to be detected through an image acquisition device connected thereto.

[0138] After obtaining the first visible light image, the electronic device may calculate the matching degree between the first visible light image and each template image. The electronic device may calculate the matching degree between the first visible light image and each template image by, but not limited to, the following methods:

[0139] In one embodiment, the electronic device can respectively extract image features of the first visible light image and image features of each template image, and then the electronic device can respectively calculate the similarity between the image features of the first visible light image and the image features of each template image, thereby obtaining the matching degree between the first visible light image and each template image.

[0140] In another embodiment, because the area where the components in the template image are located is determined when the template image is generated in advance, the electronic device can also first perform component detection on the first visible light image to obtain the first area of ​​the components in the first visible light image, and then match the obtained first area with the second area of ​​the components in each template image, and obtain the matching degree between the first visible light image and each template image based on the matching result. Among them, component detection can be achieved through a pre-trained CNN or other detection model, or through an edge detection algorithm such as a Sobel operator or a Canny operator, which is not specifically limited here.

[0141] After obtaining the matching degree between the first visible light image and each template image, the electronic device can select a target template image from each template image based on the obtained matching degree. For example, the template image with the greatest matching degree with the first visible light image is determined as the target template image, and the target template image is randomly determined from the template images with a matching degree with the first visible light image greater than a preset threshold, and so on.

[0142] Because the area where the components in the target template image are located (i.e., the second area) identifies the area where the components of interest to the user are located in the image of the template PCB board, for an area where a component in the first visible light image is located, if the degree of match between the area and the second area in the target template image is low, it means that the area may be a falsely detected area, or the components in the area are components that the user is not interested in.

[0143] Therefore, in obtaining the target template image, the electronic device determines the area matching the area where the components in the target template image are located from the area where the components in the first visible light image are located, that is, determines the target area, and then determines the components included in the target area in the first visible light image as the target components. The above-mentioned "components in the target template image" can be all components in the target template image or some components in the target template image, which is not specifically limited here.

[0144] Regarding “determining the target area”, the electronic device may implement it through, but not limited to, the following methods:

[0145] In one embodiment, the electronic device may filter out, from the region where the components in the first visible light image are located, a region whose matching degree with the region where the components in the target template image is located is greater than a second preset matching degree threshold, as the target region, based on the matching degree between the region where the components in the target template image and the region where the components in the first visible light image are located.

[0146] In another embodiment, the electronic device may map the position of the region where the components in the target template image are located to the image coordinate system of the first visible light image, and then use the region corresponding to the mapped position in the image coordinate system of the first visible light image as the target region.

[0147] In the solution provided by the embodiment of the present application, on the one hand, a template image that is relatively close to the first visible light image can be selected as a reference through the matching degree between the first visible light image of the PCB board to be detected and each template image. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection.

[0148] As an implementation method of the present application, Figure 2 As shown, when the electronic device calculates the matching degree between the first visible light image and each template image through the second area in the template image and the first area in the first visible light image, the matching degree between the first visible light image and the template image can be obtained according to steps S201-S202 for each template image:

[0149] S201, matching the obtained first area with the second area of ​​the component in the template image to obtain a first number of successfully matched first areas.

[0150] For each first area in the first visible light image, the electronic device can calculate the matching degree between the first area and the second area of ​​each component in the template image, and if the maximum matching degree (i.e., the maximum value of each matching degree obtained) is greater than the first preset matching degree threshold, the first area is determined to be a first area that has successfully matched. After executing the above steps for each first area to determine whether it is a first area that has successfully matched, the electronic device can obtain the first number of first areas that have successfully matched. Among them, the first preset matching degree threshold and the above second preset matching degree threshold can be the same value or different values, which are not specifically limited here.

[0151] For example, the first regions included in the first visible light image are region A, region B, and region C, and the second regions included in the first template image are region a, region b, and region c. Then the electronic device can calculate the matching degree between region A and region a, region b, and region c, calculate the matching degree between region B and region a, region b, and region c, and calculate the matching degree between region C and region a, region b, and region c.

[0152] Assuming that the maximum matching degree between region A and region a is 70%, the maximum matching degree between region B and region b is 94%, and the maximum matching degree between region C and region c is 98, then because the maximum matching degree of region A is less than the first preset matching degree threshold of 85%, region A is the first region where the matching fails, because the maximum matching degree of region B is greater than the first preset matching degree threshold of 85%, region B is the first region where the matching succeeds, and because the maximum matching degree of region C is greater than the first preset matching degree threshold of 85%, region C is the first region where the matching succeeds. It can be determined that the first number of first regions where the matching succeeds is 2.

[0153] In one implementation, the electronic device may calculate the IOU (Intersection over Union) between the first region and the second region as the matching degree between the first region and the second region. Specifically:

[0154] The electronic device can obtain a first area where the first area intersects the second area, and then obtain the difference between the sum of the areas of the first area and the second area and the first area as the second area. The electronic device can then calculate the ratio of the first area to the second area as the degree of matching between the first area and the second area.

[0155] by Figure 3 For example, assuming that the area represented by rectangular box A is the first area, and the area represented by rectangular box B is the second area, the electronic device can calculate Figure 3 The area of ​​the overlapping portion of the rectangular frame A and the rectangular frame B represented by the shaded portion is used as the first area where the first area intersects the second area. The electronic device can then calculate the sum of the area of ​​the rectangular frame A and the area of ​​the rectangular frame B, and calculate the difference between the sum of the areas and the first area as the second area. Finally, the electronic device can calculate the ratio of the first area to the second area as the matching degree between the first area and the second area.

[0156] In addition to calculating the IOU between the first area and the second area as the matching degree between the first area and the second area, the electronic device may also calculate parameters such as DSC (Dice Similarity Coefficient) and HD (Hausdorff Distance) between the first area and the second area as the matching degree between the first area and the second area, which is not specifically limited here.

[0157] S202, calculating a ratio of the first number to the total number of second regions in the template image, and obtaining a matching degree between the first visible light image and the template image based on the ratio.

[0158] After obtaining the first number of first areas that successfully match the second areas in the template image, the electronic device can calculate the ratio of the first number to the total number of second areas in the template image, and then determine the ratio as the degree of match between the first visible light image and the template image.

[0159] Continuing with the above example, when matching the first area in the first visible light image with the second area in the first template image, the first number of successfully matched first areas is 2, and the second areas included in the first template image are area a, area b, and area c, that is, the total number of second areas in the first template image is 3. Then the electronic device can calculate the ratio of the first number to the total number of second areas in the first template image (2 / 3) as the degree of match between the first visible light image and the first template image.

[0160] In the solution provided in the embodiment of the present application, the electronic device can determine the matching degree between the first area in the first visible light image and the second area in the template image by calculating parameters such as IOU, DSC or HD of the two areas, so as to accurately evaluate the matching degree and reduce misjudgment. Then, for each template image, when matching the first area in the first visible light image with the second area in the template image, the ratio of the number of successfully matched first areas to the total number of second areas in the template image is used to determine the matching degree between the first visible light image and the template image. In this way, the overall matching degree is determined based on the ratio of the number of successfully matched areas to the total number of areas. The method is intuitive and easy to implement, which effectively improves the reliability and stability of the matching results.

[0161] As an implementation method of an embodiment of the present application, after the electronic device determines the first number of matching first areas through the above-mentioned step S201, it may not execute step S202, but may directly use the number of successfully matched first areas as the degree of matching between the first visible light image and the template image, or calculate the ratio of the number of successfully matched first areas to the total number of first areas in the first visible light image, and then use the ratio as the degree of matching between the first visible light image and the template image.

[0162] As an implementation method of the present application, Figure 4 As shown, after the above step S103, the above method may further include step S104:

[0163] S104, removing target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as a base point, enlarging the area where the target component is located according to a preset magnification factor and / or the size of the third area.

[0164] After determining the target components, the electronic device can further remove the target components with smaller sizes from the target components, and / or enlarge the area where the target components are located. The specific removal method and the specific area enlargement method are respectively introduced below:

[0165] As for the removal method, due to the noise or blur introduced in the process of collecting the first visible light image, or the false detection in the component detection, some smaller areas may appear in the area where the component is located in the first visible light image. These smaller areas may not include a component.

[0166] Therefore, after determining the target components, the electronic device can further remove the target components whose area has a size smaller than a preset size threshold from the determined target components.

[0167] For the area magnification method, due to the influence of factors such as unclear edge features, image noise, and uneven lighting, the area where the component is located obtained through component detection may not include the complete edge of the component, so the first area can be appropriately magnified to offset the influence of these external factors to a certain extent. In addition, if the target component is to be tested for temperature later, if the abnormally high temperature of the component happens to occur at the edge of the component, then when the first area is small and cannot include the complete edge of the component, the temperature anomaly will be missed.

[0168] Therefore, after determining the target component, the electronic device can enlarge the area where the target component is located with the center point of the area where the target component is located as a base point.

[0169] In one implementation, when the electronic device magnifies the area where the target component is located, it can directly magnify it according to a preset magnification factor.

[0170] In another embodiment, in the stage of generating a template image, after obtaining the second area of ​​the component in the second visible light image, the second area can be identified in the second visible light image by means of a target box, and the identification result can be displayed to the user. At this time, the user can enlarge the second area in the second visible light image by enlarging the target box.

[0171] On this basis, when the electronic device enlarges the area where the target component is located, it can determine the area with the greatest match with the area where the target component is located in the area where the component is located in the target template image, and use it as the third area. The electronic device can then enlarge the area where the target component is located according to the size of the third area, that is, enlarge the size of the area where the target component is located to the same size as the third area.

[0172] In the solution provided by the embodiment of the present application, after determining the target components, the electronic device effectively eliminates small areas of false detection by removing the target components whose sizes are smaller than the preset size threshold. At the same time, by enlarging the area where the target components are located, it is ensured that the edges of the components are completely covered, which solves the problem of incomplete areas caused by factors such as edge blur and noise to a certain extent, and improves the comprehensiveness and accuracy of detection. In particular, for temperature detection scenarios, the amplification process avoids the omission of temperature anomalies due to edge omissions, and enhances the reliability of the system. In addition, two methods of amplification based on user interaction and amplification based on a preset amplification factor are provided, which further enhances the flexibility and adaptability of the solution.

[0173] In a specific application scenario, after detecting the target component in the first visible light image of the PCB board to be detected, the electronic device can further perform temperature detection on the target component. Figure 5-Figure 8 Introduce the specific method of temperature detection.

[0174] As an implementation method of the present application, Figure 5 As shown, the above method may also include:

[0175] S501, acquiring a thermal imaging image of a PCB board to be inspected that is acquired synchronously with a first visible light image.

[0176] In a scenario where the temperature of components on a PCB needs to be detected, while a visible light camera is used to collect a visible light image of the PCB to be detected, a thermal imaging camera can also be used to collect a thermal imaging image of the PCB to be detected. In this process, multiple visible light images and multiple thermal imaging images of the PCB to be detected can be obtained.

[0177] Therefore, after acquiring the first visible light image of the PCB board to be inspected, the electronic device can acquire a thermal imaging image having the same acquisition timestamp as the acquisition timestamp of the first visible light image according to the acquisition timestamp of the first visible light image, thereby obtaining a thermal imaging image of the PCB board to be inspected that is acquired synchronously with the first visible light image.

[0178] S502: Determine, in the thermal imaging image, a fourth area corresponding to the position of the area where the target component is located.

[0179] After determining the region where the target component is located in the first visible light image, the electronic device may determine, in the image coordinate system of the thermal imaging image, a region corresponding to the position of the region where the target component is located as the fourth region.

[0180] Regarding the method of determining the fourth area in the thermal imaging image, the electronic device may implement it in, but not limited to, the following two methods:

[0181] In the first method, the correspondence between the pixel points in the image captured by the visible light camera and the pixel points in the image captured by the thermal imaging camera can be determined in advance based on the parameters of the visible light camera that captures the first visible light image and the parameters of the thermal imaging camera that captures the thermal imaging image.

[0182] In this way, after obtaining the first visible light image and the thermal imaging image, the electronic device can map the position of the area where the target component is located in the first visible light image to the image coordinate system of the thermal imaging image according to the correspondence between the above-mentioned pixel points, thereby determining the fourth area corresponding to the position of the area where the target component is located in the thermal imaging image.

[0183] In the second method, after obtaining the original visible light image and thermal imaging image of the PCB to be inspected, the electronic device can use the dual-light fusion registration parameters to scale and crop the original visible light image to obtain an image consistent with the field of view of the thermal imaging image, and use it as the first visible light image of the PCB to be inspected. And when performing component inspection on the first visible light image of the PCB to be inspected, the coordinate system where the regional coordinates of the area where the components are located in the first visible light image can be normalized, that is, the regional coordinates of the area where the components are located in the detected first visible light image are normalized coordinates, and the coordinate values ​​are within the range of [0,1].

[0184] In this way, after obtaining the first visible light image and the thermal imaging image, the electronic device can first denormalize the regional coordinates of the area where the target component is located according to the resolution of the thermal imaging image, and then determine the fourth area corresponding to the position of the first area of ​​the target component in the coordinate system corresponding to the thermal imaging image according to the denormalized regional coordinates.

[0185] S503: Determine the temperature of the target component according to the image data of the fourth area.

[0186] After the electronic device determines the fourth area, because the grayscale value in the fourth area is positively correlated with the temperature, that is, the greater the grayscale, the higher the temperature, the electronic device can determine the temperature of the target component based on the image data in the fourth area (that is, the grayscale value in the fourth area).

[0187] In one implementation, the electronic device may first calculate the average grayscale value of each pixel in the fourth area, and then convert the average grayscale value into temperature, thereby determining the temperature of the target component.

[0188] In another embodiment, in order to ensure the accuracy of the measured temperature, the electronic device may first determine the maximum value of the grayscale value in the fourth area according to the grayscale value of each pixel in the fourth area, and then calculate the average value of the maximum grayscale value and the surrounding grayscale values, and finally convert the average value into temperature, thereby determining the temperature of the target component. Of course, after obtaining the maximum grayscale value, the electronic device may also first convert the maximum grayscale value and the surrounding grayscale values ​​of the maximum grayscale value into temperatures, and then determine the average value of the temperatures as the temperature of the target component.

[0189] As for the method of converting grayscale values ​​into temperature, the thermal imaging camera that collects thermal imaging images can be calibrated in advance to determine the corresponding relationship between the grayscale value and the temperature in the thermal imaging image collected by the thermal imaging camera, thereby ensuring that the grayscale value output by the thermal imaging camera can be accurately mapped to the actual temperature value. In this way, in the subsequent temperature detection process, after the maximum grayscale value is determined from the fourth area, the maximum grayscale value can be converted into the actual temperature according to the pre-calibrated corresponding relationship.

[0190] S504, detecting whether the temperature of the target component is abnormal.

[0191] After obtaining the temperature of the target component, the electronic device can determine whether the temperature of the target component is abnormal according to whether the temperature of the target component is greater than the alarm temperature. Among them, each target component can correspond to the same alarm temperature or different alarm temperatures (for example, different types of components correspond to different alarm temperatures), which is not specifically limited here.

[0192] In the solution provided in the embodiment of the present application, by synchronously collecting the visible light image and the thermal imaging image of the PCB board to be detected, accurate detection of the temperature of the components is achieved. In this process, through the predetermined pixel point correspondence or image preprocessing steps, the fourth area corresponding to the target component can be quickly located in the thermal imaging image, which greatly simplifies the operation process.

[0193] As an implementation method of the present application, Figure 6 As shown, the above step S503 may include:

[0194] S601, in the thermal imaging image, determining a maximum grayscale value in a fourth area.

[0195] In the process of temperature detection, we mainly focus on whether the temperature of the target component is abnormally high. Given the positive correlation between grayscale value and temperature in thermal imaging images, that is, the larger the grayscale value, the higher the temperature represented, therefore, when processing thermal imaging data, we do not need to convert all grayscale values ​​in the fourth area into temperature values. Instead, we directly locate and extract the maximum grayscale value in the fourth area. This maximum grayscale value can intuitively reflect the highest temperature point of the target component.

[0196] S602, converting the maximum grayscale value into temperature as the temperature of the target component.

[0197] After obtaining the maximum grayscale value, the electronic device may only convert the maximum grayscale value into an actual temperature, and then determine the temperature obtained after the conversion as the temperature of the target component.

[0198] In the solution provided in the embodiment of the present application, the efficiency of temperature detection is significantly improved by directly locating and extracting the maximum grayscale value of the area corresponding to the target component in the thermal imaging image. Since the grayscale value is positively correlated with the temperature, the maximum grayscale value directly reflects the highest temperature point of the target component, which is crucial for determining whether the component is overheated.

[0199] As an implementation method of the present application, Figure 7 As shown, the above step S503 may include:

[0200] S701: Obtain temperature measurement parameters of components in the third area.

[0201] For components on PCB boards, different types of components have different characteristics, which may lead to some differences in converting grayscale values ​​to temperature for different types of components. For example, different types of components have different parameters such as emissivity and reflection temperature, and for the same type of components, different materials, coatings and surface treatment processes will cause different parameters such as emissivity and reflection temperature of the components.

[0202] Therefore, in order to ensure the accuracy of temperature detection, the emissivity of the components in each second area in the template image can be preset in the generation of the template image, wherein the emissivity of the components represents its ability to emit infrared radiation energy. In addition, the reflected temperature of the components in each second area in the template image can also be preset in the generation of the template image.

[0203] Accordingly, in order to ensure the accuracy of temperature detection, when converting the grayscale value in the fourth area into temperature, the electronic device can first determine the area with the greatest matching degree with the area where the target component is located in the area where the component is located in the target template image, as the third area, and then obtain the preset emissivity and preset reflection temperature of the component in the third area from the target template image, thereby obtaining two types of factors that have a greater impact on temperature detection. Among them, the preset reflection temperature of the component can be a preset value. If the value is not preset, the ambient temperature when the thermal imaging image is obtained can be used as the preset reflection temperature.

[0204] S702: Convert the grayscale value in the fourth area into temperature according to the temperature measurement parameter as the temperature of the target component.

[0205] In one embodiment, the correspondence between the grayscale value and the temperature can be pre-calibrated at different emissivities and / or different reflection temperatures. For example, at the first emissivity and the first reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the first correspondence, at the first emissivity and the second reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the second correspondence... At the Nth emissivity and the Gth reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the Xth correspondence.

[0206] In this way, after obtaining the preset emissivity and the preset reflection temperature of the components in the third area, the electronic device can determine the target correspondence relationship between the preset emissivity and the preset reflection temperature from the multiple sets of correspondence relationships between the pre-calibrated grayscale values ​​and the temperatures. After obtaining the target correspondence relationship, the electronic device can convert the grayscale value in the fourth area into a temperature according to the target correspondence relationship as the temperature of the target component.

[0207] In another embodiment, the grayscale value and the corresponding temperature may be collected in advance at different emissivities and / or different reflection temperatures, and then a calculation formula between the emissivity, reflection temperature, grayscale value and temperature may be fitted based on the collected data.

[0208] In this way, after obtaining the preset emissivity and preset reflection temperature of the components in the third area, the electronic device can calculate the corresponding temperature according to the preset emissivity, preset reflection temperature and the grayscale value in the fourth area according to the above calculation formula as the temperature of the target component.

[0209] In the solution provided by the embodiment of the present application, by obtaining the preset emissivity and the preset reflection temperature of the components in the third area, two types of factors that affect the accuracy of temperature detection are fully considered, thereby ensuring the accuracy of temperature conversion. Secondly, by pre-calibrating the correspondence between grayscale values ​​and temperatures under different conditions, or pre-calibrating the calculation formula between grayscale values ​​and temperatures, a comprehensive correspondence database is established, so that electronic equipment can quickly realize accurate conversion of grayscale values ​​to temperature according to actual detection conditions.

[0210] As an implementation method of the present application, Figure 8 As shown, the above step S504 may include:

[0211] S801: Obtain preset alarm temperatures of components in a third area.

[0212] For components on a PCB, different types of components have different operating temperature ranges due to differences in materials, structures, functions, etc. Some components may still work normally at high temperatures, while others may begin to degrade or become damaged at lower temperatures. Therefore, in order to ensure the accuracy of temperature detection, in generating a template image, for each second area in the template image, the alarm temperature of the second area may be pre-set according to the components in the second area.

[0213] Correspondingly, when the electronic device detects whether the temperature of a target component is abnormal, it can first obtain the alarm temperature corresponding to the target component. Specifically, the electronic device can first determine the area with the greatest match with the area where the target component is located in the area where the component is located in the target template image, as the third area, and then obtain the preset alarm temperature of the component in the third area as the alarm temperature of the target component.

[0214] S802, determine whether the temperature of the target component is greater than a preset alarm temperature. If yes, execute step S803, if not, execute step S804.

[0215] Because the preset alarm temperature of the third area is set for the components in the third area, and the components in the third area are the same as the target components, the electronic device can determine whether the temperature of the target component is greater than the preset alarm temperature after obtaining the preset alarm temperature of the third area.

[0216] S803, determining whether the temperature of the target component is abnormal.

[0217] If it is greater than, it is determined that the temperature of the target component is abnormal, and temperature alarm information for the target component can be output.

[0218] S804, determining that the temperature of the target component is normal.

[0219] If it is not greater than, it is determined that the temperature of the target component is normal. In this case, no information may be output, or normal temperature information for the target component may be output.

[0220] In the solution provided by the embodiment of the present application, the differences between components are fully considered, and the corresponding alarm temperature is set for the working characteristics of each component. By matching the target component with the area in the template image, the preset alarm temperature corresponding to the target component is quickly obtained, thereby improving the accuracy and pertinence of temperature detection. Finally, based on the comparison result between the actual temperature of the target component and the preset alarm temperature, the temperature abnormality or normal information is output in time, which provides strong support for subsequent troubleshooting and maintenance, and realizes PDA (PCB Detection & Alarm).

[0221] As an implementation method of an embodiment of the present application, each second area in the generated template image may have a unique ID. When the first area in the first visible light image is subsequently matched with the second area in the target template image, for each successfully matched first area, the ID of the second area in the target template image with the highest degree of match with the first area may be assigned to the first area, thereby generating a unique ID for each successfully matched first area in the first visible light image.

[0222] For example, the first regions that are successfully matched in the first visible light image are region a and region b. For region a, the second region with the highest matching degree with region a in the target template image is region A. For region b, the second region with the highest matching degree with region b in the target template image is region B. Assuming that the ID of region A is "0005" and the ID of region B is "0007", ID "0005" can be assigned to region a and ID "0007" can be assigned to region b.

[0223] Correspondingly, when the electronic device obtains the preset emissivity, preset reflection temperature and preset alarm temperature of the target component in the first visible light image, it can first obtain the ID of the first area of ​​the target component, and then determine a second area in the target template image whose ID is the same as the ID of the first area as the third area corresponding to the first area, and then obtain the preset emissivity, preset reflection temperature and preset alarm temperature of the component in the third area.

[0224] In the solution provided in the embodiment of the present application, when the electronic device matches the first visible light image with the target template image, it can associate the first area in the first visible light image with the second area in the target template image through the ID, so that when the temperature measurement parameters and alarm temperature are subsequently obtained, they can be directly obtained through the ID, thereby improving the efficiency of the execution of the solution.

[0225] Corresponding to the above-mentioned PCB component detection method, the embodiment of the present application further provides a PCB component detection device. The PCB component detection device provided by the embodiment of the present application is introduced below.

[0226] like Fig. 9 As shown, a PCB component detection device includes:

[0227] A matching module 901 is used to calculate the matching degree between the first visible light image of the PCB board to be inspected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board;

[0228] An image selection module 902 is used to select a target template image from each template image based on the obtained matching degree;

[0229] The component determination module 903 is used to determine the components included in the target area in the first visible light image as target components, wherein the target area is an area that matches the area where the components in the target template image are located.

[0230] In the scheme provided by the embodiment of the present application, the electronic device can calculate the matching degree between the first visible light image of the PCB board to be detected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, the template image that is closer to the first visible light image can be selected as a reference through the matching degree between the first visible light image of the PCB board to be detected and each template image. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection.

[0231] As an implementation of the embodiment of the present application, the matching module 901 may include:

[0232] A component detection unit, used for performing component detection on the first visible light image of the PCB board to be detected, and obtaining a first region of components in the first visible light image;

[0233] The region matching unit is used to match the obtained first region with the second region of the component in each template image, and obtain the matching degree between the first visible light image and each template image based on the matching result.

[0234] As an implementation of the embodiment of the present application, the region matching unit may include:

[0235] A matching number determination subunit is used to match the obtained first area with the second area of ​​the component in the template image for each template image, and obtain a first number of first areas that are successfully matched;

[0236] The matching degree calculation subunit is used to calculate the ratio of the first number corresponding to the template image output by the matching number determination subunit to the total number of second areas in the template image for each template image, and obtain the matching degree between the first visible light image and the template image based on the ratio.

[0237] As an implementation of an embodiment of the present application, the above-mentioned matching quantity determination subunit can be specifically used for:

[0238] For each first area, the matching degree between the first area and the second area of ​​the component in the template image is calculated respectively. If the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, the first area is determined to be a successfully matched first area; and a first number of successfully matched first areas is obtained.

[0239] As an implementation of the embodiment of the present application, the matching degree between the first area and the second area may be calculated in the following manner:

[0240] Obtaining a first area where the first area intersects the second area;

[0241] Get the difference between the sum of the areas of the first and second areas and the first area as the second area:

[0242] The ratio of the first area to the second area is calculated as the matching degree between the first region and the second region.

[0243] As an implementation of the embodiment of the present application, the component determination module 903 may include:

[0244] A region screening unit, configured to screen out, from the region where the component in the first visible light image is located, a region whose matching degree with the region where the component in the target template image is located is greater than a second preset matching degree threshold value as a target region;

[0245] The target component determination unit is used to determine the components included in the target area as target components.

[0246] As an implementation of the embodiment of the present application, the above device may further include:

[0247] A component adjustment module is used to remove target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as a base point, enlarge the area where the target component is located according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

[0248] As an implementation of the embodiment of the present application, the above device may further include:

[0249] A thermal imaging image acquisition module, used to acquire a thermal imaging image of the PCB board to be inspected which is acquired synchronously with the first visible light image;

[0250] A region determination module, used to determine, in the thermal imaging image, a fourth region corresponding to the position of the region where the target component is located;

[0251] a temperature determination module, used to determine the temperature of the target component according to the image data of the fourth area;

[0252] The temperature detection module is used to detect whether the temperature of the target component is abnormal.

[0253] As an implementation of the embodiment of the present application, the image data of the fourth area includes a grayscale value, and the temperature determination module can be specifically used for:

[0254] Obtain temperature measurement parameters of components in the third area, wherein the third area is the area where the components in the target template image are located, and has the greatest matching degree with the area where the target components are located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature; according to the temperature measurement parameters, convert the grayscale values ​​in the fourth area into temperature as the temperature of the target component.

[0255] As an implementation of the embodiment of the present application, the image data of the fourth area includes a grayscale value, and the temperature determination module can be specifically used for:

[0256] In the thermal imaging image, a maximum grayscale value in the fourth area is determined; and the maximum grayscale value is converted into a temperature as the temperature of the target component.

[0257] As an implementation of the embodiment of the present application, the temperature detection module may include:

[0258] An alarm temperature acquisition unit, used to acquire a preset alarm temperature of a component in a third area, wherein the third area is an area where the component in the target template image is located and has the highest matching degree with the area where the target component is located;

[0259] The temperature detection unit is used to determine whether the temperature of the target component is greater than a preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

[0260] As an implementation of the embodiment of the present application, the template image generation method may include:

[0261] For the second visible light image of the template PCB board, perform the following operations:

[0262] Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction;

[0263] and / or,

[0264] Based on parameter information set for the initial region where the components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

[0265] The present application also provides an electronic device, such as Fig.10 As shown, including:

[0266] Memory 1001, used for storing computer programs;

[0267] The processor 1002 is used to implement the PCB component detection method described in any of the above embodiments when executing the program stored in the memory 1001.

[0268] In the scheme provided by the embodiment of the present application, the electronic device can calculate the matching degree between the first visible light image of the PCB board to be detected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, the template image that is closer to the first visible light image can be selected as a reference through the matching degree between the first visible light image of the PCB board to be detected and each template image. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection.

[0269] As an implementation method of the present application, the electronic device may be a detection device, such as Fig.11 As shown, the detection device includes, in addition to the memory 1001 and the processor 1002, a first camera 1003 and a second camera 1004. The first camera 1003 is used to collect visible light images of the PCB board to be detected, and the second camera 1004 is used to collect thermal imaging images of the PCB board to be detected.

[0270] In one embodiment, the detection device may also have a display screen. The detection device can display the first area of ​​the component in the first visible light image on the display screen, and can also display the temperature detection results of the target component on the display screen. It can also display various template images for user adjustment, etc. There is no specific limitation on the content displayed on the display screen.

[0271] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1002, the communication interface, and the memory 1001 communicate with each other via the communication bus.

[0272] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0273] The communication interface is used for communication between the above electronic device and other devices.

[0274] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0275] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0276] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the PCB component detection method described in any of the above embodiments is implemented.

[0277] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the PCB component detection method described in any of the above embodiments.

[0278] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0279] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0280] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0281] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A PCB component detection method, characterized in that: The method comprises: Calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; Selecting a target template image from each template image based on the obtained matching degree; Components included in a target area in the first visible light image are determined as target components, wherein the target area is an area that matches an area where components in the target template image are located.

2. The method according to claim 1, characterized in that The calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image includes: Perform component detection on the first visible light image of the PCB board to be detected to obtain a first region of components in the first visible light image; The obtained first regions are matched with the second regions of the components in each template image, and the matching degrees between the first visible light image and each template image are obtained based on the matching results.

3. The method according to claim 2, characterized in that The step of matching the obtained first region with the second region of the component in each template image, and obtaining the matching degree between the first visible light image and each template image based on the matching result, includes: For each template image, the matching degree between the first visible light image and the template image is obtained in the following manner: Matching the obtained first region with the second region of the component in the template image to obtain a first number of successfully matched first regions; A ratio of the first number to the total number of second regions in the template image is calculated, and a matching degree between the first visible light image and the template image is obtained based on the ratio.

4. The method according to claim 3, characterized in that The step of matching the obtained first region with the second region of the component in the template image to obtain a first number of successfully matched first regions includes: For each first region, respectively calculate the matching degree between the first region and the second region of the component in the template image, and if the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, determine that the first region is a successfully matched first region; A first number of first regions that are matched successfully is obtained.

5. The method according to claim 4, characterized in that The matching degree between the first region and the second region is calculated as follows: Obtaining a first area where the first area intersects the second area; The difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area: A ratio of the first area to the second area is calculated as a matching degree between the first region and the second region.

6. The method according to claim 1, characterized in that The step of determining the components included in the target area in the first visible light image as target components includes: In the region where the components in the first visible light image are located, a region whose matching degree with the region where the components in the target template image are located is greater than a second preset matching degree threshold is selected as the target region; The components included in the target area are determined as target components.

7. The method according to claim 1, characterized in that After determining the components included in the target area in the first visible light image as target components, the method further includes: From the determined target components, remove the target components whose area size is smaller than a preset size threshold; and / or, Taking the center point of the area where the target component is located as the base point, the area where the target component is located is enlarged according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Acquire a thermal imaging image of the PCB board to be inspected that is acquired synchronously with the first visible light image; In the thermal imaging image, determining a fourth area corresponding to the position of the area where the target component is located; determining the temperature of the target component according to the image data of the fourth area; Detect whether the temperature of the target component is abnormal.

9. The method according to claim 8, characterized in that The image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes: Acquire temperature measurement parameters of components in a third region, wherein the third region is a region where components in the target template image are located and has the highest matching degree with the region where the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature; According to the temperature measurement parameter, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

10. The method according to claim 8, characterized in that The image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes: In the thermal imaging image, determining a maximum grayscale value in the fourth area; The maximum grayscale value is converted into a temperature as the temperature of the target component.

11. The method according to claim 8, characterized in that The detecting whether the temperature of the target component is abnormal includes: Acquire a preset alarm temperature of a component in a third area, wherein the third area is an area in the target template image where the component is located and has the highest matching degree with the area where the target component is located; It is determined whether the temperature of the target component is greater than the preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

12. The method according to any one of claims 1 to 7, characterized in that: The template image is generated in a manner including: For the second visible light image of the template PCB board, perform the following operations: Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction; and / or, Based on parameter information set for an initial region where components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

13. A PCB component detection device, characterized in that: The device comprises: A matching module, used for calculating the matching degree between the first visible light image of the PCB board to be inspected and each template image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; An image selection module, used for selecting a target template image from each template image based on the obtained matching degree; The component determination module is used to determine the components included in the target area in the first visible light image as target components, wherein the target area is an area that matches the area where the components in the target template image are located.

14. The device according to claim 13, characterized in that The matching module includes: a component detection unit, used to perform component detection on the first visible light image of the PCB board to be detected, and obtain a first region of components in the first visible light image; a region matching unit, used to match the obtained first region with the second region of components in each template image, and obtain the matching degree between the first visible light image and each template image based on the matching result; and / or, The region matching unit includes: a matching number determining subunit, which is used to match the obtained first region with the second region of the component in the template image for each template image, and obtain a first number of first regions that are successfully matched; a matching degree calculating subunit, which is used to calculate, for each template image, a ratio of the first number corresponding to the template image output by the matching number determining subunit to the total number of second regions in the template image, and obtain a matching degree between the first visible light image and the template image based on the ratio; and / or, The matching number determination subunit is specifically used to calculate, for each first area, the matching degree between the first area and the second area of ​​the component in the template image, and if the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, determine that the first area is a first area that has successfully matched; and obtain a first number of first areas that have successfully matched; and / or, The matching degree between the first region and the second region is calculated in the following manner: a first area where the first region and the second region intersect is obtained; a difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area; and a ratio of the first area to the second area is calculated as the matching degree between the first region and the second region. and / or, The component determination module includes: a region screening unit, used to screen out a region where the component in the first visible light image is located, and the region whose matching degree with the region where the component in the target template image is located is greater than a second preset matching degree threshold, as a target region; a target component determination unit, used to determine the component included in the target region as a target component; and / or, The device further includes: the component adjustment module, which is used to remove the target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as a base point, amplifying the area where the target component is located according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the components in the target template image are located, and has the greatest matching degree with the area where the target component is located; and / or, The device further includes: a thermal imaging image acquisition module, used to acquire a thermal imaging image of the PCB board to be detected that is acquired synchronously with the first visible light image; a region determination module, used to determine, in the thermal imaging image, a fourth region corresponding to the position of the region where the target component is located; a temperature determination module, used to determine the temperature of the target component based on image data of the fourth region; and a temperature detection module, used to detect whether the temperature of the target component is abnormal. and / or, The image data of the fourth area includes grayscale values. The temperature determination module is specifically used to obtain temperature measurement parameters of components in the third area, wherein the third area is an area where components in the target template image are located, and the area with the greatest matching degree with the area where the target component is located. The temperature measurement parameter is a parameter used to convert grayscale values ​​into temperature. According to the temperature measurement parameter, the grayscale value in the fourth area is converted into temperature as the temperature of the target component. and / or, The image data of the fourth area includes grayscale values, and the temperature determination module is specifically used to determine the maximum grayscale value in the fourth area in the thermal imaging image; convert the maximum grayscale value into temperature as the temperature of the target component; and / or, The temperature detection module includes: an alarm temperature acquisition unit, used to acquire a preset alarm temperature of a component in a third area, wherein the third area is an area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located; a temperature detection unit, used to determine whether the temperature of the target component is greater than the preset alarm temperature, if so, determining that the temperature of the target component is abnormal, and if not, determining that the temperature of the target component is normal; and / or, The method for generating the template image includes: for a second visible light image of a template PCB board, performing the following operations: adjusting the initial area in the second visible light image based on an adjustment instruction for the initial area where the components in the second visible light image are located, wherein the adjustment instruction includes: adding instructions, deleting instructions and / or resizing instructions; and / or, based on parameter information set for the initial area where the components in the second visible light image are located, establishing a correspondence between the initial area in the second visible light image and the parameter information, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

15. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-12 when executing a program stored in a memory.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images