PCB component detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a PCB component detection method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the zooming and translation of a template image and / or a first visible light image of a to-be-detected PCB for each template image, so as to reduce the deviation between the view field and the view angle of the collected template image and the first visible light image, and the matching degree between the two is further calculated. Selecting a target template image based on the obtained matching degree; and determining a component included in an area matched with the area where the component in the target template image is located in the first visible light image as a target component. As the area where the component is located in the pre-generated template image is relatively accurate, the accuracy of component detection can be remarkably improved by determining the target component based on the template image. Automatic image alignment is realized through zooming and translation, so that the collection of the first visible light image is not limited by the collection view field and view angle of the template image any more.

Description

Technical Field

[0001] The present application relates to the technical field of PCB detection, and in particular to a PCB component detection method, device, electronic equipment and storage medium. Background Art

[0002] With the continuous improvement of the integration of electronic products, the density and complexity of components included in PCB (Printed Circuit Board) boards have also increased significantly, which has put higher requirements on the detection of components.

[0003] The existing PCB component detection method is to collect images of PCB boards, determine the edge information of components in the image through models such as CNN (Convolutional Neural Network), and thus determine the area of ​​components in the image to complete component detection. However, due to factors such as edge detection accuracy, when using the above method for PCB component detection, there will be false detections, resulting in low detection accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a PCB component detection method, device, electronic device and storage medium to improve the accuracy of component detection. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a PCB component detection method, the method comprising:

[0006] For each template image, scaling and translating the template image and / or the first visible light image of the PCB to be inspected to obtain a scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of collecting the template image and the field of view and the viewing angle of collecting the first visible light image, wherein the template image is a visible light image of the template PCB and includes components of interest to the user in the template PCB;

[0007] For each template image, based on the scaling and translation result corresponding to the template image, calculating the matching degree between the first visible light image and the template image;

[0008] Selecting a target template image from each template image based on the obtained matching degree;

[0009] Components included in a target area in the first visible light image are determined as target components, wherein the target area is an area that matches an area where components in the target template image are located.

[0010] Optionally, before scaling and translating the template image and / or the first visible light image of the PCB board to be inspected for each template image, the method further includes:

[0011] Perform component detection on the first visible light image of the PCB board to be detected to obtain a first region of components in the first visible light image;

[0012] For each template image, scaling and translating the template image and / or the first visible light image of the PCB board to be detected to obtain a scaling and translation result corresponding to the template image includes:

[0013] For each template image, based on the ratio between the acquisition distance of the template image and the acquisition distance of the first visible light image, the first area of ​​the component in the first visible light image and / or the second area of ​​the component in the template image are scaled, and based on the scaling result, the first area of ​​the component in the first visible light image is translated and / or the second area of ​​the component in the template image is translated, so that the first area in the first visible light image at the first orientation is aligned with the second area in the template image at the first orientation, and the scaling and translation result corresponding to the template image is obtained.

[0014] Optionally, based on the scaling result, translating the first region of the component in the first visible light image and / or translating the second region of the component in the template image so that the first region in the first visible light image at the first orientation is aligned with the second region in the template image at the first orientation, includes:

[0015] Based on the scaling result corresponding to the template image, determining a first region in the first visible light image at a first orientation, and determining a second region in the template image at the first orientation;

[0016] In a first coordinate system of the first visible light image, overall translation is performed on all first regions in the first visible light image so that the first regions at the first orientation are aligned with a first reference point in the first coordinate system;

[0017] In the second coordinate system of the template image, all second regions in the template image are translated as a whole, so that the second regions in the first orientation are aligned with the second reference points in the second coordinate system, wherein the first reference points are aligned with the second reference points.

[0018] Optionally, based on the scaling result, translating the first region of the component in the first visible light image and / or translating the second region of the component in the template image so that the first region in the first visible light image at the first orientation is aligned with the second region in the template image at the first orientation, includes:

[0019] Based on the scaling result corresponding to the template image, determining a first region in the first visible light image at a first orientation, and determining a second region in the template image at the first orientation;

[0020] In the coordinate system of the first visible light image, all the second areas in the template image are translated as a whole so that the second areas at the first position are aligned with the first areas at the first position in the first visible light image; or, in the coordinate system of the template image, all the first areas in the first visible light image are translated as a whole so that the first areas at the first position are aligned with the second areas at the first position in the template image.

[0021] Optionally, after calculating, for each template image, the degree of matching between the first visible light image and the template image based on the scaling and translation result corresponding to the template image, the method further includes:

[0022] Determining whether a maximum degree of match between the first visible light image and each template image is greater than a preset threshold;

[0023] If it is greater than, executing the step of selecting a target template image from each template image based on the obtained matching degree;

[0024] If it is not greater than, the first orientation is updated to another orientation among the preset multiple orientations except the current first orientation, and the step of translating the first area of ​​the component in the first visible light image and / or translating the second area of ​​the component in the template image based on the scaling result is returned to so that the first area in the first visible light image at the first orientation is aligned with the second area in the template image at the first orientation.

[0025] Optionally, for each template image, calculating the matching degree between the first visible light image and the template image based on the scaling and translation result corresponding to the template image includes:

[0026] For each template image, based on the scaling and translation result corresponding to the template image, the first area of ​​the component in the first visible light image is matched with the second area of ​​the component in the template image, and the matching degree between the first visible light image and the template image is obtained based on the matching result.

[0027] Optionally, for each template image, based on a zoom and translation result corresponding to the template image, matching a first region of the component in the first visible light image with a second region of the component in the template image, and obtaining a matching degree between the first visible light image and the template image based on the matching result, includes:

[0028] For each template image, the matching degree between the first visible light image and the template image is obtained in the following manner:

[0029] Based on the scaling and translation result corresponding to the template image, matching the first region of the component in the first visible light image with the second region of the component in the template image to obtain a first number of first regions that are successfully matched;

[0030] A ratio of the first number to the total number of second regions in the template image is calculated, and a matching degree between the first visible light image and the template image is obtained based on the ratio.

[0031] Optionally, matching the first region of the component in the first visible light image with the second region of the component in the template image to obtain a first number of successfully matched first regions includes:

[0032] For each first region, respectively calculate the matching degree between the first region and the second region of the component in the template image, and if the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, determine that the first region is a successfully matched first region;

[0033] A first number of first regions that are successfully matched is obtained.

[0034] Optionally, the matching degree between the first region and the second region is calculated in the following manner:

[0035] Obtaining a first area where the first area intersects the second area;

[0036] The difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area:

[0037] A ratio of the first area to the second area is calculated as a matching degree between the first region and the second region.

[0038] Optionally, determining the components included in the target area in the first visible light image as target components includes:

[0039] In the region where the components in the first visible light image are located, a region whose matching degree with the region where the components in the target template image are located is greater than a second preset matching degree threshold is selected as the target region;

[0040] The components included in the target area are determined as target components.

[0041] Optionally, after determining the components included in the target area in the first visible light image as target components, the method further includes:

[0042] From the determined target components, remove the target components whose area size is smaller than a preset size threshold;

[0043] and / or,

[0044] Taking the center point of the area where the target component is located as the base point, the area where the target component is located is enlarged according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

[0045] Optionally, the method further includes:

[0046] Acquire a thermal imaging image of the PCB board to be inspected that is acquired synchronously with the first visible light image;

[0047] In the thermal imaging image, determining a fourth area corresponding to the position of the area where the target component is located;

[0048] determining the temperature of the target component according to the image data of the fourth area;

[0049] Detect whether the temperature of the target component is abnormal.

[0050] Optionally, the image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes:

[0051] In the thermal imaging image, determining a maximum grayscale value in the fourth area;

[0052] The maximum grayscale value is converted into a temperature as the temperature of the target component.

[0053] Optionally, the image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes:

[0054] Acquire temperature measurement parameters of components in a third region, wherein the third region is a region where components in the target template image are located and has the highest matching degree with the region where the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature;

[0055] According to the temperature measurement parameter, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

[0056] Optionally, before converting the grayscale value in the fourth area into temperature according to the temperature measurement parameter and using the temperature as the temperature of the target component, the method further includes:

[0057] Acquiring a collection distance of the thermal imaging image;

[0058] The step of converting the grayscale value in the fourth area into temperature according to the temperature measurement parameter comprises:

[0059] According to the temperature measurement parameters and the acquisition distance of the thermal imaging image, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

[0060] Optionally, the detecting whether the temperature of the target component is abnormal includes:

[0061] Acquire a preset alarm temperature of a component in a third area, wherein the third area is an area in the target template image where the component is located and has the highest matching degree with the area where the target component is located;

[0062] It is determined whether the temperature of the target component is greater than the preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

[0063] Optionally, the template image is generated by:

[0064] For the second visible light image of the template PCB board, perform the following operations:

[0065] Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction;

[0066] and / or,

[0067] Based on parameter information set for an initial region where components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

[0068] In a second aspect, an embodiment of the present application provides a PCB component detection device, the device comprising:

[0069] A zooming and translation module is used for zooming and translating the template image and / or the first visible light image of the PCB board to be detected for each template image to obtain a zooming and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of collecting the template image and the field of view and the viewing angle of collecting the first visible light image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board;

[0070] A matching module, configured to calculate, for each template image, a degree of matching between the first visible light image and the template image based on a scaling and translation result corresponding to the template image;

[0071] An image selection module, used for selecting a target template image from each template image based on the obtained matching degree;

[0072] The component determination module is used to determine the components included in the target area in the first visible light image as target components, wherein the target area is an area that matches the area where the components in the target template image are located.

[0073] Optionally, the device further comprises:

[0074] A component detection module, used for performing component detection on the first visible light image of the PCB board to be detected, and obtaining a first region of components in the first visible light image;

[0075] The scaling and translation module is specifically used to scale, for each template image, a first area of ​​components in the first visible light image and / or a second area of ​​components in the template image based on a ratio between a collection distance of the template image and a collection distance of the first visible light image, and based on the scaling result, translate the first area of ​​components in the first visible light image and / or translate the second area of ​​components in the template image so that a first area in the first visible light image at a first orientation is aligned with a second area in the template image at a first orientation, thereby obtaining a scaling and translation result corresponding to the template image.

[0076] Optionally, the zoom and translation module includes:

[0077] a region determining unit, configured to determine a first region at a first position in the first visible light image and a second region at the first position in the template image based on a scaling result corresponding to the template image;

[0078] A first region translation unit, configured to translate all first regions in the first visible light image as a whole in a first coordinate system of the first visible light image so that the first regions in the first orientation are aligned with a first reference point in the first coordinate system;

[0079] The second area translation unit is used to translate all second areas in the template image as a whole in the second coordinate system of the template image, so that the second area in the first orientation is aligned with the second reference point in the second coordinate system, wherein the first reference point is aligned with the second reference point.

[0080] Optionally, the zoom and translation module includes:

[0081] a region determining unit, configured to determine a first region at a first position in the first visible light image and a second region at the first position in the template image based on a scaling result corresponding to the template image;

[0082] A region translation unit is used to translate all second regions in the template image as a whole in the coordinate system of the first visible light image, so that the second region at the first position is aligned with the first region at the first position in the first visible light image; or, in the coordinate system of the template image, to translate all first regions in the first visible light image as a whole, so that the first region at the first position is aligned with the second region at the first position in the template image.

[0083] Optionally, the device further comprises:

[0084] A judgment module is used to judge whether the maximum matching degree between the first visible light image and each template image is greater than a preset threshold; when the judgment result is yes, the execution of the image selection module is triggered; when the judgment result is no, the first orientation is updated to other orientations among the preset multiple orientations except the current first orientation, and the execution of the zoom and translation module is triggered.

[0085] Optionally, the matching module includes:

[0086] The area matching unit is used to match the first area of ​​the component in the first visible light image with the second area of ​​the component in the template image for each template image based on the scaling and translation result corresponding to the template image, and obtain the matching degree between the first visible light image and the template image based on the matching result.

[0087] Optionally, the region matching unit includes:

[0088] a matching number determination subunit, configured to match, for each template image, a first region of components in the first visible light image with a second region of components in the template image based on a scaling and translation result corresponding to the template image, to obtain a first number of first regions that are successfully matched;

[0089] The matching degree calculation subunit is used to calculate the ratio of the first number to the total number of second areas in the template image, and obtain the matching degree between the first visible light image and the template image based on the ratio.

[0090] Optionally, the matching quantity determination subunit is specifically used to:

[0091] For each first area, the matching degree between the first area and the second area of ​​the component in the template image is calculated respectively. If the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, the first area is determined to be a successfully matched first area; and a first number of successfully matched first areas is obtained.

[0092] Optionally, the matching degree between the first region and the second region is calculated in the following manner:

[0093] Obtaining a first area where the first area intersects the second area;

[0094] The difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area:

[0095] A ratio of the first area to the second area is calculated as a matching degree between the first region and the second region.

[0096] Optionally, the component determination module includes:

[0097] A region screening unit, configured to screen out, from the region where the component in the first visible light image is located, a region whose matching degree with the region where the component in the target template image is located is greater than a second preset matching degree threshold value as a target region;

[0098] The target component determination unit is used to determine the components included in the target area as target components.

[0099] Optionally, the device further comprises:

[0100] The component adjustment module is used to remove target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as the base point, enlarge the area where the target component is located according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

[0101] Optionally, the device further comprises:

[0102] A thermal imaging image acquisition module, used to acquire a thermal imaging image of the PCB board to be inspected which is acquired synchronously with the first visible light image;

[0103] A region determination module, used to determine, in the thermal imaging image, a fourth region corresponding to the position of the region where the target component is located;

[0104] A temperature determination module, used for determining the temperature of the target component according to the image data of the fourth area;

[0105] The temperature detection module is used to detect whether the temperature of the target component is abnormal.

[0106] Optionally, the temperature determination module is specifically used to: determine the maximum grayscale value in the fourth area in the thermal imaging image;

[0107] The maximum grayscale value is converted into a temperature as the temperature of the target component.

[0108] Optionally, the temperature determination module is specifically used to:

[0109] Acquire temperature measurement parameters of components in a third region, wherein the third region is a region where components in the target template image are located and has the highest matching degree with the region where the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature;

[0110] According to the temperature measurement parameter, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

[0111] Optionally, the device further comprises:

[0112] An acquisition distance acquisition module, used to acquire the acquisition distance of the thermal imaging image;

[0113] The temperature determination module is specifically configured to convert the grayscale value in the fourth area into a temperature as the temperature of the target component according to the temperature measurement parameter and the acquisition distance of the thermal imaging image.

[0114] Optionally, the temperature detection module includes:

[0115] An alarm temperature acquisition unit, used to acquire a preset alarm temperature of a component in a third area, wherein the third area is an area in the target template image where the component is located and has the highest matching degree with the area where the target component is located;

[0116] The temperature detection unit is used to determine whether the temperature of the target component is greater than the preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

[0117] Optionally, the template image is generated by:

[0118] For the second visible light image of the template PCB board, perform the following operations:

[0119] Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction;

[0120] and / or,

[0121] Based on parameter information set for an initial region where components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

[0122] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0123] Memory, used to store computer programs;

[0124] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.

[0125] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented.

[0126] Beneficial effects of the embodiments of the present application:

[0127] In the solution provided by the embodiment of the present application, the electronic device can scale and translate the template image and / or the first visible light image of the PCB board to be detected for each template image, and obtain the scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of the template image and the field of view and the viewing angle of the first visible light image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; for each template image, based on the scaling and translation result corresponding to the template image, calculate the matching degree between the first visible light image and the template image; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, through the matching degree between the first visible light image of the PCB board to be detected and each template image, a template image that is closer to the first visible light image can be selected as a reference. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection. In addition, before calculating the matching degree between the first visible light image and the template image, the consistency of the acquisition field of view and the acquisition angle of view of the first visible light image and the template image is ensured by scaling and translation operations, which provides a basis for accurate matching, so that the acquisition of the first visible light image is no longer limited to the acquisition field of view and acquisition angle of view that are completely consistent with the template image.

[0128] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0130] Figure 1 A schematic diagram of a flow chart of a PCB component detection method provided in an embodiment of the present application;

[0131] Figure 2 Based on Figure 1 A schematic diagram of the embodiment in which the template image and the first visible light image do not completely overlap in area;

[0132] Figure 3 Based on Figure 1 A schematic diagram of scaling the second area in the illustrated embodiment;

[0133] Figure 4 Based on Figure 1 A schematic diagram of a process flow of a translation method of the illustrated embodiment;

[0134] Figure 5 Based on Figure 4 A schematic diagram of the translation mode of the illustrated embodiment;

[0135] Figure 6 Based on Figure 1 A schematic flow chart of another translation method of the illustrated embodiment;

[0136] Figure 7a Based on Figure 6 A schematic diagram of the translation method of the illustrated embodiment;

[0137] Figure 7b Based on Figure 6 Another schematic diagram of the translation mode of the illustrated embodiment;

[0138] Figure 8 Based on Figure 1 A schematic diagram of a specific process of the PCB component detection method of the illustrated embodiment;

[0139] Fig. 9 Based on Figure 8 A schematic diagram of alignment failure of the illustrated embodiment;

[0140] Fig.10 Based on Figure 1 A schematic diagram of a flow chart of a method for obtaining a matching degree between images in the illustrated embodiment;

[0141] Fig.11 Based on Fig.10 A schematic diagram of the intersection-over-union ratio between regions in the illustrated embodiment;

[0142] Fig.12 Based on Figure 1 A schematic diagram of a specific process of the PCB component detection method of the illustrated embodiment;

[0143] Fig.13 Based on Figure 1 A schematic diagram of a flow chart of a temperature detection method of the illustrated embodiment;

[0144] Fig.14Based on Fig.13 A schematic diagram of a flow chart of a temperature determination method in the illustrated embodiment;

[0145] Fig.15 Based on Fig.13 A schematic flow chart of a method of converting a gray value into a temperature in the illustrated embodiment;

[0146] Fig.16 Based on Fig.13 A schematic diagram of a flow chart of a method for detecting whether the temperature is abnormal in the illustrated embodiment;

[0147] Fig.17 A schematic diagram of the structure of a PCB component detection device provided in an embodiment of the present application;

[0148] Fig.18 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0149] Fig.19 Based on Fig.18 A schematic diagram of a specific structure of an electronic device of the illustrated embodiment. DETAILED DESCRIPTION

[0150] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0151] In order to improve the accuracy of component detection, the embodiments of the present application provide a PCB component detection method, device, electronic device, computer-readable storage medium and computer program product. The following first introduces a PCB component detection method provided in the embodiments of the present application.

[0152] A PCB component detection method provided in an embodiment of the present application can be applied to any electronic device that needs to perform component detection on a PCB board. For example, the electronic device can be a fixed stand detection device, a handheld detection device, or a device with data processing capabilities connected to an image acquisition device, etc. No specific limitation is made here. For the sake of clear description, it is referred to as an electronic device below.

[0153] like Figure 1 As shown, a PCB component detection method includes:

[0154] S101, for each template image, scaling and translating the template image and / or the first visible light image of the PCB board to be inspected, to obtain a scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and viewing angle of acquiring the template image and the field of view and viewing angle of acquiring the first visible light image.

[0155] The template image is a visible light image of the template PCB board and includes components of interest to the user in the template PCB board.

[0156] It can be understood that the purpose of the so-called zooming and translation is because, when the handheld device collects the template image and the handheld device collects the first visible light image, the posture of the handheld device and the distance from the target are different, resulting in different collected field of view sizes and viewing angles. The purpose of zooming and translation in this application is to reduce or even eliminate the image differences caused by the different postures and distances of the above-mentioned handheld devices.

[0157] S102 : For each template image, based on the scaling and translation result corresponding to the template image, calculating the matching degree between the first visible light image and the template image.

[0158] S103: Select a target template image from each template image based on the obtained matching degree.

[0159] S104: Determine components included in the target area in the first visible light image as target components.

[0160] The target area is an area that matches the area where the components in the target template image are located.

[0161] In the solution provided by the embodiment of the present application, the electronic device can scale and translate the template image and / or the first visible light image of the PCB board to be detected for each template image, and obtain the scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of the template image and the field of view and the viewing angle of the first visible light image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; for each template image, based on the scaling and translation result corresponding to the template image, calculate the matching degree between the first visible light image and the template image; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, through the matching degree between the first visible light image of the PCB board to be detected and each template image, a template image that is closer to the first visible light image can be selected as a reference. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection. In addition, before calculating the matching degree between the first visible light image and the template image, the consistency of the acquisition field of view and the acquisition angle of view of the first visible light image and the template image is ensured by scaling and translation operations, which provides a basis for accurate matching, so that the acquisition of the first visible light image is no longer limited to the acquisition field of view and acquisition angle of view that are completely consistent with the template image.

[0162] Here, the method for generating the template image in the embodiment of the present application is first introduced, wherein the electronic device for generating the template image and the electronic device for executing the PCB component detection method provided in the embodiment of the present application can be the same device or different devices, and no specific limitation is made here.

[0163] First, in the actual scenario of inspecting PCB components, the types of PCB boards to be inspected are foreseeable, so these types of PCB boards can be determined as template PCB boards, and then the visible light image of the template PCB board is obtained, that is, the second visible light image is obtained.

[0164] For example, for a certain factory, the PCB boards it generates include: Class A PCB boards, Class B PCB boards and Class C PCB boards. It can be foreseen that when the factory detects PCB components, the PCB boards to be detected are likely to include at least one of Class A PCB boards, Class B PCB boards and Class C PCB boards. Therefore, Class A PCB boards, Class B PCB boards and Class C PCB boards can all be determined as template PCB boards, and then visible light images of these three types of PCB boards can be obtained.

[0165] The electronic device may then determine an initial region where the components in the second visible light image are located.

[0166] In one embodiment, the electronic device can perform component detection on the second visible light image to obtain a second detection result including an initial region where the component in the second visible light image is located (i.e., a second region of the component in the second visible light image), the type of the component, and / or the confidence of the detection result of the component. The second region of the component can be specifically characterized by the coordinates of the second region, and the component detection can be achieved by a pre-trained CNN or other detection model, or by an edge detection algorithm such as a Sobel operator or a Canny operator, which is not specifically limited here.

[0167] In another embodiment, the electronic device may also obtain a second detection result corresponding to the second visible light image manually determined by the user. Specifically, the second region of the component in the second visible light image included in the second detection result may be manually drawn by the user, the type of the component may be manually marked by the user, and the confidence of the component detection result may be the confidence of the drawing and marking result of the component manually input by the user.

[0168] Finally, after obtaining the second detection result of the second visible light image, the electronic device may perform at least one of the following operations on the second visible light image to obtain a template image:

[0169] The first operation: the electronic device adjusts the second area of ​​the component in the second visible light image based on the adjustment instruction for the second area of ​​the component in the second visible light image, wherein the adjustment instruction may include: a deletion instruction, an addition instruction and / or a size adjustment instruction, specifically:

[0170] For the deletion instruction, in one embodiment, the deletion instruction may be an instruction preset by the user, that is, the user may preset the type of components (e.g., resistors and / or capacitors, etc.), size thresholds, and / or confidence thresholds of the detection results that the user is interested in. On this basis, after obtaining the second detection result of the second visible light image, the electronic device may delete, in the second visible light image, components whose corresponding type is not the component type preset by the user, whose initial area where the component is located has a size smaller than the size threshold, and / or whose corresponding confidence is not greater than the preset confidence threshold.

[0171] For example, the user pre-sets the type of component he is interested in as a capacitor, the size threshold is "3 pixels long and 3 pixels wide", and the confidence threshold is 80%. Assume that after the second visible light image is detected for components, the detection results obtained include: the first component (type is a capacitor, the confidence is 88%, the length of the second area is 4 pixels and the width is 5 pixels), the second component (type is a resistor, the confidence is 90%, the length of the second area is 3 pixels and the width is 5 pixels), the third component (type is a capacitor, the confidence is 60%, the length of the second area is 4 pixels and the width is 4 pixels), and the fourth component is (type is a capacitor, the confidence is 81%, the length of the second area is 2 pixels and the width is 1 pixel). Then, according to the above-mentioned preset component type, size threshold and confidence threshold, the electronic device can delete the second component, the third component and the fourth component from the second visible light image. Among them, deleting the component can specifically be the detection result of deleting the component.

[0172] For the deletion instruction, in another embodiment, the deletion instruction can also be an instruction issued by the user in real time, that is, after the electronic device obtains the second area of ​​the component in the second visible light image, it can mark the second area in the second visible light image by means of a target frame, and display the marking result to the user. The user can select the second area to be deleted according to actual usage requirements, thereby generating a deletion instruction. Then the electronic device can delete the second area in the second visible light image based on the deletion instruction.

[0173] For example, after viewing the second regions displayed in the second visible light image, the user finds that one of the second regions does not contain any components, that is, the second region is a falsely detected region. Then, the user can manually identify the second region as a second region to be deleted, thereby generating a deletion instruction. When the electronic device receives the deletion instruction, it can delete the falsely detected second region.

[0174] For the adding instruction, when the electronic device displays the second area to the user, the user can add a new second area according to actual usage requirements, thereby generating an adding instruction. Then, the electronic device can add a new second area to the second visible light image based on the adding instruction.

[0175] For example, after viewing the second regions displayed in the second visible light image, the user finds that a component of interest has been missed, and can manually add a target frame at the component to identify the second region of the component. The electronic device can add a new second region to the second visible light image based on the adding instruction, that is, record information such as the location of the newly added second region in the detection result in the second visible light image.

[0176] For the size adjustment instruction, when the electronic device displays the second area to the user, the user can adjust the size of the displayed second area according to actual usage requirements, thereby generating a size adjustment instruction. The electronic device can then adjust the size information of the second area in the second visible light image based on the size adjustment instruction.

[0177] For example, after viewing each second area displayed in the second visible light image, the user finds that a second area does not completely cover the components in the second area, so the user can manually enlarge the size of the second area so that the second area completely covers the components in the second area. Then, the electronic device can update the size information of the second area in the detection result in the second visible light image based on the size adjustment instruction.

[0178] In the second operation, after obtaining the second region of the component in the second visible light image, the electronic device can identify the second region in the second visible light image by means of a target frame, and display the identification result to the user. Then, the user can set the temperature measurement parameters and / or alarm temperature and other parameter information of the second region according to the type of the component in the second region. After obtaining the parameter information set by the user for the second region in the second detection result, the electronic device can establish a corresponding relationship between the second region and the parameter information.

[0179] For example, after obtaining the second area of ​​each component in the second visible light image, the electronic device can identify each second area in the second visible light image by means of a target box, and display the identification result to the user. Assuming that the component in the first second area is a resistor and the component in the second second area is a capacitor, the user can set the temperature measurement parameters and alarm temperature corresponding to the first second area according to the characteristics of the resistor, and set the temperature measurement parameters and alarm temperature corresponding to the second second area according to the characteristics of the capacitor. After obtaining the above parameter information, the electronic device can establish a correspondence between the second area and the parameter information.

[0180] After obtaining the template images of each template PCB board in the above manner, each template image and the acquisition distance of each template image can be stored in an electronic device that executes the PCB component detection method provided in the embodiment of the present application.

[0181] The following will introduce Figure 1 PCB component detection method shown.

[0182] When it is necessary to inspect the PCB components included in the PCB board to be inspected, the electronic device can obtain a first visible light image of the PCB board to be inspected.

[0183] For example, when the electronic device is a fixed pedestal detection device, a handheld detection device or other detection device, the electronic device itself may include a camera, on this basis, the electronic device can collect the first visible light image of the PCB board to be detected through its own camera. For another example, when the electronic device is a device with data processing capabilities such as a computer, it can collect the first visible light image of the PCB board to be detected through an image acquisition device connected thereto.

[0184] After obtaining the first visible light image, if the electronic device directly matches the first visible light image with each template image, even if the various types of template PCBs corresponding to the template image include the PCB board to be detected, the matching may fail. This is because for the same PCB board, the image acquisition distance and acquisition direction (i.e., acquisition field of view and acquisition angle of view) when generating the template image of the PCB board may be inconsistent with the image acquisition distance and acquisition direction when detecting the PCB board.

[0185] For example, for Class A PCB boards, when generating a template image of Class A PCB boards, the image is collected at a distance of 20 cm from the Class A PCB boards, and the collection direction is facing the center point of the Class A PCB boards. However, when detecting Class A PCB boards, due to the influence of factors such as the accuracy of manually fixing Class A PCB boards and the shaking of handheld detection equipment, the first visible light image is collected at a distance of 30 cm from the Class A PCB boards, and the collection direction deviates from the center point of the Class A PCB boards. This leads to the following Figure 2 As shown, after the template image of the type A PCB board and the first visible light image are aligned, the areas where the components in the template image of the type A PCB board are located (area 1 and area 2) and the areas where the components in the first visible light image are located (area 1' and area 2') do not completely overlap.

[0186] Therefore, after obtaining the first visible light image, the electronic device can first perform zooming and translation operations to eliminate the influence of the acquisition distance and the acquisition direction, and then perform a matching operation. The zooming and translation operations in the embodiments of the present application will be introduced separately below.

[0187] For zoom operation.

[0188] Electronic devices can scale based on image proportions, specifically:

[0189] In one embodiment, for each template image, the electronic device may calculate the ratio of the template image to the first visible light image based on the size information of the template image and the size information of the first visible light image, and then scale the template image according to the ratio so that the template image is scaled to the same size as the first visible light image, or scale the first visible light image according to the ratio so that the first visible light image is scaled to the same size as the template image.

[0190] In another embodiment, the electronic device may calculate the ratio between the size information of each template image and the preset size information, and then scale each template image according to the ratio corresponding to each template image, so that the size information of each template image is scaled to the preset size information. At the same time, the electronic device may also calculate the ratio between the size information of the first visible light image and the preset size information, and then scale the first visible light image according to the ratio, so that the size information of the first visible light image is scaled to the preset size information.

[0191] Electronic devices can also be scaled based on acquisition distance, specifically:

[0192] In one implementation, for each template image, the electronic device may first calculate the ratio between the acquisition distance of the template image and the acquisition distance of the first visible light image, and then perform full-image scaling on the template image or the first visible light image according to the ratio.

[0193] In another embodiment, because the area where the components in the template image are located (i.e., the second area in the template image) is determined when the template image is generated in advance, the electronic device can first perform component detection on the first visible light image before scaling to obtain the area where the components in the first visible light image are located, i.e., obtain the first area of ​​the components in the first visible light image. Component detection can be achieved through a pre-trained CNN or other detection model, or through an edge detection algorithm such as a Sobel operator or a Canny operator, which is not specifically limited here.

[0194] On this basis, when the electronic device is scaling, for each template image, it can first calculate the ratio between the acquisition distance of the template image and the acquisition distance of the first visible light image, and then the electronic device can scale the size information of the second area in the template image and the distance between the center point of the second area and the center point of the template image according to the ratio, or scale the size information of the first area in the first visible light image and the distance between the center point of the first area and the center point of the first visible light image according to the ratio.

[0195] For example, the electronic device may scale the second area in the template image according to the following formula:

[0196] ;

[0197] in, represents the acquisition distance of the template image, Indicates the acquisition distance of the first visible light image.

[0198] ;

[0199] ;

[0200] ;

[0201] in, Indicates the original width of the second area, Indicates the width of the second area after scaling. represents the original height of the second region, Indicates the height of the second area after scaling. represents the original distance between the center point of the second region and the center point of the template image, Represents the scaled distance between the center point of the second region and the center point of the template image.

[0202] like Figure 3 As shown, the second area before scaling in the template image is as follows Figure 3 As shown in the left figure of the figure. The electronic device uses the above formula to calculate the original width of the second area in the template image. Original Height , the original distance between the center point of the second region and the center point of the template image Scale to get the scaled width of the second area Scaled height , the scaled distance between the center point of the second region and the center point of the template image At this time, the scaled second area in the template image is as follows Figure 3 As shown in the right figure.

[0203] For translation operation.

[0204] The electronic device can translate the image itself, specifically:

[0205] For each template image, after scaling the template image and / or the first visible light image, the electronic device can further translate the template image and / or the first visible light image based on the scaling result (that is, based on the two images obtained after scaling, the electronic device can further translate at least one of the two images) so that the area where the components in the first position in the first visible light image are located is aligned with the area where the components in the first position in the template image are located.

[0206] The electronic device can also translate the area where the components in the image are located, specifically:

[0207] After scaling the template image and / or the first visible light image, the electronic device can further translate the area where the components in the template image (i.e., the second area of ​​the components in the template image) and / or the area where the components in the first visible light image (i.e., the first area of ​​the components in the first visible light image) as a whole based on the scaling result (i.e., based on the two images obtained after scaling, the electronic device can further translate the area where the components in at least one of the two images are located) so that the first area in the first orientation in the first visible light image is aligned with the second area in the first orientation in the template image. The specific method of translating the area where the components in the image are located as a whole will be described in detail in subsequent embodiments and will not be repeated here.

[0208] The first orientation may be any orientation among a plurality of preset orientations, for example, the first orientation may be any orientation among the upper left, lower left, upper right, lower right, left side, right side, upper side and lower side. It should be noted that, taking the upper left as an example of the first orientation, when both areas are located in the upper left (one is closer to the top and the other is closer to the left), the electronic device may respectively calculate the distances between the two areas and the upper left vertex of the image, and determine the area with the shortest corresponding distance as the area in the first orientation.

[0209] The alignment of two regions may be the alignment of the center points of the two regions, or the alignment of the vertices of the two regions in the same orientation, for example, the center point of region A is aligned with the center point of region B, or the upper left vertex of region A is aligned with the upper left vertex of region B.

[0210] For each template image, the electronic device can perform the above-mentioned zooming and translation operations to obtain the zooming and translation results corresponding to the template image. The zooming and translation results corresponding to the template image include: the image obtained after zooming and translating the template image and / or the first visible light image. For example, for the first template image, when performing the zooming and translation operations, the first template image is first zoomed, and then the zoomed first template image and the first visible light image are translated. Then the zooming and translation results corresponding to the first template image include: the first template image after zooming and translating and the first visible light image after translating.

[0211] Then, the electronic device can calculate the matching degree between the first visible light image and the template image based on the scaling and translation result corresponding to the template image for each template image. The electronic device can implement “calculating the matching degree between the first visible light image and the template image” by, but not limited to, the following methods:

[0212] In one embodiment, for each template image, the electronic device can extract the image features of the first visible light image and the image features of the template image based on the scaling and translation results corresponding to the template image, and then the electronic device can calculate the similarity between the image features of the first visible light image and the image features of the template image. Thus, the electronic device can obtain the matching degree between the first visible light image and each template image.

[0213] In another embodiment, because the area where the components in the template image are located is determined when the template image is generated in advance, for each template image, the electronic device can also first perform component detection on the first visible light image based on the scaling and translation results corresponding to the template image to obtain the first area of ​​the components in the first visible light image, and then match the obtained first area with the second area of ​​the components in the template image, and obtain the matching degree between the first visible light image and the template image based on the matching result. Thus, the electronic device can also obtain the matching degree between the first visible light image and each template image. Among them, component detection can be achieved through pre-trained CNN and other detection models, or through edge detection algorithms such as Sobel operator and Canny operator, which are not specifically limited here.

[0214] After obtaining the matching degree between the first visible light image and each template image, the electronic device can select a target template image from each template image based on the obtained matching degree. For example, the template image with the greatest matching degree with the first visible light image is determined as the target template image, and the target template image is randomly determined from the template images with a matching degree with the first visible light image greater than a preset threshold, and so on.

[0215] Because the area where the components in the target template image are located (i.e., the second area) identifies the area where the components of interest to the user are located in the image of the template PCB board, for an area where a component in the first visible light image is located, if the degree of match between the area and the second area in the target template image is low, it means that the area may be a falsely detected area, or the components in the area are components that the user is not interested in.

[0216] Therefore, in obtaining the target template image, the electronic device determines the area matching the area where the components in the target template image are located from the area where the components in the first visible light image are located, that is, determines the target area, and then determines the components included in the target area in the first visible light image as the target components. The above-mentioned "components in the target template image" can be all components in the target template image or some components in the target template image, which is not specifically limited here.

[0217] Regarding “determining the target area”, the electronic device may implement it through, but not limited to, the following methods:

[0218] In one embodiment, the electronic device may filter out, from the region where the components in the first visible light image are located, a region whose matching degree with the region where the components in the target template image is located is greater than a second preset matching degree threshold, as the target region, based on the matching degree between the region where the components in the target template image and the region where the components in the first visible light image are located.

[0219] In another embodiment, the electronic device may map the position of the region where the components in the target template image are located to the image coordinate system of the first visible light image, and then use the region corresponding to the mapped position in the image coordinate system of the first visible light image as the target region.

[0220] In the solution provided by the embodiment of the present application, on the one hand, a template image that is relatively close to the first visible light image can be selected as a reference by the matching degree between the first visible light image of the PCB board to be detected and each template image. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image according to the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection. In addition, before calculating the matching degree between the first visible light image and the template image, the consistency of the acquisition field of view and the acquisition angle of view of the first visible light image and the template image is ensured by scaling and translation operations, which provides a basis for accurate matching, so that the acquisition of the first visible light image is no longer limited to the acquisition field of view and acquisition angle of view that are completely consistent with the template image.

[0221] As an implementation method of the present application, Figure 4 As shown, when the electronic device translates the area where the components in the image are located as a whole, it can be implemented specifically through steps S401-S403:

[0222] S401, based on a scaling result corresponding to a template image, determining a first region in a first visible light image at a first orientation, and determining a second region in the template image at the first orientation.

[0223] Before performing a translation operation based on the zoom result corresponding to the template image, the electronic device may first determine two regions in the same orientation in the first visible light image and the template image according to the first orientation, that is, determine the first region in the first visible light image in the first orientation, and determine the second region in the template image in the first orientation. The first orientation may be any orientation among a plurality of preset orientations, for example, the first orientation may be any orientation among the upper left, lower left, upper right, lower right, left, right, upper side, and lower side.

[0224] For example, if the first orientation is the upper left corner, the electronic device may determine the first area at the upper left corner from the first area in the first visible light image, and determine the second area at the upper left corner from the second area in the template image.

[0225] S402 , in a first coordinate system of the first visible light image, overall translate all first regions in the first visible light image so that the first regions at a first orientation are aligned with a first reference point in the first coordinate system.

[0226] S403: In the second coordinate system of the template image, translate all second regions in the template image as a whole, so that the second regions in the first orientation are aligned with the second reference points in the second coordinate system.

[0227] After obtaining two areas in the same orientation in the first visible light image and the template image, the electronic device can translate all the first areas in the first visible light image as a whole in the first coordinate system of the first visible light image, so that the first areas in the first orientation are aligned with the first reference point in the first coordinate system. At the same time, the electronic device can also translate all the second areas in the template image as a whole in the second coordinate system of the template image, so that the second areas in the first orientation are aligned with the second reference point in the second coordinate system. Among them, the first reference point is aligned with the second reference point, and the alignment of the area with the reference point can refer to the alignment of the center point of the area with the reference point, and can also refer to the alignment of the vertex of the area with the reference point, which is not specifically limited here.

[0228] by Figure 5 For example, in the scaling result corresponding to the template image, the second area in the template image is area 1, area 2 and area 3, the first area in the first visible light image is area 1', area 2' and area 3', and the original positions of the above-mentioned areas in the image are Figure 5 As shown in the left figure.

[0229] After the electronic device determines that the second area in the first orientation in the template image is area 1 and the first area in the first visible light image is area 1', on the one hand, in the second coordinate system of the template image, all the second areas in the template image can be translated as a whole so that the upper left vertex of area 1 is aligned with the upper left vertex of the template image (i.e., the second reference point in the second coordinate system). On the other hand, in the first coordinate system of the first visible light image, all the first areas in the first visible light image can be translated as a whole so that the upper left vertex of area 1' is aligned with the upper left vertex of the first visible light image (i.e., the first reference point in the first coordinate system). Finally, Figure 5 The translation result is shown in the right figure.

[0230] Based on the above translation operation, by setting the upper left vertex of the template image with the upper left vertex of the first visible light image (i.e. aligning the first reference point with the second reference point), the area 1' at the first orientation in the first visible light image can be aligned with the area 1 at the first orientation in the template image.

[0231] In the solution provided in the embodiment of the present application, the electronic device can ensure accurate alignment during the translation process by determining two areas in the same orientation in the first visible light image and the template image, namely the first area in the first orientation and the second area in the first orientation. This method of determining the area based on orientation reduces the risk of misalignment and improves the accuracy of alignment. And by means of overall translation, all first areas and all second areas are adjusted synchronously, making the alignment process more efficient and not changing the relative positional relationship between the first areas and the relative positional relationship between the second areas.

[0232] As an implementation method of the present application, Figure 6 As shown, when the electronic device translates the area where the components in the image are located as a whole, it can be implemented specifically through steps S601-S602:

[0233] S601, based on a scaling result corresponding to a template image, determining a first region in a first visible light image at a first orientation, and determining a second region in the template image at the first orientation.

[0234] The specific implementation method of step S601 is the same as that of the above step S401, which will not be repeated here.

[0235] S602, in the coordinate system of the first visible light image, translate all the second areas in the template image as a whole so that the second areas at the first position are aligned with the first areas at the first position in the first visible light image; or, in the coordinate system of the template image, translate all the first areas in the first visible light image as a whole so that the first areas at the first position are aligned with the second areas at the first position in the template image.

[0236] After obtaining two areas in the same orientation in the first visible light image and the template image, the electronic device can map all second areas in the template image to the first coordinate system of the first visible light image, and then, in the first coordinate system of the first visible light image, keep the first area in the first visible light image stationary and translate all second areas in the template image as a whole, so that the second area in the first orientation is aligned with the first area in the first orientation in the first visible light image.

[0237] For example, Figure 7a As shown, the electronic device can place all first regions (region 1', region 2' and region 3') in the first visible light image and all second regions (region 1, region 2 and region 3) in the template image in the first coordinate system of the first visible light image. At this time, the positions of the first region and the second region are as shown in FIG. Figure 7aThen the electronic device can keep the first area in the first visible light image still and translate all the second areas in the template image as a whole, so that the area 1 at the upper left of the template image is aligned with the area 1' at the upper left of the first visible light image. The positions of the first area and the second area after alignment are shown in FIG. Figure 7a As shown in the right figure.

[0238] After obtaining the two areas in the same orientation in the first visible light image and the template image, the electronic device can also map all the first areas in the first visible light image to the second coordinate system of the template image, and then, in the second coordinate system of the template image, keep the second areas in the template image stationary and translate all the first areas in the first visible light image as a whole, so that the first areas in the first orientation are aligned with the second areas in the first orientation in the template image.

[0239] For example, Figure 7b As shown, the electronic device can place all first regions (region 1', region 2' and region 3') in the first visible light image and all second regions (region 1, region 2 and region 3) in the template image in the second coordinate system of the template image. At this time, the positions of the first regions and the second regions are as shown in FIG. Figure 7b Then the electronic device can keep the second area in the template image fixed and translate all the first areas in the first visible light image as a whole, so that the area 1' at the upper left of the first visible light image is aligned with the area 1 at the upper left of the template image. The positions of the first area and the second area after alignment are shown in FIG. Figure 7b As shown in the right figure.

[0240] In the solution provided in the embodiment of the present application, the electronic device can ensure accurate alignment during the translation process by determining two areas in the same orientation in the first visible light image and the template image, namely the first area in the first orientation and the second area in the first orientation. This method of determining areas based on orientation reduces the risk of misalignment and improves the accuracy of alignment. And by means of overall translation, all first areas or all second areas in the template image are adjusted, making the alignment process more efficient, and will not change the relative positional relationship between the first areas, and the relative positional relationship between the second areas.

[0241] As an implementation method of the present application, Figure 8 As shown, the above-mentioned PCB component detection method may specifically include:

[0242] S801, performing component detection on a first visible light image of a PCB board to be detected, and obtaining a first region of components in the first visible light image.

[0243] S802, for each template image, based on the ratio between the acquisition distance of the template image and the acquisition distance of the first visible light image, scale the first area of ​​the component in the first visible light image and / or the second area of ​​the component in the template image, and based on the scaling result, translate the first area of ​​the component in the first visible light image and / or translate the second area of ​​the component in the template image to align the first area in the first orientation in the first visible light image with the second area in the template image at the first orientation, and obtain the scaling and translation result corresponding to the template image.

[0244] The specific manner of scaling and translating the region where the components in the image are located recorded in steps S801-S802 has been described in detail in the above embodiments and will not be repeated here.

[0245] S102 : For each template image, based on the scaling and translation result corresponding to the template image, calculating the matching degree between the first visible light image and the template image.

[0246] in, Figure 8 The specific implementation method of steps S102-S104 shown in has been described in detail in the above embodiments and will not be repeated here.

[0247] S803, determine whether the maximum matching degree between the first visible light image and each template image is greater than a preset threshold; if so, execute S103; if not, execute step S804, update the first orientation to other orientations among the preset multiple orientations except the current first orientation, and return to step S802 to perform the translation operation again.

[0248] In the process of performing component detection on the first visible light image of the PCB board to be inspected and obtaining the first area of ​​the components in the first visible light image, missed detection may occur. At this time, when aligning the first area in the first visible light image at the first position with the second area in the template image at the first position, if the first area in the first visible light image that should have been at the first position is not detected, then a matching failure will occur after the alignment.

[0249] by Fig. 9For example, the second regions in the template image are region 1, region 2, and region 3, and the first regions that should be detected in the first visible light image are region 1', region 2', and region 3', which correspond to region 1, region 2, and region 3, respectively. However, region 1' is missed during the actual detection, so when determining the first region in the upper left corner of the first visible light image, region 2' will be determined as the first region in the upper left corner, and then region 1 in the upper left corner of the template image will be aligned with region 2', and an incorrect alignment will occur, resulting in a matching failure when matching is performed based on the alignment result.

[0250] Therefore, after obtaining the matching degree between the first visible light image and each template image, the electronic device can determine whether the maximum matching degree between the first visible light image and each template image is greater than a preset threshold. If it is greater, it means that there is a template image in each template image that successfully matches the first visible light image. At this time, step S103 can be executed to continue the component detection process.

[0251] If it is not greater than, it means that all template images have failed to match the first visible light image. This may be because the first area in the first visible light image that should have been in the first orientation was not detected, resulting in a misalignment, which in turn led to a matching failure. At this time, the electronic device can update the first orientation to other orientations in the preset multiple orientations except the current first orientation, and then return to step S101 to perform the translation operation in step S101 again. When the electronic device returns to step S101, because it has been scaled before, it can only perform the translation operation in step S101 at this time, without performing the scaling operation in step S102.

[0252] Continuing with the above example, when the electronic device determines that the maximum matching degree between the first visible light image and each template image is not greater than the preset matching degree threshold, the first position can be updated to the lower right corner, and then the first position can be updated to the lower right corner by translating. Fig. 9 The area 3 at the lower right corner in the template image is aligned with the area 3 ′ at the lower right corner in the first visible light image.

[0253] S103: Select a target template image from each template image based on the obtained matching degree.

[0254] S104: Determine components included in the target area in the first visible light image as target components.

[0255] In the solution provided in the embodiment of the present application, when faced with a possible missed detection in the first visible light image, if the matching fails after the initial alignment, different alignment methods can be tried by updating the first orientation and re-performing the translation operation, effectively addressing the problem of incorrect alignment caused by missed detection. In addition, when re-performing the translation operation, the solution avoids unnecessary scaling steps, thereby optimizing the processing flow and reducing computing resource consumption.

[0256] As an implementation method of the present application, Fig.10 As shown, when the electronic device calculates the matching degree between the first visible light image and each template image through the second area in the template image and the first area in the first visible light image, the matching degree between the first visible light image and the template image can be obtained according to steps S1001-S1002 for each template image:

[0257] S1001, based on the scaling and translation result corresponding to the template image, matching a first region of the component in the first visible light image with a second region of the component in the template image to obtain a first number of successfully matched first regions.

[0258] After obtaining the zoom and translation result, the electronic device can calculate the matching degree between each first area in the first visible light image in the zoom and translation result and the second area of ​​each component in the template image, and if the maximum matching degree (i.e., the maximum value of each matching degree obtained) is greater than the first preset matching degree threshold, the first area is determined to be a first area that has successfully matched. After executing the above steps for each first area to determine whether it is a first area that has successfully matched, the electronic device can obtain the first number of first areas that have successfully matched. Among them, the first preset matching degree threshold and the above second preset matching degree threshold can be the same value or different values, which are not specifically limited here.

[0259] For example, the first regions included in the first visible light image are region A, region B, and region C, and the second regions included in the first template image are region a, region b, and region c. Then the electronic device can calculate the matching degree between region A and region a, region b, and region c, calculate the matching degree between region B and region a, region b, and region c, and calculate the matching degree between region C and region a, region b, and region c.

[0260] Assuming that the maximum matching degree between region A and region a is 70%, the maximum matching degree between region B and region b is 94%, and the maximum matching degree between region C and region c is 98, then because the maximum matching degree of region A is less than the first preset matching degree threshold of 85%, region A is the first region where the matching fails, because the maximum matching degree of region B is greater than the first preset matching degree threshold of 85%, region B is the first region where the matching succeeds, and because the maximum matching degree of region C is greater than the first preset matching degree threshold of 85%, region C is the first region where the matching succeeds. It can be determined that the first number of first regions where the matching succeeds is 2.

[0261] In one implementation, the electronic device may calculate the IOU (Intersection over Union) between the first region and the second region as the matching degree between the first region and the second region. Specifically:

[0262] The electronic device can obtain a first area where the first area intersects the second area, and then obtain the difference between the sum of the areas of the first area and the second area and the first area as the second area. The electronic device can then calculate the ratio of the first area to the second area as the degree of matching between the first area and the second area.

[0263] by Fig.11 For example, assuming that the area represented by rectangular box A is the first area, and the area represented by rectangular box B is the second area, the electronic device can calculate Fig.11 The area of ​​the overlapping portion of the rectangular frame A and the rectangular frame B represented by the shaded portion is used as the first area where the first area intersects the second area. The electronic device can then calculate the sum of the area of ​​the rectangular frame A and the area of ​​the rectangular frame B, and calculate the difference between the sum of the areas and the first area as the second area. Finally, the electronic device can calculate the ratio of the first area to the second area as the matching degree between the first area and the second area.

[0264] In addition to calculating the IOU between the first area and the second area as the matching degree between the first area and the second area, the electronic device may also calculate parameters such as DSC (Dice Similarity Coefficient) and HD (Hausdorff Distance) between the first area and the second area as the matching degree between the first area and the second area, which is not specifically limited here.

[0265] S1002, calculating a ratio of the first number to the total number of second regions in the template image, and obtaining a matching degree between the first visible light image and the template image based on the ratio.

[0266] After obtaining the first number of first areas that successfully match the second areas in the template image, the electronic device can calculate the ratio of the first number to the total number of second areas in the template image, and then determine the ratio as the degree of match between the first visible light image and the template image.

[0267] Continuing with the above example, when matching the first area in the first visible light image with the second area in the first template image, the first number of successfully matched first areas is 2, and the second areas included in the first template image are area a, area b, and area c, that is, the total number of second areas in the first template image is 3. Then the electronic device can calculate the ratio of the first number to the total number of second areas in the first template image (2 / 3) as the degree of match between the first visible light image and the first template image.

[0268] In the solution provided in the embodiment of the present application, the electronic device can determine the matching degree between the first area in the first visible light image and the second area in the template image by calculating parameters such as IOU, DSC or HD of the two areas, so as to accurately evaluate the matching degree and reduce misjudgment. Then, for each template image, when matching the first area in the first visible light image with the second area in the template image, the ratio of the number of successfully matched first areas to the total number of second areas in the first template image is used to determine the matching degree between the first visible light image and the template image. In this way, the overall matching degree is determined based on the ratio of the number of successfully matched areas to the total number of areas. The method is intuitive and easy to implement, which effectively improves the reliability and stability of the matching results.

[0269] As an implementation method of an embodiment of the present application, after the electronic device determines the first number of matching first areas through the above-mentioned step S1001, it may not execute step S1002, but may directly use the number of successfully matched first areas as the degree of matching between the first visible light image and the template image, or calculate the ratio of the number of successfully matched first areas to the total number of first areas in the first visible light image, and then use the ratio as the degree of matching between the first visible light image and the template image.

[0270] As an implementation method of the present application, Fig.12 As shown, after the above step S104, the above method may further include step S105:

[0271] S105, removing the target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as the base point, enlarging the area where the target component is located according to a preset magnification factor and / or the size of the third area.

[0272] After determining the target components, the electronic device can further remove the smaller misdetected target components from the target components, and / or enlarge the area where the target components are located. The specific removal method and the specific area enlargement method are respectively introduced below:

[0273] As for the removal method, due to the noise or blur introduced in the process of collecting the first visible light image, or the false detection in the component detection, some smaller areas may appear in the area where the component is located in the first visible light image. These smaller areas may not include a component.

[0274] Therefore, after determining the target components, the electronic device can further remove the target components whose area has a size smaller than a preset size threshold from the determined target components.

[0275] For the area magnification method, due to the influence of factors such as unclear edge features, image noise, and uneven lighting, the area where the component is located obtained through component detection may not include the complete edge of the component, so the first area can be appropriately magnified to offset the influence of these external factors to a certain extent. In addition, if the target component is to be tested for temperature later, if the abnormally high temperature of the component happens to occur at the edge of the component, then when the first area is small and cannot include the complete edge of the component, the temperature anomaly will be missed.

[0276] Therefore, after determining the target component, the electronic device can enlarge the area where the target component is located with the center point of the area where the target component is located as a base point.

[0277] In one implementation, when the electronic device magnifies the area where the target component is located, it can directly magnify it according to a preset magnification factor.

[0278] In another embodiment, in the stage of generating a template image, after obtaining the second area of ​​the component in the second visible light image, the second area can be identified in the second visible light image by means of a target box, and the identification result can be displayed to the user. At this time, the user can enlarge the second area in the second visible light image by enlarging the target box.

[0279] On this basis, when the electronic device enlarges the area where the target component is located, it can determine the area with the greatest match with the area where the target component is located in the area where the component is located in the target template image, and use it as the third area. The electronic device can then enlarge the area where the target component is located according to the size of the third area, that is, enlarge the size of the area where the target component is located to the same size as the third area.

[0280] In the solution provided by the embodiment of the present application, after determining the target components, the electronic device effectively eliminates small areas of false detection by removing the target components whose sizes are smaller than the preset size threshold. At the same time, by enlarging the area where the target components are located, it is ensured that the edges of the components are completely covered, which solves the problem of incomplete areas caused by factors such as edge blur and noise to a certain extent, and improves the comprehensiveness and accuracy of detection. In particular, for temperature detection scenarios, the amplification process avoids the omission of temperature anomalies due to edge omissions, and enhances the reliability of the system. In addition, two methods of amplification based on user interaction and amplification based on a preset amplification factor are provided, which further enhances the flexibility and adaptability of the solution.

[0281] In a specific application scenario, after detecting the target component in the first visible light image of the PCB board to be detected, the electronic device can further perform temperature detection on the target component. Figure 13-Figure 16 Introduce the specific method of temperature detection.

[0282] As an implementation method of the present application, Fig.13 As shown, the above method may also include:

[0283] S1301, acquiring a thermal imaging image of a PCB board to be inspected that is acquired synchronously with the first visible light image.

[0284] In a scenario where the temperature of components on a PCB needs to be detected, while a visible light camera is used to collect a visible light image of the PCB to be detected, a thermal imaging camera can also be used to collect a thermal imaging image of the PCB to be detected. In this process, multiple visible light images and multiple thermal imaging images of the PCB to be detected can be obtained.

[0285] Therefore, after acquiring the first visible light image of the PCB board to be inspected, the electronic device can acquire a thermal imaging image having the same acquisition timestamp as the acquisition timestamp of the first visible light image according to the acquisition timestamp of the first visible light image, thereby obtaining a thermal imaging image of the PCB board to be inspected that is acquired synchronously with the first visible light image.

[0286] S1302: Determine, in the thermal imaging image, a fourth area corresponding to the position of the area where the target component is located.

[0287] After determining the area where the target component is located in the first visible light image, the electronic device can determine the area corresponding to the position of the area where the target component is located in the image coordinate system of the thermal imaging image as the fourth area. The first visible light image used in temperature detection is the original first visible light image, that is, the first area in the first visible light image used in temperature detection is not scaled or translated.

[0288] Regarding the method of determining the fourth area in the thermal imaging image, the electronic device may implement it in, but not limited to, the following two methods:

[0289] In the first method, the correspondence between the pixel points in the image captured by the visible light camera and the pixel points in the image captured by the thermal imaging camera can be determined in advance based on the parameters of the visible light camera that captures the first visible light image and the parameters of the thermal imaging camera that captures the thermal imaging image.

[0290] In this way, after obtaining the first visible light image and the thermal imaging image, the electronic device can map the position of the area where the target component is located in the first visible light image to the image coordinate system of the thermal imaging image according to the correspondence between the above-mentioned pixel points, thereby determining the fourth area corresponding to the position of the area where the target component is located in the thermal imaging image.

[0291] In the second method, after obtaining the original visible light image and thermal imaging image of the PCB to be inspected, the electronic device can use the dual-light fusion registration parameters to scale and crop the original visible light image to obtain an image consistent with the field of view of the thermal imaging image, and use it as the first visible light image of the PCB to be inspected. And when performing component inspection on the first visible light image of the PCB to be inspected, the coordinate system where the regional coordinates of the area where the components are located in the first visible light image can be normalized, that is, the regional coordinates of the area where the components are located in the detected first visible light image are normalized coordinates, and the coordinate values ​​are within the range of [0,1].

[0292] In this way, after obtaining the first visible light image and the thermal imaging image, the electronic device can first denormalize the regional coordinates of the area where the target component is located according to the resolution of the thermal imaging image, and then determine the fourth area corresponding to the position of the first area of ​​the target component in the coordinate system corresponding to the thermal imaging image according to the denormalized regional coordinates.

[0293] S1303: Determine the temperature of the target component according to the image data of the fourth area.

[0294] After the electronic device determines the fourth area, because the grayscale value in the fourth area is positively correlated with the temperature, that is, the greater the grayscale, the higher the temperature, the electronic device can determine the temperature of the target component based on the image data in the fourth area (that is, the grayscale value in the fourth area).

[0295] In one implementation, the electronic device may first calculate the average grayscale value of each pixel in the fourth area, and then convert the average grayscale value into temperature, thereby determining the temperature of the target component.

[0296] In another embodiment, in order to ensure the accuracy of the measured temperature, the electronic device may first determine the maximum value of the grayscale value in the fourth area according to the grayscale value of each pixel in the fourth area, and then calculate the average value of the maximum grayscale value and the surrounding grayscale values, and finally convert the average value into temperature, thereby determining the temperature of the target component. Of course, after obtaining the maximum grayscale value, the electronic device may also first convert the maximum grayscale value and the surrounding grayscale values ​​of the maximum grayscale value into temperatures, and then determine the temperature of the target component according to the average value of the temperatures.

[0297] As for the method of converting grayscale values ​​into temperature, the thermal imaging camera that collects thermal imaging images can be calibrated in advance to determine the corresponding relationship between the grayscale value and the temperature in the thermal imaging image collected by the thermal imaging camera, thereby ensuring that the grayscale value output by the thermal imaging camera can be accurately mapped to the actual temperature value. In this way, in the subsequent temperature detection process, after the maximum grayscale value is determined from the fourth area, the maximum grayscale value can be converted into the actual temperature according to the pre-calibrated corresponding relationship.

[0298] S1304, detecting whether the temperature of the target component is abnormal.

[0299] After obtaining the temperature of the target component, the electronic device can determine whether the temperature of the target component is abnormal according to whether the temperature of the target component is greater than the alarm temperature. Among them, each target component can correspond to the same alarm temperature or different alarm temperatures (for example, different types of components correspond to different alarm temperatures), which is not specifically limited here.

[0300] In the solution provided in the embodiment of the present application, by synchronously collecting the visible light image and the thermal imaging image of the PCB board to be detected, accurate detection of the temperature of the components is achieved. In this process, through the predetermined pixel point correspondence or image preprocessing steps, the fourth area corresponding to the target component can be quickly located in the thermal imaging image, which greatly simplifies the operation process.

[0301] As an implementation method of the present application, Fig.14As shown, the above step S1303 may include:

[0302] S1401, in the thermal imaging image, determining the maximum grayscale value in the fourth area.

[0303] In the process of temperature detection, we mainly focus on whether the temperature of the target component is abnormally high. Given the positive correlation between grayscale value and temperature in thermal imaging images, that is, the larger the grayscale value, the higher the temperature represented, therefore, when processing thermal imaging data, we do not need to convert all grayscale values ​​in the fourth area into temperature values. Instead, we directly locate and extract the maximum grayscale value in the fourth area. This maximum grayscale value can intuitively reflect the highest temperature point of the target component.

[0304] S1402, converting the maximum grayscale value into temperature as the temperature of the target component.

[0305] After obtaining the maximum grayscale value, the electronic device may only convert the maximum grayscale value into an actual temperature, and then determine the temperature obtained after the conversion as the temperature of the target component.

[0306] In the solution provided in the embodiment of the present application, the efficiency of temperature detection is significantly improved by directly locating and extracting the maximum grayscale value of the area corresponding to the target component in the thermal imaging image. Since the grayscale value is positively correlated with the temperature, the maximum grayscale value directly reflects the highest temperature point of the target component, which is crucial for determining whether the component is overheated.

[0307] As an implementation method of the present application, Fig.15 As shown, the above step S1303 may include:

[0308] S1501, obtaining temperature measurement parameters of components in the third area.

[0309] For components on PCB boards, different types of components have different characteristics, which may lead to some differences in converting grayscale values ​​to temperature for different types of components. For example, different types of components have different parameters such as emissivity and reflection temperature, and for the same type of components, different materials, coatings and surface treatment processes will lead to different parameters such as emissivity and reflection temperature of components.

[0310] Therefore, in order to ensure the accuracy of temperature detection, the emissivity of the components in each second area in the template image can be preset in the generation of the template image, wherein the emissivity of the components represents its ability to emit infrared radiation energy. In addition, the reflected temperature of the components in each second area in the template image can also be preset in the generation of the template image.

[0311] Accordingly, in order to ensure the accuracy of temperature detection, when converting the grayscale value in the fourth area into temperature, the electronic device can first determine the area with the greatest matching degree with the area where the target component is located in the area where the component is located in the target template image, as the third area, and then obtain the preset emissivity and preset reflection temperature of the component in the third area from the target template image, thereby obtaining two types of factors that have a greater impact on temperature detection. Among them, the preset reflection temperature of the component can be a preset value. If the value is not preset, the ambient temperature when the thermal imaging image is obtained can be used as the preset reflection temperature.

[0312] S1502: Convert the grayscale value in the fourth area into temperature according to the temperature measurement parameter as the temperature of the target component.

[0313] In one embodiment, the correspondence between the grayscale value and the temperature can be pre-calibrated at different emissivities and / or different reflection temperatures. For example, at the first emissivity and the first reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the first correspondence, at the first emissivity and the second reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the second correspondence... At the Nth emissivity and the Gth reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the Xth correspondence.

[0314] In this way, after obtaining the preset emissivity and the preset reflection temperature of the components in the third area, the electronic device can determine the target correspondence relationship between the preset emissivity and the preset reflection temperature from the multiple sets of correspondence relationships between the pre-calibrated grayscale values ​​and the temperatures. After obtaining the target correspondence relationship, the electronic device can convert the grayscale value in the fourth area into a temperature according to the target correspondence relationship as the temperature of the target component.

[0315] In another embodiment, the grayscale value and the corresponding temperature may be collected in advance at different emissivities and / or different reflection temperatures, and then a calculation formula between the emissivity, reflection temperature, grayscale value and temperature may be fitted based on the collected data.

[0316] In this way, after obtaining the preset emissivity and preset reflection temperature of the components in the third area, the electronic device can calculate the corresponding temperature according to the preset emissivity, preset reflection temperature and the grayscale value in the fourth area according to the above calculation formula as the temperature of the target component.

[0317] In the solution provided by the embodiment of the present application, by obtaining the preset emissivity and the preset reflection temperature of the components in the third area, two types of factors that affect the accuracy of temperature detection are fully considered, thereby ensuring the accuracy of temperature conversion. Secondly, by pre-calibrating the correspondence between grayscale values ​​and temperatures under different conditions, or pre-calibrating the calculation formula between grayscale values ​​and temperatures, a comprehensive correspondence database is established, so that electronic equipment can quickly realize accurate conversion of grayscale values ​​to temperature according to actual detection conditions.

[0318] As an implementation method of an embodiment of the present application, because when collecting the thermal imaging image of the PCB board to be inspected, the collection distance of the thermal imaging image will also affect the collected thermal imaging image, the electronic device can further obtain the collection distance of the thermal imaging image when acquiring the thermal imaging image.

[0319] Accordingly, when converting the grayscale value into temperature, the electronic device may consider not only the temperature measurement parameter but also the acquisition distance of the thermal imaging image. That is, the above step S1502 may include:

[0320] According to the temperature measurement parameters and the acquisition distance of the thermal imaging image, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

[0321] In one embodiment, the correspondence between the grayscale value and the temperature can be pre-calibrated at different emissivities, different collection distances, and / or different reflection temperatures. For example, at the first emissivity, the first collection distance, and the first reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the first correspondence, at the first emissivity, the first collection distance, and the second reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the second correspondence... At the Nth emissivity, the Mth collection distance, and the Gth reflection temperature, the correspondence between the grayscale value and the temperature is calibrated as the Xth correspondence.

[0322] In this way, after obtaining the preset emissivity and preset reflection temperature of the components in the third area, as well as the collection distance of the thermal imaging image, the electronic device can determine the target correspondence relationship corresponding to the preset emissivity, the collection distance of the thermal imaging image, and the preset reflection temperature from the above-mentioned multiple sets of correspondence relationships between the pre-calibrated grayscale values ​​and the temperature. After obtaining the target correspondence relationship, the electronic device can convert the grayscale value in the fourth area into a temperature according to the target correspondence relationship as the temperature of the target component.

[0323] In another embodiment, the grayscale value and the corresponding temperature can be pre-collected at different emissivities, different collection distances and / or different reflection temperatures, and then a calculation formula between the emissivity, reflection temperature, grayscale value, collection distance and temperature can be fitted based on the collected data.

[0324] In this way, after obtaining the preset emissivity and preset reflection temperature of the components in the third area, as well as the collection distance of the thermal imaging image, the electronic device can calculate the corresponding temperature according to the preset emissivity, preset reflection temperature, collection distance of the thermal imaging image and the grayscale value in the fourth area according to the above calculation formula, as the temperature of the target component.

[0325] In the solution provided in the embodiment of the present application, by obtaining the preset emissivity of the components in the third area, the preset reflection temperature and the acquisition distance of the thermal imaging image, the three factors that affect the accuracy of temperature detection are fully considered, thereby ensuring the accuracy of temperature conversion. Secondly, by pre-calibrating the correspondence between the grayscale value and the temperature under different conditions, or the calculation formula between the pre-calibrated grayscale value and the temperature, a comprehensive correspondence database is established, so that the electronic device can quickly realize the accurate conversion of grayscale value to temperature according to the actual detection conditions.

[0326] As an implementation method of the present application, Fig.16 As shown, the above step S1304 may include:

[0327] S1601, obtaining preset alarm temperatures of components in a third area.

[0328] For components on a PCB, different types of components have different operating temperature ranges due to differences in materials, structures, functions, etc. Some components may still work normally at high temperatures, while others may begin to degrade or become damaged at lower temperatures. Therefore, in order to ensure the accuracy of temperature detection, in generating a template image, for each second area in the template image, the alarm temperature of the second area may be pre-set according to the components in the second area.

[0329] Correspondingly, when the electronic device detects whether the temperature of a target component is abnormal, it can first obtain the alarm temperature corresponding to the target component. Specifically, the electronic device can first determine the area with the greatest match with the area where the target component is located in the area where the component is located in the target template image, as the third area, and then obtain the preset alarm temperature of the component in the third area as the alarm temperature of the target component.

[0330] S1602, determine whether the temperature of the target component is greater than the preset alarm temperature. If yes, execute step S1603, if not, execute step S1604.

[0331] Because the preset alarm temperature of the third area is set for the components in the third area, and the components in the third area are the same as the target components, the electronic device can determine whether the temperature of the target component is greater than the preset alarm temperature after obtaining the preset alarm temperature of the third area.

[0332] S1603, determining whether the temperature of the target component is abnormal.

[0333] If it is greater than, it is determined that the temperature of the target component is abnormal, and temperature alarm information for the target component can be output.

[0334] S1604, determine that the temperature of the target component is normal.

[0335] If it is not greater than, it is determined that the temperature of the target component is normal. In this case, no information may be output, or normal temperature information for the target component may be output.

[0336] In the solution provided by the embodiment of the present application, the differences between components are fully considered, and the corresponding alarm temperature is set for the working characteristics of each component. By matching the target component with the area in the template image, the preset alarm temperature corresponding to the target component is quickly obtained, thereby improving the accuracy and pertinence of temperature detection. Finally, based on the comparison result between the actual temperature of the target component and the preset alarm temperature, the temperature abnormality or normal information is output in time, which provides strong support for subsequent troubleshooting and maintenance, and realizes PDA (PCB Detection & Alarm).

[0337] As an implementation method of an embodiment of the present application, each second area in the generated template image may have a unique ID. When the first area in the first visible light image is subsequently matched with the second area in the target template image, for each successfully matched first area, the ID of the second area in the target template image with the highest degree of match with the first area may be assigned to the first area, thereby generating a unique ID for each successfully matched first area in the first visible light image.

[0338] For example, the first regions that are successfully matched in the first visible light image are region a and region b. For region a, the second region with the highest matching degree with region a in the target template image is region A. For region b, the second region with the highest matching degree with region b in the target template image is region B. Assuming that the ID of region A is "0005" and the ID of region B is "0007", ID "0005" can be assigned to region a and ID "0007" can be assigned to region b.

[0339] Correspondingly, when the electronic device obtains the preset emissivity, preset reflection temperature and preset alarm temperature of the target component in the first visible light image, it can first obtain the ID of the first area of ​​the target component, and then determine a second area in the target template image whose ID is the same as the ID of the first area as the third area corresponding to the first area, and then obtain the preset emissivity, preset reflection temperature and preset alarm temperature of the component in the third area.

[0340] In the solution provided in the embodiment of the present application, when the electronic device matches the first visible light image with the target template image, it can associate the first area in the first visible light image with the second area in the target template image through the ID, so that when the temperature measurement parameters and alarm temperature are subsequently obtained, they can be directly obtained through the ID, thereby improving the efficiency of the execution of the solution.

[0341] Corresponding to the above-mentioned PCB component detection method, the embodiment of the present application further provides a PCB component detection device. The PCB component detection device provided by the embodiment of the present application is introduced below.

[0342] like Fig.17 As shown, a PCB component detection device includes:

[0343] The zoom and translation module 1701 is used to zoom and translate the template image and / or the first visible light image of the PCB to be detected for each template image, and obtain a zoom and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of collecting the template image and the field of view and the viewing angle of collecting the first visible light image, wherein the template image is a visible light image of the template PCB and includes components of interest to the user in the template PCB;

[0344] A matching module 1702 is used to calculate, for each template image, a matching degree between the first visible light image and the template image based on a zooming and translation result corresponding to the template image;

[0345] An image selection module 1703 is used to select a target template image from each template image based on the obtained matching degree;

[0346] The component determination module 1704 is used to determine the components included in the target area in the first visible light image as target components, wherein the target area is an area that matches the area where the components in the target template image are located.

[0347] In the solution provided by the embodiment of the present application, the electronic device can scale and translate the template image and / or the first visible light image of the PCB board to be detected for each template image, and obtain the scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of the template image and the field of view and the viewing angle of the first visible light image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; for each template image, based on the scaling and translation result corresponding to the template image, calculate the matching degree between the first visible light image and the template image; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, through the matching degree between the first visible light image of the PCB board to be detected and each template image, a template image that is closer to the first visible light image can be selected as a reference. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection. In addition, before calculating the matching degree between the first visible light image and the template image, the consistency of the acquisition field of view and the acquisition angle of view of the first visible light image and the template image is ensured by scaling and translation operations, which provides a basis for accurate matching, so that the acquisition of the first visible light image is no longer limited to the acquisition field of view and acquisition angle of view that are completely consistent with the template image.

[0348] As an implementation of the embodiment of the present application, the above device may further include:

[0349] A component detection module, used for performing component detection on the first visible light image of the PCB board to be detected, and obtaining a first region of components in the first visible light image;

[0350] The above-mentioned zooming and translation module 1701 can be specifically used to, for each template image, scale the first area of ​​the component in the first visible light image and / or the second area of ​​the component in the template image based on the ratio between the acquisition distance of the template image and the acquisition distance of the first visible light image, and based on the scaling result, translate the first area of ​​the component in the first visible light image and / or translate the second area of ​​the component in the template image so that the first area in the first visible light image at the first orientation is aligned with the second area in the template image at the first orientation, thereby obtaining the zooming and translation result corresponding to the template image.

[0351] As an implementation of the embodiment of the present application, the above-mentioned zooming and translation module 1701 may include:

[0352] A region determining unit, configured to determine a first region at a first position in the first visible light image and a second region at the first position in the template image based on a scaling result corresponding to the template image;

[0353] A first region translation unit, configured to translate all first regions in the first visible light image as a whole in a first coordinate system of the first visible light image so that the first regions in the first orientation are aligned with a first reference point in the first coordinate system;

[0354] The second region translation unit is used to translate all second regions in the template image as a whole in the second coordinate system of the template image so that the second region in the first orientation is aligned with the second reference point in the second coordinate system, wherein the first reference point is aligned with the second reference point.

[0355] As an implementation of the embodiment of the present application, the above-mentioned zooming and translation module 1701 may include:

[0356] A region determining unit, configured to determine a first region at a first position in the first visible light image and a second region at the first position in the template image based on a scaling result corresponding to the template image;

[0357] A region translation unit is used to translate all second regions in the template image as a whole in the coordinate system of the first visible light image so that the second region at the first position is aligned with the first region at the first position in the first visible light image; or, in the coordinate system of the template image, to translate all first regions in the first visible light image as a whole so that the first region at the first position is aligned with the second region at the first position in the template image.

[0358] As an implementation of the embodiment of the present application, the above device may further include:

[0359] The judgment module is used to judge whether the maximum matching degree between the first visible light image and each template image is greater than a preset threshold; when the judgment result is yes, the execution of the image selection module 1703 is triggered; when the judgment result is no, the first orientation is updated to other orientations among the preset multiple orientations except the current first orientation, and the execution of the zoom and translation module 1701 is triggered.

[0360] As an implementation of the embodiment of the present application, the matching module 1702 may include:

[0361] The area matching unit is used to match the first area of ​​the component in the first visible light image with the second area of ​​the component in the template image for each template image based on the scaling and translation result corresponding to the template image, and obtain the matching degree between the first visible light image and the template image based on the matching result.

[0362] As an implementation of the embodiment of the present application, the above-mentioned area matching unit may include:

[0363] a matching number determination subunit, configured to match, for each template image, a first region of the component in the first visible light image with a second region of the component in the template image based on a scaling and translation result corresponding to the template image, to obtain a first number of first regions that are successfully matched;

[0364] The matching degree calculation subunit is used to calculate the ratio of the first number to the total number of second areas in the template image, and obtain the matching degree between the first visible light image and the template image based on the ratio.

[0365] As an implementation of an embodiment of the present application, the above-mentioned matching quantity determination subunit can be specifically used for:

[0366] For each first area, the matching degree between the first area and the second area of ​​the component in the template image is calculated respectively. If the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, the first area is determined to be a successfully matched first area; and a first number of successfully matched first areas is obtained.

[0367] As an implementation of the embodiment of the present application, the matching degree between the first area and the second area may be calculated in the following manner:

[0368] Obtaining a first area where the first area intersects the second area;

[0369] Get the difference between the sum of the areas of the first and second areas and the first area as the second area:

[0370] The ratio of the first area to the second area is calculated as the matching degree between the first region and the second region.

[0371] As an implementation of the embodiment of the present application, the above-mentioned component determination module 1704 may include:

[0372] A region screening unit, configured to screen out, from the region where the component in the first visible light image is located, a region whose matching degree with the region where the component in the target template image is located is greater than a second preset matching degree threshold value as a target region;

[0373] The target component determination unit is used to determine the components included in the target area as target components.

[0374] As an implementation of the embodiment of the present application, the above device may further include:

[0375] The component adjustment module is used to remove the target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as the base point, enlarge the area where the target component is located according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

[0376] As an implementation of the embodiment of the present application, the above device may further include:

[0377] A thermal imaging image acquisition module, used to acquire a thermal imaging image of the PCB board to be inspected which is acquired synchronously with the first visible light image;

[0378] A region determination module, used to determine, in the thermal imaging image, a fourth region corresponding to the position of the region where the target component is located;

[0379] a temperature determination module, used to determine the temperature of the target component according to the image data of the fourth area;

[0380] The temperature detection module is used to detect whether the temperature of the target component is abnormal.

[0381] As an implementation of the embodiment of the present application, the temperature determination module can be specifically used for:

[0382] In the thermal imaging image, determining a maximum grayscale value in the fourth region;

[0383] Convert the maximum grayscale value into temperature and use it as the temperature of the target component.

[0384] As an implementation of the embodiment of the present application, the temperature determination module can be specifically used for:

[0385] Obtaining temperature measurement parameters of components in a third region, wherein the third region is a region in which components in the target template image are located and has the greatest matching degree with the region in which the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature;

[0386] According to the temperature measurement parameters, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

[0387] As an implementation of the embodiment of the present application, the above device may further include:

[0388] A collection distance acquisition module is used to obtain the collection distance of the thermal imaging image;

[0389] The temperature determination module can be specifically used to convert the grayscale value in the fourth area into temperature as the temperature of the target component according to the temperature measurement parameters and the acquisition distance of the thermal imaging image.

[0390] As an implementation of the embodiment of the present application, the temperature detection module may include:

[0391] An alarm temperature acquisition unit, used to acquire a preset alarm temperature of a component in a third area, wherein the third area is an area where the component in the target template image is located and has the highest matching degree with the area where the target component is located;

[0392] The temperature detection unit is used to determine whether the temperature of the target component is greater than a preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

[0393] As an implementation of the embodiment of the present application, the template image generation method may include:

[0394] For the second visible light image of the template PCB board, perform the following operations:

[0395] Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction;

[0396] and / or,

[0397] Based on parameter information set for the initial region where the components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

[0398] The present application also provides an electronic device, such as Fig.18 As shown, including:

[0399] Memory 1801, used for storing computer programs;

[0400] The processor 1802 is used to implement the PCB component detection method described in any of the above embodiments when executing the program stored in the memory 1801.

[0401] In the solution provided by the embodiment of the present application, the electronic device can scale and translate the template image and / or the first visible light image of the PCB board to be detected for each template image, and obtain the scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of the template image and the field of view and the viewing angle of the first visible light image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; for each template image, based on the scaling and translation result corresponding to the template image, calculate the matching degree between the first visible light image and the template image; select a target template image from each template image based on the obtained matching degree; determine the components included in the target area in the first visible light image as the target components, wherein the target area is an area that matches the area where the components in the target template image are located. On the one hand, through the matching degree between the first visible light image of the PCB board to be detected and each template image, a template image that is closer to the first visible light image can be selected as a reference. On this basis, because the area where the components in the pre-generated template image are located is relatively accurate, based on the area where the components in the selected template image are located, the corresponding component area in the first visible light image can be accurately identified and located, and then the components in the located area are determined as target components, thereby significantly improving the accuracy of component detection. On the other hand, the template image contains components that the user is interested in. Therefore, when determining the target components in the first visible light image based on the template image, the components that the user is not interested in can be automatically filtered out, thereby reducing the redundancy of the detection results and further improving the accuracy of component detection. In addition, before calculating the matching degree between the first visible light image and the template image, the consistency of the acquisition field of view and the acquisition angle of view of the first visible light image and the template image is ensured by scaling and translation operations, which provides a basis for accurate matching, so that the acquisition of the first visible light image is no longer limited to the acquisition field of view and acquisition angle of view that are completely consistent with the template image.

[0402] As an implementation method of the present application, the electronic device may be a handheld detection device, such as Fig.19As shown, the handheld detection device may include, in addition to the memory 1801 and the processor 1802, a first camera 1803, a second camera 1804 and a distance measuring device 1805. The first camera 1803 is used to collect a visible light image of the PCB board to be detected, the second camera 1804 is used to collect a thermal imaging image of the PCB board to be detected, and the distance measuring device 1805 is used to obtain the collection distance of the visible light image and the thermal imaging image.

[0403] In one embodiment, the handheld detection device may also have a display screen. The handheld detection device can display the first area of ​​the component in the first visible light image on the display screen, and can also display the temperature detection results of the target component on the display screen. It can also display various template images for user adjustment, etc. There is no specific limitation on the content displayed on the display screen.

[0404] In addition, the electronic device may further include a communication bus and / or a communication interface, and the processor 1802, the communication interface, and the memory 1801 communicate with each other via the communication bus.

[0405] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0406] The communication interface is used for communication between the above electronic device and other devices.

[0407] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0408] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0409] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the PCB component detection method described in any of the above embodiments is implemented.

[0410] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the PCB component detection method described in any of the above embodiments.

[0411] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0412] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0413] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0414] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A PCB component detection method, characterized in that: The method comprises: For each template image, scaling and translating the template image and / or the first visible light image of the PCB to be inspected to obtain a scaling and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of collecting the template image and the field of view and the viewing angle of collecting the first visible light image, wherein the template image is a visible light image of the template PCB and includes components of interest to the user in the template PCB; For each template image, based on the scaling and translation result corresponding to the template image, calculating the matching degree between the first visible light image and the template image; Selecting a target template image from each template image based on the obtained matching degree; Components included in a target area in the first visible light image are determined as target components, wherein the target area is an area that matches an area where components in the target template image are located.

2. The method according to claim 1, characterized in that Before scaling and translating the template image and / or the first visible light image of the PCB board to be inspected for each template image, the method further includes: Perform component detection on the first visible light image of the PCB board to be detected, and obtain a first region of components in the first visible light image; For each template image, scaling and translating the template image and / or the first visible light image of the PCB board to be detected to obtain a scaling and translation result corresponding to the template image includes: For each template image, based on the ratio between the acquisition distance of the template image and the acquisition distance of the first visible light image, the first area of ​​the component in the first visible light image and / or the second area of ​​the component in the template image are scaled, and based on the scaling result, the first area of ​​the component in the first visible light image is translated and / or the second area of ​​the component in the template image is translated, so that the first area in the first visible light image at the first orientation is aligned with the second area in the template image at the first orientation, and the scaling and translation result corresponding to the template image is obtained.

3. The method according to claim 2, characterized in that The method of translating the first region of the component in the first visible light image and / or translating the second region of the component in the template image based on the scaling result so that the first region in the first visible light image at the first orientation is aligned with the second region in the template image at the first orientation includes: Based on the scaling result corresponding to the template image, determining a first region in the first visible light image at a first orientation, and determining a second region in the template image at the first orientation; In a first coordinate system of the first visible light image, overall translation is performed on all first regions in the first visible light image so that the first regions at the first orientation are aligned with a first reference point in the first coordinate system; In the second coordinate system of the template image, all second regions in the template image are translated as a whole, so that the second regions in the first orientation are aligned with the second reference points in the second coordinate system, wherein the first reference points are aligned with the second reference points.

4. The method according to claim 2, characterized in that: The method of translating the first region of the component in the first visible light image and / or translating the second region of the component in the template image based on the scaling result so that the first region in the first visible light image at the first orientation is aligned with the second region in the template image at the first orientation includes: Based on the scaling result corresponding to the template image, determining a first region in the first visible light image at a first orientation, and determining a second region in the template image at the first orientation; In the coordinate system of the first visible light image, all the second areas in the template image are translated as a whole so that the second areas at the first position are aligned with the first areas at the first position in the first visible light image; or, in the coordinate system of the template image, all the first areas in the first visible light image are translated as a whole so that the first areas at the first position are aligned with the second areas at the first position in the template image.

5. The method according to claim 2, characterized in that: After calculating the matching degree between the first visible light image and the template image based on the scaling and translation result corresponding to the template image for each template image, the method further includes: Determining whether a maximum degree of match between the first visible light image and each template image is greater than a preset threshold; If it is greater than, executing the step of selecting a target template image from each template image based on the obtained matching degree; If it is not greater than, the first orientation is updated to another orientation among the preset multiple orientations except the current first orientation, and the step of translating the first area of ​​the component in the first visible light image and / or translating the second area of ​​the component in the template image based on the scaling result is returned to so that the first area in the first visible light image at the first orientation is aligned with the second area in the template image at the first orientation.

6. The method according to claim 1, characterized in that The calculating, for each template image, a degree of matching between the first visible light image and the template image based on a scaling and translation result corresponding to the template image, includes: For each template image, based on the scaling and translation result corresponding to the template image, the first area of ​​the component in the first visible light image is matched with the second area of ​​the component in the template image, and the matching degree between the first visible light image and the template image is obtained based on the matching result.

7. The method according to claim 6, characterized in that The method of matching, for each template image, a first region of the component in the first visible light image with a second region of the component in the template image based on a zooming and translation result corresponding to the template image, and obtaining a matching degree between the first visible light image and the template image based on the matching result, includes: For each template image, the matching degree between the first visible light image and the template image is obtained in the following manner: Based on the scaling and translation result corresponding to the template image, matching the first region of the component in the first visible light image with the second region of the component in the template image to obtain a first number of first regions that are successfully matched; A ratio of the first number to the total number of second regions in the template image is calculated, and a matching degree between the first visible light image and the template image is obtained based on the ratio.

8. The method according to claim 7, characterized in that The step of matching the first region of the component in the first visible light image with the second region of the component in the template image to obtain a first number of first regions that are successfully matched includes: For each first region, respectively calculate the matching degree between the first region and the second region of the component in the template image, and if the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, determine that the first region is a successfully matched first region; A first number of first regions that are matched successfully is obtained.

9. The method according to claim 8, characterized in that The matching degree between the first region and the second region is calculated as follows: Obtaining a first area where the first area intersects the second area; The difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area: A ratio of the first area to the second area is calculated as a matching degree between the first region and the second region.

10. The method according to claim 1, characterized in that The step of determining the components included in the target area in the first visible light image as target components includes: In the region where the components in the first visible light image are located, a region whose matching degree with the region where the components in the target template image are located is greater than a second preset matching degree threshold is selected as the target region; The components included in the target area are determined as target components.

11. The method according to claim 1, characterized in that: After determining the components included in the target area in the first visible light image as target components, the method further includes: From the determined target components, remove the target components whose area size is smaller than a preset size threshold; and / or, Taking the center point of the area where the target component is located as the base point, the area where the target component is located is enlarged according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located.

12. The method according to any one of claims 1 to 11, characterized in that: The method further comprises: Acquire a thermal imaging image of the PCB board to be inspected that is acquired synchronously with the first visible light image; In the thermal imaging image, determining a fourth area corresponding to the position of the area where the target component is located; determining the temperature of the target component according to the image data of the fourth area; Detect whether the temperature of the target component is abnormal.

13. The method according to claim 12, characterized in that The image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes: In the thermal imaging image, determining a maximum grayscale value in the fourth area; The maximum grayscale value is converted into a temperature as the temperature of the target component.

14. The method according to claim 12, characterized in that The image data of the fourth area includes a grayscale value, and determining the temperature of the target component according to the image data of the fourth area includes: Acquire temperature measurement parameters of components in a third region, wherein the third region is a region where components in the target template image are located and has the highest matching degree with the region where the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature; According to the temperature measurement parameter, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

15. The method according to claim 14, characterized in that Before converting the grayscale value in the fourth area into temperature according to the temperature measurement parameter as the temperature of the target component, the method further includes: Acquiring a collection distance of the thermal imaging image; The step of converting the grayscale value in the fourth area into temperature according to the temperature measurement parameter comprises: According to the temperature measurement parameters and the acquisition distance of the thermal imaging image, the grayscale value in the fourth area is converted into temperature as the temperature of the target component.

16. The method according to claim 12, characterized in that The detecting whether the temperature of the target component is abnormal includes: Acquire a preset alarm temperature of a component in a third area, wherein the third area is an area in the target template image where the component is located and has the highest matching degree with the area where the target component is located; It is determined whether the temperature of the target component is greater than the preset alarm temperature. If so, it is determined that the temperature of the target component is abnormal; if not, it is determined that the temperature of the target component is normal.

17. The method according to any one of claims 1 to 11, characterized in that: The template image is generated in a manner including: For the second visible light image of the template PCB board, perform the following operations: Based on an adjustment instruction for the initial area where the components in the second visible light image are located, adjusting the initial area in the second visible light image, wherein the adjustment instruction includes: an adding instruction, a deleting instruction and / or a resizing instruction; and / or, Based on parameter information set for an initial region where components in the second visible light image are located, a correspondence between the initial region in the second visible light image and the parameter information is established, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

18. A PCB component detection device, characterized in that: The device comprises: A zooming and translation module is used for zooming and translating the template image and / or the first visible light image of the PCB board to be detected for each template image to obtain a zooming and translation result corresponding to the template image, so as to reduce the deviation between the field of view and the viewing angle of collecting the template image and the field of view and the viewing angle of collecting the first visible light image, wherein the template image is a visible light image of the template PCB board and includes the components of interest to the user in the template PCB board; A matching module, configured to calculate, for each template image, a degree of matching between the first visible light image and the template image based on a scaling and translation result corresponding to the template image; An image selection module, used for selecting a target template image from each template image based on the obtained matching degree; The component determination module is used to determine the components included in the target area in the first visible light image as target components, wherein the target area is an area that matches the area where the components in the target template image are located.

19. The device according to claim 18, characterized in that The device also includes: A component detection module, used for performing component detection on the first visible light image of the PCB board to be detected, and obtaining a first region of components in the first visible light image; The scaling and translation module is specifically used to scale, for each template image, a first region of components in the first visible light image and / or a second region of components in the template image based on a ratio between a collection distance of the template image and a collection distance of the first visible light image, and to translate, based on the scaling result, the first region of components in the first visible light image and / or the second region of components in the template image so that a first region in the first visible light image at a first orientation is aligned with a second region in the template image at a first orientation, thereby obtaining a scaling and translation result corresponding to the template image; and / or, The zoom and translation module includes: an area determination unit, used to determine, based on the zoom result corresponding to the template image, a first area in the first visible light image at a first orientation, and determine a second area in the template image at the first orientation; a first area translation unit, used to perform an overall translation of all first areas in the first visible light image in a first coordinate system of the first visible light image, so that the first areas in the first orientation are aligned with a first reference point in the first coordinate system; a second area translation unit, used to perform an overall translation of all second areas in the template image in a second coordinate system of the template image, so that the second areas in the first orientation are aligned with a second reference point in the second coordinate system, wherein the first reference point is aligned with the second reference point; and / or, The zoom and translation module includes: a region determining unit, used to determine, based on the zoom result corresponding to the template image, a first region in the first visible light image at a first position, and determine a second region in the template image at the first position; a region translation unit, used to perform an overall translation of all second regions in the template image in a coordinate system of the first visible light image, so that the second region in the first position is aligned with the first region in the first position in the first visible light image; or, in the coordinate system of the template image, perform an overall translation of all first regions in the first visible light image, so that the first region in the first position is aligned with the second region in the template image at the first position; and / or, The device further includes: a judgment module, configured to judge whether the maximum matching degree between the first visible light image and each template image is greater than a preset threshold; if the judgment result is yes, trigger the execution of the image selection module; if the judgment result is no, update the first orientation to other orientations among the preset multiple orientations except the current first orientation, and trigger the execution of the zoom and translation module; and / or, The matching module includes: a region matching unit, for each template image, based on the scaling and translation result corresponding to the template image, matching a first region of the component in the first visible light image with a second region of the component in the template image, and obtaining a matching degree between the first visible light image and the template image based on the matching result; and / or, The region matching unit includes: a matching number determining subunit, used for matching the first region of the component in the first visible light image with the second region of the component in the template image for each template image based on the scaling and translation result corresponding to the template image, to obtain a first number of first regions that are successfully matched; a matching degree calculating subunit, used for calculating a ratio of the first number to the total number of second regions in the template image, and obtaining a matching degree between the first visible light image and the template image based on the ratio; and / or, The matching number determination subunit is specifically used to calculate, for each first area, the matching degree between the first area and the second area of ​​the component in the template image, and if the maximum value of the calculated matching degrees is greater than a first preset matching degree threshold, determine that the first area is a first area that has successfully matched; and obtain a first number of first areas that have successfully matched; and / or, The matching degree between the first region and the second region is calculated in the following manner: a first area where the first region and the second region intersect is obtained; a difference between the sum of the areas of the first region and the second region and the first area is obtained as the second area; and a ratio of the first area to the second area is calculated as the matching degree between the first region and the second region. and / or, The component determination module includes: a region screening unit, used to screen out a region where the component in the first visible light image is located, and the region whose matching degree with the region where the component in the target template image is located is greater than a second preset matching degree threshold, as a target region; a target component determination unit, used to determine the component included in the target region as a target component; and / or, The device further includes: the component adjustment module, which is used to remove the target components whose area size is smaller than a preset size threshold from the determined target components; and / or, taking the center point of the area where the target component is located as a base point, amplifying the area where the target component is located according to a preset magnification factor and / or the size of a third area, wherein the third area is the area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located; and / or, The device further includes: a thermal imaging image acquisition module, used to acquire a thermal imaging image of the PCB board to be detected that is acquired synchronously with the first visible light image; a region determination module, used to determine, in the thermal imaging image, a fourth region corresponding to the position of the region where the target component is located; a temperature determination module, used to determine the temperature of the target component based on image data of the fourth region; and a temperature detection module, used to detect whether the temperature of the target component is abnormal. and / or, The temperature determination module is specifically used to determine the maximum grayscale value in the fourth area in the thermal imaging image; convert the maximum grayscale value into a temperature as the temperature of the target component; and / or, The temperature determination module is specifically used to obtain temperature measurement parameters of components in a third area, wherein the third area is an area in which components in the target template image are located and has the highest matching degree with the area in which the target component is located, and the temperature measurement parameters are parameters used to convert grayscale values ​​into temperature; according to the temperature measurement parameters, the grayscale values ​​in the fourth area are converted into temperature as the temperature of the target component; and / or, The device further includes: an acquisition distance acquisition module, used to acquire the acquisition distance of the thermal imaging image; the temperature determination module, specifically used to convert the grayscale value in the fourth area into temperature as the temperature of the target component according to the temperature measurement parameter and the acquisition distance of the thermal imaging image; and / or, The temperature detection module includes: an alarm temperature acquisition unit, used to acquire a preset alarm temperature of a component in a third area, wherein the third area is an area where the component in the target template image is located, and has the greatest matching degree with the area where the target component is located; a temperature detection unit, used to determine whether the temperature of the target component is greater than the preset alarm temperature, if so, determining that the temperature of the target component is abnormal, and if not, determining that the temperature of the target component is normal; and / or, The method for generating the template image includes: for a second visible light image of a template PCB board, performing the following operations: adjusting the initial area in the second visible light image based on an adjustment instruction for the initial area where the components in the second visible light image are located, wherein the adjustment instruction includes: adding instructions, deleting instructions and / or resizing instructions; and / or, based on parameter information set for the initial area where the components in the second visible light image are located, establishing a correspondence between the initial area in the second visible light image and the parameter information, wherein the parameter information includes: temperature measurement parameters and / or alarm temperature.

20. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-17 when executing a program stored in a memory.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Images