Circuit board positioning method, device, equipment and storage medium

Abstract

The present application discloses a circuit board positioning method, apparatus, device, and storage medium, relating to the field of post-furnace positioning technology for circuit boards. The method comprises: acquiring a target image obtained by photographing a conveyor belt used to transport circuit boards; segmenting the target image to obtain corresponding board masks for the circuit boards, with each circuit board corresponding to a board mask; determining a circumscribed rectangular area of ​​the circuit board in the target image based on the board mask; and determining the positioning result of the circuit board based on the circumscribed rectangular area. The above method can solve the technical problem of low positioning accuracy when using feature points to locate complex circuit boards in related technologies.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of post-furnace positioning of circuit boards, and in particular to a circuit board positioning method, device, equipment, and storage medium. Background Art

[0002] Post-furnace automated optical inspection (AOI) is the final step in PCB assembly production. It can detect board defects, such as the correct placement of components. Typically, after passing through post-furnace processing, boards are scattered on the conveyor belt. Therefore, accurately locating boards on the conveyor belt is crucial for detecting defects.

[0003] At present, the circuit boards on the conveyor belt are mainly located by feature point matching. The specific process is as follows: the circuit boards on the conveyor belt are photographed to obtain an image, wherein the circuit boards on the conveyor belt are the circuit boards after passing through the furnace. Then, feature point detection methods such as ORB (Oriented FAST and Rotated BRIEF), Scale-Invariant Feature Transform (SIFT) or SURF (Speeded Up Robust Features) are used to detect the feature points of the circuit board in the image. Then, the feature points are matched with the feature points of the template, and the circuit board is corrected according to the matching results, that is, the circuit board is corrected to maintain maximum consistency with the template, and the corrected circuit board is used as the located circuit board for subsequent detection of defective conditions on the circuit board.

[0004] However, when the circuit board has messy wires, different bases, or repeated textures, the accuracy of circuit board feature point extraction will be reduced. At this time, the wrong feature points will lead to wrong matching results, which will make the circuit board positioning accuracy low, which is not conducive to the subsequent detection of circuit board defects. Summary of the Invention

[0005] The embodiments of the present application provide a circuit board positioning method, apparatus, device, and storage medium to solve the technical problem of low positioning accuracy when using feature points to position complex circuit boards in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a circuit board positioning method, comprising:

[0007] Acquiring a target image obtained by photographing a conveyor belt, wherein the conveyor belt is used to transport circuit boards;

[0008] Segmenting a board mask corresponding to the circuit board from the target image, where each circuit board corresponds to one board mask;

[0009] Determine a circumscribed rectangular area of ​​the circuit board in the target image according to the board mask;

[0010] The positioning result of the circuit board is determined according to the circumscribed rectangular area.

[0011] In a second aspect, an embodiment of the present application further provides a circuit board positioning device, comprising:

[0012] An image acquisition module, configured to acquire a target image obtained by photographing a conveyor belt used to transport circuit boards;

[0013] a mask determination module, configured to segment the board mask corresponding to the circuit board from the target image, wherein each circuit board corresponds to one board mask;

[0014] an area determination module, configured to determine a circumscribed rectangular area of ​​the circuit board in the target image according to the board mask;

[0015] A positioning determination module is used to determine the positioning result of the circuit board according to the circumscribed rectangular area.

[0016] In a third aspect, an embodiment of the present application further provides a circuit board positioning device, comprising:

[0017] one or more processors;

[0018] a memory for storing one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the circuit board positioning method as described in the first aspect.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the circuit board positioning method as described in the first aspect.

[0021] The above-mentioned circuit board positioning method, device, equipment and storage medium obtain a target image of the conveyor belt when the conveyor belt transports the circuit board, and then segment the board mask of each circuit board from the target image, determine the circumscribed rectangular area of ​​the circuit board in the target image according to the board mask, and position the circuit board according to the circumscribed rectangular area. The technical means solves the technical problem of low positioning accuracy when using feature points to position complex circuit boards in the related art. In the above-mentioned positioning process, it is only necessary to identify the board mask of the circuit board to achieve circuit board positioning, and there is no need to identify the feature points of the circuit board, which reduces the dependence on the feature points. Even if the circuit board has messy wires, different bases or repeated textures, the circuit board can be positioned more accurately, which facilitates the subsequent detection of circuit board defects and improves detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a circuit board positioning method provided in an embodiment of the present application.

[0023] Figure 2 A schematic diagram of a board mask provided in an embodiment of the present application;

[0024] Figure 3 An example diagram of a circumscribed rectangular area provided in an embodiment of the present application;

[0025] Figure 4 A flowchart of a circuit board positioning method provided in an embodiment of the present application;

[0026] Figure 5 Another example diagram of a circumscribed rectangular area provided in an embodiment of the present application;

[0027] Figure 6 A schematic structural diagram of a circuit board positioning device provided in an embodiment of the present application;

[0028] Figure 7 A schematic structural diagram of a circuit board positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended to explain the present application, not to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present application, not all structures.

[0030] The circuit board positioning method provided in the embodiment of the present application can be executed by a circuit board positioning device, which can be implemented by software and / or hardware. The circuit board positioning device can be composed of two or more physical entities, or it can be composed of one physical entity, and the embodiment does not limit this. For example, the circuit board positioning device can be an electronic device with data processing capabilities such as a computer and a tablet computer. In the embodiment, the circuit board positioning method is used to position the circuit board on the conveyor belt. Among them, positioning the circuit board can be understood as accurately finding the circuit board in the image captured by the conveyor belt when the conveyor belt transports the circuit board, so as to facilitate subsequent detection of adverse conditions.

[0031] Figure 1 This is a flow chart of a circuit board positioning method provided in an embodiment of the present application. Figure 1 , the method specifically includes:

[0032] Step 110 : Acquire a target image obtained by photographing a conveyor belt, where the conveyor belt is used to transport circuit boards.

[0033] Exemplarily, a conveyor belt is used to transport circuit boards, and each circuit board is scattered on the conveyor belt. In one embodiment, the circuit boards transported on the conveyor belt are those that have undergone post-processing in a furnace, and the connection relationship and arrangement of components on each circuit board are identical. While the circuit boards are placed on the conveyor belt for transport, a camera is used to photograph the conveyor belt to obtain an image containing the circuit boards on the conveyor belt. In this embodiment, the captured image is recorded as a target image. The camera can be installed in the circuit board positioning device or as an external device. In this embodiment, an external device is used as an example. In this case, the camera transmits the target image to the circuit board positioning device via wired or wireless communication. Optionally, the camera and the conveyor belt can have a fixed or flexible relative positional relationship. In this embodiment, a fixed relative positional relationship is used as an example. In this case, each target image captured by the camera has the same shooting angle. Exemplarily, the camera can photograph the conveyor belt at a set frequency, or the conveyor belt and camera can be manually controlled to capture all circuit boards during transport. It can be understood that the circuit board positioning device can obtain each target image taken by the shooting device and perform subsequent processing on each target image.

[0034] It should be noted that the size of the target image is not limited in this embodiment.

[0035] Step 120: Segment the board mask corresponding to the circuit board from the target image, where each circuit board corresponds to one board mask.

[0036] For example, the board mask can reflect the outline of the circuit board and the placement of the circuit board in the target image. Figure 2 A schematic diagram of a board mask provided in an embodiment of the present application can be understood as follows: Figure 2 is a binary image, where the white area is the board mask corresponding to a circuit board, and the black area is other parts of the target image (such as the conveyor belt in the target image). Figure 2 It can be seen that the circuit board is located at the edge of the target image, and some areas may be located outside the target image, and the circuit board is placed at an angle. It can be understood that after processing the target image, the board mask corresponding to each circuit board in the target image can be segmented. Each board mask can be located in a binary image, and the placement and state of each board mask in the binary image are the same as the placement and state of the corresponding circuit board in the target image. That is, the aforementioned binary image and the target image have the same size and the same two-dimensional coordinate system. In one embodiment, the origin of the two-dimensional coordinate system of both images is the pixel in the upper left corner.

[0037] It should be noted that in this embodiment, the target image includes at least one circuit board. However, in actual applications, there are cases where the target image does not include a circuit board. In such cases, the circuit board mask will not be segmented from the target image. When it is determined that the circuit board mask cannot be segmented from the target image, processing of the target image is terminated.

[0038] Exemplarily, the specific process of segmenting the board mask of the circuit board in the target image can be: identifying the pixel points representing the circuit board in the target image, and obtaining the board mask of the circuit board based on the pixel points representing the circuit board, and then segmenting the board mask, that is, obtaining a binary image containing the board mask. The specific technical means for implementing the above process can be set according to actual conditions, such as using neural network technology to construct a segmentation neural network model to segment the board mask of the circuit board in the target image using the segmentation neural network model. In one embodiment, SOLOv2 is used as the segmentation neural network model, wherein SOLOv2 is an efficient real-time segmentation neural network. The specific network structure and model parameters of SOLOv2 can be set according to actual conditions, and the embodiment does not limit this. The SOLOv2 used in this step is a trained segmentation neural network model. In one embodiment, the implementation method for obtaining the plate mask using SOLOv2 is as follows: the target image is divided into multiple grids. If the center of mass of the object in the target image falls in a certain grid, then the grid has two tasks: one is to use the classification branch to predict the semantics of the object in the grid, and the other is to use the mask branch to predict the instance mask of the object (in the embodiment, the instance mask is a plate mask). For example, the number of grids is pre-set, and then the target image is divided into multiple networks according to the number of grids. For example, if the number of networks is S×S, then the target image is divided into S×S grids, where S is greater than 1. The number of grids can be a pre-set fixed number, and the number of networks can also be determined in combination with the size of the target image. For example, the larger the size of the target image, the more grids there are. Afterwards, it is determined whether the center of mass of the object in the target image falls in any network, wherein the center of mass of the object can also be replaced by the center of the object. If it falls in any network, the classification branch is used to predict the semantics of the object. The classification branch can predict the semantics of an object within a defined semantic set. In one embodiment, the defined semantic set includes background and circuit board. In this case, the classification branch can determine whether the object in the target image is background or circuit board, thereby identifying the object in the target image that has the semantics of a circuit board. The mask branch is then used to predict the instance mask of the object with the category of circuit board to obtain the corresponding board mask. Specifically, the mask branch determines the object with the category of circuit board in the target image and generates a corresponding instance mask based on the object, i.e., a board mask. Subsequently, a binary image representing the board mask is output.

[0039] Step 130: Determine the circumscribed rectangular area of ​​the circuit board in the target image according to the board mask.

[0040] In this embodiment, the circumscribed rectangular area of ​​the circuit board in the target image refers to the smallest rectangular area that contains the circuit board. That is, the circumscribed rectangular area contains the circuit board, and the area of ​​the circumscribed rectangular area is the minimum area that contains the circuit board. For example, the circumscribed rectangular area can be determined by a board mask, that is, the circumscribed rectangular area can also be understood as the smallest rectangular area that contains the board mask. In this case, each circuit board corresponds to a circumscribed rectangular area. It is understood that the circumscribed rectangular area can be a rectangular area or a square area, and this embodiment is not limited to this.

[0041] Exemplarily, the method for determining the circumscribed rectangular area can be set according to actual conditions. For example, the minAreaRect() method of OpenCV is used to obtain the circumscribed rectangular area. Among them, OpenCV is a cross-platform computer vision and machine learning software library released under the BSD license (open source). MinAreaRect refers to the minimum regular rectangle that covers the contour. At this time, the circumscribed rectangular area of ​​the board mask can be obtained through the minAreaRect() method in OpenCV. Generally speaking, the circumscribed rectangular area constructed by this method is usually a non-tilted rectangle, wherein the non-tilted rectangle can be understood as each side of the circumscribed rectangular area is parallel to the corresponding boundaries in the target image (or the binary image where the board contour is located), and the circumscribed rectangular area is generally not tilted. For example, Figure 3 An example diagram of a circumscribed rectangular area provided in an embodiment of the present application, which shows the use of OpenCV minAreaRect() to Figure 2 The board mask in the image is processed to obtain the circumscribed rectangular area 11, from Figure 3 It can be seen that the sides of the circumscribed rectangular area 11 are parallel to the corresponding boundary in the binary image (including the case where they coincide with the boundary). It should be noted that in the subsequent positioning process of the embodiment, the circuit board needs to be extracted from the target image based on the circumscribed rectangular area. However, due to Figure 3 The board mask in is in a tilted state. Therefore, the bounding rectangle area determined by the tilted board mask is not the true minimum bounding rectangle area. Figure 3 The upper right corner and lower right corner of the circumscribed rectangular area contain more black areas (i.e., background parts). At this time, according to Figure 3The circumscribed rectangular area shown will contain more background parts when extracting the circuit board from the target image. In one embodiment, in order to reduce the influence of the background part on the positioning result, the outline of the board mask can also be used to determine the circumscribed rectangular area. For example, the edge in the outline of the board mask is used as the edge of the minimum circumscribed rectangle. At this time, a minimum circumscribed rectangle corresponding to each edge is determined, and then the area corresponding to the minimum circumscribed rectangle with the smallest area in each minimum circumscribed rectangle is taken as the circumscribed rectangular area. After such processing, when the circuit board is tilted, the corresponding circumscribed rectangular area can also be a tilted rectangular area, so that the circumscribed rectangular area is as close to the circuit board as possible. It can be understood that since the outline of the board mask may be an irregular shape, for example, Figure 2 The board mask shown is an irregular shape with convexities and concavities in the outline of its edge. In this case, not every edge has a bounding rectangle area. Therefore, in one embodiment, only the edges with the minimum bounding rectangle can be selected, and the minimum bounding rectangle with the smallest area among the minimum bounding rectangles corresponding to these edges can be selected as the calculated bounding rectangle area. Alternatively, the convex hull of the board mask is determined, and each edge in the convex hull is used as the edge of the minimum bounding rectangle. Then, the minimum bounding rectangle when each edge contains the board mask is calculated. At this time, each edge in the convex hull corresponds to a minimum bounding rectangle. Then, the area corresponding to the minimum bounding rectangle with the smallest area is selected as the bounding rectangle area. Among them, the convex hull is a concept in computational geometry (graphics). In the embodiment, the convex hull can be understood as a convex polygon obtained by connecting the outermost points (which can also be understood as pixel points) in the board mask. At this time, the bounding rectangle area obtained based on the outline or convex hull of the board mask is closer to the board mask, reducing the background part in the area. It can be understood that only part of the circuit board may be located on the conveyor belt, for example, Figure 2 Only part of the circuit board corresponding to the middle card mask is located on the conveyor belt. At this time, when determining the circumscribed rectangular area containing the card mask, only the circumscribed rectangular area corresponding to the part of the circuit board on the conveyor belt can be retained, that is, the intersection of the circumscribed rectangular area and the binary image where the card mask is located (or with the target image) is used as the circumscribed rectangular area containing the card mask. It can be understood that the intersection may not be a rectangle, but may be any shape.

[0042] Step 140: Determine the positioning result of the circuit board according to the circumscribed rectangular area.

[0043] For example, the positioning result is an identification result of the circuit board in the target image, and the positioning result can be used to detect defective conditions of the circuit board.

[0044] In one embodiment, the circuit board image contained in the circumscribed rectangular area is extracted from the target image according to the circumscribed rectangular area as the positioned image, that is, as the positioning result of the circuit board.

[0045] In one embodiment, the image extracted based on the circumscribed rectangular area is recorded as a board image. At this time, since the circuit boards are transported by the conveyor belt, the circuit boards are in a scattered state, that is, the circuit boards on the conveyor belt may be placed in various directions. For example, Figure 2 The circuit board corresponding to the middle board mask is tilted on the conveyor belt. In this case, the board image extracted based on the circumscribed rectangular area is also a tilted rectangular image. The tilted board image is not conducive to subsequent processing, such as when comparing it with a reference board image corresponding to the circuit board (an image taken when the circuit board is accurately and non-tilted) to find incorrectly installed components. Therefore, in the embodiment, after extracting the tilted board image, it is necessary to rotate the tilted board image into a non-tilted board image, that is, the edges of the rotated board image are parallel to the corresponding edges in the target image. Afterwards, the non-tilted board image is used as the positioning result. Generally speaking, for a non-tilted board image, the circuit board also has multiple placement states. For example, taking a certain corner of the circuit board as a reference, for ease of description, this corner is recorded as the target angle. When the board image is a rectangular image, the circuit board corresponds to four placement states: the target angle is located at the upper left corner, the lower left corner, the upper right corner, and the lower right corner. Among them, only one placement state is closest to the circuit board shown in the reference board image. Therefore, it is necessary to find the placement state closest to the reference board image in the non-tilted board image. Therefore, in an embodiment, the non-tilted board image can be rotated to correspond to each placement state. Then, after each rotation, the similarity between the non-tilted board image and the reference board image is calculated, and the most similar placement state is selected based on the similarity. The non-tilted board image in this placement state is used as the positioning result. In one embodiment, after rotating a tilted board image into a non-tilted board image, the non-tilted board image may be corrected first to map the non-tilted board image to a reference board image, i.e., the position of each pixel in the non-tilted board image in the reference board image may be determined, wherein a homography matrix between the reference board image and the circumscribed rectangular area may be determined, so that the non-tilted board image can be mapped to the reference board image via the homography matrix.

[0046] It is understood that when a circuit board has a missing corner, it is first determined whether the missing corner is too large. If so, the circuit board is not positioned. It is also understood that when a circuit board has a missing corner, the missing corner will appear as a conveyor belt in the target image, that is, as background. Therefore, it is possible to determine whether the number of pixels representing the background in the board image exceeds a threshold. If so, the missing corner is too large. The threshold can be set based on actual conditions. By setting the threshold, it can be ensured that only circuit boards with smaller missing corners are positioned.

[0047] The above-mentioned technical means solves the technical problem of low positioning accuracy when using feature points to position complex circuit boards in the related art. In the above-mentioned positioning process, it is only necessary to identify the board mask of the circuit board to achieve circuit board positioning. There is no need to identify the feature points of the circuit board, which reduces the degree of dependence on feature points. Even if the circuit board has messy wires, different bases or repeated textures, the circuit board can be positioned more accurately, which facilitates the subsequent detection of defective conditions of the circuit board and improves the detection efficiency.

[0048] Figure 4 This is a flow chart of a circuit board positioning method provided in an embodiment of the present application. This embodiment is a specific implementation based on the above embodiment.

[0049] Specifically, refer to Figure 4 , the circuit board positioning method includes:

[0050] Step 210 , acquiring a target image obtained by photographing a conveyor belt, which is used to transport circuit boards. Step 220 , segmenting the board masks corresponding to the circuit boards from the target image, where each circuit board corresponds to one board mask.

[0051] Step 230: Obtain a board outline of the circuit board according to the board mask.

[0052] For example, since the board mask can determine the shape of the circuit board, the outline of the circuit board can be obtained based on the board mask. Specifically, edge pixels of the board mask are obtained, and the outline of the circuit board is obtained based on the edge pixels. In this embodiment, the outline of the circuit board is recorded as the board outline. In this case, each board mask corresponds to a board outline.

[0053] Step 240: Determine the circumscribed rectangular area of ​​the circuit board in the target image based on the convex hull of the board outline.

[0054] Exemplarily, the convex hull corresponding to the board outline is obtained, and the convex hull includes the outermost points of the board outline. Then, the circumscribed rectangular area of ​​the circuit board is determined based on the convex hull. In one embodiment, when determining the circumscribed rectangular area, the minimum circumscribed rectangle corresponding to each side of the convex hull is determined respectively. In the embodiment, each corresponding minimum circumscribed rectangle is recorded as an alternative circumscribed rectangle. At this time, each side corresponds to an alternative circumscribed rectangle, and one side of the alternative circumscribed rectangle is collinear with the corresponding side in the convex hull (i.e., overlaps) and the alternative circumscribed rectangle includes the board outline. Then, the alternative circumscribed rectangle with the smallest area among the alternative circumscribed rectangles corresponding to each side is selected as the final circumscribed rectangular area of ​​the circuit board.

[0055] In one embodiment, step 240 specifically includes steps 241 to 245:

[0056] Step 241: Obtain the convex hull of the board outline.

[0057] Exemplarily, the outermost point in the board outline is first found, wherein the outermost point can be determined by the coordinates of the board outline. In one embodiment, a two-dimensional coordinate system is established for the binary image representing the board mask. Optionally, the two-dimensional coordinate system is established in the same manner as the two-dimensional coordinate system of the target image. For example, the pixel point in the upper left corner is used as the origin of the two-dimensional coordinate system to establish the two-dimensional coordinate system. At this time, each pixel point in the board outline has a corresponding two-dimensional coordinate, and the two-dimensional coordinates of the pixel point are the same as the two-dimensional coordinates of the corresponding pixel point of the circuit board in the target image. Afterwards, the pixel point at the outermost edge is determined based on the two-dimensional coordinates of each pixel point in the board outline, and is used as the outermost point of the board outline. Afterwards, the adjacent outermost points are connected in sequence to obtain a convex polygon, which is the convex hull of the board outline.

[0058] Step 242: Determine a candidate circumscribed rectangle corresponding to each edge in the convex hull. The candidate circumscribed rectangle is the minimum circumscribed rectangle that contains the outline of the board. One edge in the candidate circumscribed rectangle is collinear with the corresponding edge in the convex hull.

[0059] Exemplarily, for a convex polygon, the minimum circumscribed rectangle containing the convex polygon is generally collinear with one of the sides of the convex polygon. Therefore, in an embodiment, an alternative circumscribed rectangle corresponding to each side of the convex hull is determined separately. One side of the alternative circumscribed rectangle is collinear with the corresponding side of the convex hull and is the minimum circumscribed rectangle that currently contains the convex hull (which can also be understood as containing the board outline). In one embodiment, when determining the alternative circumscribed rectangle corresponding to each side, the straight line on which the side is located can be used as the side of the alternative circumscribed rectangle, and a rectangular area is drawn on the straight line, and the rectangular area contains the convex hull. After that, the rectangular area with the smallest area is determined as the alternative circumscribed rectangle corresponding to the side. It is understandable that for the convex hull, not every side is parallel to the coordinate axis of the two-dimensional coordinate system where the board outline is located. Therefore, when drawing the alternative circumscribed rectangle based on the sides of the convex hull, the alternative circumscribed rectangle may be a tilted rectangle. It is understandable that for the circuit board positioning device, the difficulty of drawing a tilted rectangle is higher than that of a non-tilted rectangle. A non-tilted rectangle refers to a rectangle in which two adjacent sides are parallel to two coordinate axes of the two-dimensional coordinate system where the board outline is located. Therefore, in an embodiment, the convex hull can be rotated first so that when drawing the candidate circumscribed rectangles for each side, non-tilted rectangles are drawn. In this case, determining the candidate circumscribed rectangle corresponding to each side of the convex hull includes steps 2421 to 2424:

[0060] Step 2421: Determine the first angle corresponding to each edge in the convex hull, where the first angle is the angle between the corresponding edge and the set coordinate axis, and the set coordinate axis is the coordinate axis used in the coordinate system where the board outline is located.

[0061] In the embodiment, the candidate circumscribed rectangle corresponding to each side in the convex hull is determined in sequence, and the specific technical means for determining the candidate circumscribed rectangle of each side is the same. Therefore, in the embodiment, taking an edge in the convex hull as an example, how to determine the candidate circumscribed rectangle corresponding to the edge is described. For ease of understanding, the edge of the convex hull for which the candidate circumscribed rectangle is currently determined is recorded as the target edge.

[0062] Exemplarily, the angle between the target side in the convex hull and the set coordinate axis is determined. In the embodiment, the angle is recorded as the first angle. Wherein, the set coordinate axis is the coordinate axis used in the two-dimensional coordinate system where the board outline is located. It is understandable that the coordinate axis of the two-dimensional coordinate system includes an X-axis and a Y-axis. Therefore, the set coordinate axis can be an X-axis or a Y-axis, and the embodiment does not limit this. Afterwards, when determining the angle between the target side in the convex hull and the set coordinate axis, it can be when the target side intersects with the set coordinate axis, determining the angle between the target side and the set coordinate axis, and determining the angle as the first angle. When the target side does not intersect with the set coordinate axis, extending the target side and intersecting with the set coordinate axis, then determining the angle between the extended target side and the set coordinate axis, and then using the angle as the first angle corresponding to the target side. It is understandable that in actual applications, other methods can also be used to determine the first angle, and the embodiment does not limit this.

[0063] Step 2422: Reversely rotate each point in the convex hull according to the first angle so that the side corresponding to the first angle is parallel to the set coordinate axis after reverse rotation.

[0064] Exemplarily, reverse rotation is a relative concept, which corresponds to forward rotation, and both reverse rotation and forward rotation are rotations in the two-dimensional coordinate system corresponding to the board outline. In the embodiment, assuming that the first angle is θ, reverse rotation refers to rotating all the points in the convex hull by -θ, and after the rotation, the target edge in the convex hull (i.e., the edge corresponding to the first angle) is parallel to the set coordinate axis. Optionally, when all the points in the convex hull are reversely rotated, the board mask in the convex hull is also synchronously reversed. It is understandable that after the rotation, there may be a situation where the target edge coincides with the set coordinate axis. In the embodiment, this situation is classified as a situation where the coordinate axis is parallel to the set coordinate axis.

[0065] Step 2423: Determine the minimum circumscribed rectangle containing the outline of the board after reverse rotation, and the side corresponding to the first angle is collinear with a side in the minimum circumscribed rectangle after reverse rotation.

[0066] Exemplarily, after the convex hull is reversely rotated, a bounding rectangle containing the board outline (i.e., containing the board mask) is drawn based on the target edge (i.e., the edge corresponding to the first angle), and one side of the bounding rectangle is collinear with the target edge, where collinearity can be understood as the two being on a straight line. At this time, since the target edge is parallel to the set coordinate axis after reverse rotation, the bounding rectangle drawn based on the target edge is a non-inclined rectangle. Afterwards, the bounding rectangle with the smallest area is determined in the drawn bounding rectangles, that is, the minimum bounding rectangle is determined. It should be noted that since the convex hull is drawn based on the outermost points of the board outline, when drawing a bounding rectangle containing the board outline based on the target edge, a bounding rectangle containing the convex hull can also be drawn based on the target edge.

[0067] Step 2424: forwardly rotate the minimum bounding rectangle according to the first angle, and use the forwardly rotated minimum bounding rectangle as a candidate bounding rectangle for the side corresponding to the first angle.

[0068] Exemplarily, after determining the minimum bounding rectangle of the target side (i.e., the side corresponding to the first angle), the minimum bounding rectangle is rotated forward according to the first angle. The minimum bounding rectangle after the forward rotation can include the board outline before the reverse rotation (i.e., include the convex hull before the reverse rotation) and one of its sides is collinear with the target side in the convex hull before the reverse rotation. In this case, the minimum bounding rectangle after the forward rotation is recorded as the candidate bounding rectangle of the target side. In this case, the candidate bounding rectangle can be a tilted rectangle.

[0069] In one embodiment, assuming the first angle is θ, the minimum circumscribed rectangle is {X min ,Y min ,X max ,Y max}, at this time, when the minimum circumscribed rectangle is rotated in the positive direction, the four corner points (X min ,Y min ), (X min ,Y max ), (X max ,Y min ), (X max ,Y max ) are rotated by θ respectively, and then the four corner points are connected in sequence to obtain the alternative circumscribed rectangle of the target edge.

[0070] According to the above method, the candidate circumscribed rectangle corresponding to each side of the convex hull can be obtained.

[0071] Step 243: Calculate the area of ​​each candidate circumscribed rectangle.

[0072] In the embodiment, the method for calculating the area of ​​the candidate circumscribed rectangle is not limited.

[0073] It is understandable that in actual applications, there may be a circuit board where only part of the area is displayed in the target image. In this case, the board mask is also only a board mask of part of the area. Then, among the determined alternative bounding rectangles, there may be an alternative bounding rectangle that exceeds the boundary of the image where the board mask is located, that is, there may be an alternative bounding rectangle that exceeds the boundary of the target image. In this case, only the area corresponding to the partial area of ​​the alternative bounding rectangle retained in the target image can be calculated. Accordingly, this step also includes: if part of the area of ​​the alternative bounding rectangle exceeds the boundary of the target image, the area of ​​the intersection area of ​​the alternative bounding rectangle and the target image is used as the area of ​​the alternative bounding rectangle.

[0074] Exemplarily, a determination is made as to whether any portion of the candidate bounding rectangle exceeds the boundary of the target image, where the boundary of the target image can also be understood as the image frame of the target image. Optionally, the candidate bounding rectangle is drawn in the target image according to its two-dimensional coordinates. Thereafter, a determination is made as to whether the candidate bounding rectangle exceeds the boundary of the target image. If not, the area of ​​the candidate bounding rectangle is calculated normally. If so, the intersection area of ​​the candidate bounding rectangle and the target image is determined, and the area of ​​the intersection area is used as the area of ​​the candidate bounding rectangle.

[0075] Step 244: Select the candidate circumscribed rectangle with the smallest area.

[0076] It can be understood that since the size of the circuit board in each candidate circumscribed rectangle is fixed, the smaller the area of ​​the candidate circumscribed rectangle, the smaller the area representing the background inside it. Accordingly, in this embodiment, among the candidate circumscribed rectangles, the candidate circumscribed rectangle with the smallest area is selected.

[0077] Step 245: The display area of ​​the candidate circumscribed rectangle with the smallest area in the target image is used as the circumscribed rectangular area of ​​the circuit board in the target image.

[0078] Exemplarily, the display area of ​​the candidate bounding rectangle with the smallest area in the target image is determined, that is, the candidate bounding rectangle with the smallest area is drawn into the target image according to the two-dimensional coordinates of its sides, and the area of ​​the candidate bounding rectangle with the smallest area in the target image is used as the display area. Thereafter, the display area is determined as the bounding rectangle area of ​​the circuit board. It is understandable that when the candidate bounding rectangle with the smallest area may partially exceed the boundary of the target image, the intersection area of ​​the candidate bounding rectangle with the target image is used as the bounding rectangle area. For example, Figure 5 This is another example diagram of a circumscribed rectangular area provided in an embodiment of the present application. Figure 5 The circumscribed rectangular area 12 of the plate mask is the circumscribed rectangular area determined according to the above technical means. The circumscribed rectangular area 12 is an inclined area, and the circumscribed rectangular area 12 only retains the portion that does not exceed the image.

[0079] Step 250: Extract the board image of the circuit board from the target image according to the circumscribed rectangular area.

[0080] In one embodiment, an image within a circumscribed rectangular area of ​​a target image is extracted and used as a circuit board image of the circuit board. Specifically, the circuit board image includes the circuit board and contains minimal background. Optionally, when extracting the image, background pixels can be set to 0. Specifically, when extracting the image, pixels representing the background are determined based on a circuit board mask (i.e., pixels not in the circuit board mask are considered background pixels), and then the pixel values ​​of these pixels are set to 0.

[0081] It can be understood that when extracting the board image, if the board image is a tilted image, it is rotated into a non-tilted board image and used as the final extracted board image.

[0082] Step 260: Match the board image with the reference board image, and determine the positioning result of the circuit board according to the matching result.

[0083] Exemplarily, the reference board image refers to an image obtained by pre-photographing a reference circuit board. The reference circuit board and the circuit board conveyed on the conveyor belt have the same functions. The connection relationship and arrangement of components on the reference circuit board are the same as the connection relationship and arrangement of components on the conveyed circuit board, and the reference circuit board is a circuit board without any defects. By comparing the reference circuit board and the conveyed circuit board, any defects in the conveyed circuit board can be determined. The reference circuit board in the reference board image is a non-tilted circuit board. In one embodiment, the same type of circuit boards are conveyed on the conveyor belt, and therefore, each circuit board can share a reference board image.

[0084] Exemplarily, after obtaining a reference board image, the board image is matched with the reference board image to obtain a positioning result for the circuit board. In one embodiment, the matching specifically determines the placement state of the board image when the board image is most similar to the reference board image, and uses the board image in this placement state as the positioning result. Exemplarily, a board image is extracted from a target image, the board image is rotated multiple times, and the similarity between the board image after each rotation and the reference board image is compared. If the board image after a certain rotation is most similar to the reference board image, the currently rotated board image is determined as the obtained positioning result, which can facilitate subsequent comparison with the reference board image to determine any defects.

[0085] In one embodiment, when matching a board image with a reference board image, the board image is corrected and then compared with the reference board image. In this case, matching the board image with the reference board image includes steps 261 to 263:

[0086] Step 261: Acquire a reference board image of a circuit board.

[0087] Exemplarily, a reference board image is imported.

[0088] Step 262: Correct the board image according to the reference board image to obtain a corrected board image.

[0089] Exemplarily, correcting the board image refers to mapping the board image to the reference board image, so as to facilitate the subsequent comparison of the corrected board image and the reference board image. In the embodiment, the corrected board image is recorded as the board correction image. The means for correcting the board image can be set according to the actual situation. In one embodiment, the correction of the board image is achieved in the form of a homography matrix. At this time, step 262 specifically includes steps 2621-2622.

[0090] Step 2621: When the aspect ratio of the reference board image and the aspect ratio of the circumscribed rectangular area satisfy a similarity relationship, determine a homography matrix between the reference board image and the circumscribed rectangular area.

[0091] For example, if the circumscribed rectangular area is a tilted rectangular area, the circumscribed rectangular area is first rotated into a non-tilted rectangular area. At this time, the rotation direction and angle of the circumscribed rectangular area are the same as the rotation direction and angle used when rotating the tilted board image into the non-tilted board image in step 250. At this time, the non-tilted circumscribed rectangular area is recorded as R, and R = {X' min ,Y' min ,X' max ,Y' max}, at this time, the four corner points (i.e. vertices) of the circumscribed rectangular area are (X' min ,Y' min ), (X' max ,Y' min ), (X' max ,Y' max ), (X' min ,Y' max ). The reference board image is denoted as R', and R'={0,0,w-1,h-1}. At this time, the four corner points (i.e., vertices) of the reference board image are: (0,0), (w-1,0), (w-1,h-1), (0,h-1). It can be understood that when the circuit board and the reference circuit board adopt the same placement state, such as the circuit board and the reference circuit board are both rectangular circuit boards, the same placement state of the two circuit boards means that the long sides of the circuit board and the reference circuit board are placed as horizontal sides. At this time, the aspect ratio of the circumscribed rectangular area containing the circuit board and the reference board image should be similar. In the embodiment, the placement state of the circuit board is not fixed. Therefore, there may be a situation where the aspect ratio of the circumscribed rectangular area of ​​the circuit board and the reference board image are not similar. Therefore, in the embodiment, it is necessary to first ensure that the aspect ratio of the circumscribed rectangular area of ​​the circuit board and the reference board image are similar, and then perform correction to ensure the accuracy of the correction.

[0092] For example, the aspect ratio of the reference board image is obtained, wherein the width of the reference board image R' is w and the length is h, and then the aspect ratio of the reference board image is determined to be w / h. Similarly, the aspect ratio of the bounding rectangle area is obtained, wherein the width (X') of the bounding rectangle area R is max -X' min +1), the length is (Y' max -Y' min +1), then determine the aspect ratio of the circumscribed rectangular area as (X' max -X' min +1) / (Y' max -Y' min +1). Next, determine whether the aspect ratio of the reference plate image and the aspect ratio of the circumscribed rectangular area satisfy a similarity relationship. A similarity relationship means that the two aspect ratios are equal or approximately equal. It should be noted that the criteria for determining approximate equality can be set based on actual conditions, such as setting a ratio threshold. When the ratio between the two aspect ratios is within the ratio threshold, the two are determined to be approximately equal. If the aspect ratio of the reference plate image and the aspect ratio of the circumscribed rectangular area satisfy a similarity relationship, it means that the long side of the reference plate image and the long side of the circumscribed rectangular area are both wide or tall. At this point, the homography matrix can be directly calculated based on the four corner points of the circumscribed rectangular area and the reference plate image. Among them, the embodiment of the calculation method of the homography matrix is ​​not limited. For example, the function in OpenCV is called to calculate the homography matrix between the two images (i.e., the circumscribed rectangular area and the reference plate image) through the two-dimensional coordinates of 4 pairs of corresponding corner points (respectively, the corner point of the upper left corner, the corner point of the upper right corner, the corner point of the lower right corner, and the corner point of the lower left corner). After that, the points in the circumscribed rectangular area can be converted to the corresponding positions of the reference plate image through the homography matrix.

[0093] It will be appreciated that if the aspect ratio of the reference board image and the aspect ratio of the circumscribed rectangular area do not satisfy a similar relationship, the circumscribed rectangular area is rotated by the second angle so that the aspect ratio of the rotated circumscribed rectangular area satisfies a similar relationship with the aspect ratio of the reference board image. For example, if the aspect ratio of the reference board image and the aspect ratio of the circumscribed rectangular area do not satisfy a similar relationship, this indicates that their aspect ratios are opposite, meaning that one of the long sides of the reference board image and the long side of the circumscribed rectangular area serves as the width or the other serves as the height. In this case, the circumscribed rectangular area needs to be rotated so that the aspect ratios of the two satisfy a similar relationship.

[0094] In the embodiment, the rotation angle used when rotating the circumscribed rectangular area is recorded as the second angle. Generally speaking, rotating the circumscribed rectangular area by 90° can make the width and height of the circumscribed rectangular area interchangeable. Therefore, in one embodiment, 90° is used as the second angle. Optionally, the four corner points (X' min,Y' min ), (X' max ,Y' min ), (X' max ,Y' max ), (X' min ,Y' max ) are rotated 90° clockwise, and the four corner points of the circumscribed rectangular area after rotation are: (X' min ,Y' max ), (X' min ,Y' min ), (X' max ,Y' min ), (X' max ,Y' max ), where the aspect ratios of the rotated bounding rectangle and the reference plate image satisfy a similar relationship. Subsequently, a homography matrix is ​​determined based on the four corner points of the rotated bounding rectangle and the four corner points of the reference plate image. The homography matrix is ​​determined in the same manner as previously described and will not be further elaborated here.

[0095] Step 2622: Process the board image using the homography matrix to obtain a board-corrected image.

[0096] Exemplarily, when using the homography matrix to process the board image, the two-dimensional coordinates of each pixel point in the board image (i.e., similarity point) are multiplied by the homography matrix to transform each pixel point to the corresponding position of the reference board image to obtain the board correction image. It can be understood that the aspect ratio of the board correction image satisfies the similarity relationship with the aspect ratio of the reference board image.

[0097] Step 263: Match the board calibration image with the reference board image.

[0098] In one embodiment, a circuit board may have a missing corner. In this case, the circuit board itself is defective and may not function properly. Therefore, positioning of the circuit board in this case is not required, and subsequent defect detection is no longer required. Accordingly, in this embodiment, before step 263, the following steps are further performed: obtaining the number of zero-valued pixels in the board calibration image, where zero-valued pixels represent background pixels; and when the number of pixels is less than a threshold, matching the board calibration image with the reference board image.

[0099] Exemplarily, since the pixels representing the background have been set to 0 when the board image is extracted in step 250, the number of pixels with a pixel value of zero in the board correction image can be directly obtained in this step. This number is then compared with a number threshold, where the number threshold is a pre-set threshold. When the number is less than the number threshold, it indicates that the number of background pixels in the board image is small and there is no missing corner, or that the missing corner is not obvious and does not affect normal use, that is, it has no effect on subsequent defect detection. In this case, step 263 is directly executed. If the number is greater than the number threshold, it indicates that the number of background pixels in the board image is large, that is, the missing corner of the circuit board is too large. In this case, the subsequent steps can be discontinued, that is, the current circuit board is skipped and not matched with the reference board image. Optionally, when it is determined that the missing corner of the circuit board is too large, the circuit board can be marked in the target image to facilitate subsequent processing of the circuit board by the staff.

[0100] In one embodiment, in order to ensure the accuracy of the matching, before step 263 , the process further includes: scaling the board correction image and the reference board image to a target size respectively.

[0101] Among them, the target size can be set according to actual conditions. Generally speaking, the target size is smaller than the size of the plate correction image and the size of the reference plate image. In the embodiment, the target size of 32×32 is described as an example. Exemplarily, the plate correction image and the reference plate image are scaled to the target size respectively. At this time, the specific technical means used in scaling the image are not limited in the embodiment. It can be understood that after the scaling is completed, the plate correction image and the reference plate image have the same size, and the size is smaller than the size before scaling. This not only ensures the accuracy of the matching (that is, ensures the accuracy of the subsequent similarity calculation), but also reduces the amount of calculated data during matching. It should be noted that the plate correction image and the reference plate image mentioned later can be considered as the scaled plate correction image and the reference plate image.

[0102] For example, since the aspect ratio of the corrected board image and the reference board image have already been aligned during board image correction, it is necessary to determine the state of the corrected board image when it is most similar to the reference board image while maintaining the same aspect ratio, and use the corrected board image in this state as the positioning result. In this case, step 263 specifically includes steps 2631-2633.

[0103] Step 2631: Calculate a first similarity between the board correction image and the reference board image.

[0104] Exemplarily, the structural similarity (SSIM) is used to calculate the similarity between the board correction image and the reference board image. In the embodiment, the currently calculated similarity is recorded as the first similarity.

[0105] Afterwards, the width and height of the board correction image are obtained. If the width and height are equal or approximately equal, it means that the board correction image is a rectangular image, and step 2633 is executed. If the width and height are not equal, it means that the board correction image is a square image, and step 2632 is executed.

[0106] It should be noted that, in the embodiment, the judgment standard of approximately equality can be set according to actual conditions, such as setting a ratio threshold. When the ratio between the width and the height is within the ratio threshold, it is determined that the two are approximately equal.

[0107] Step 2632: If the board-calibrated image is a square image, rotate the board-calibrated image three times, each time by 90°, and calculate the second similarity between the board-calibrated image and the reference board image after each rotation.

[0108] For example, for a square image, the aspect ratio remains unchanged after each rotation of 90° in the same direction. Therefore, if the board correction image is a square image, it means that there are four rotation conditions of 0°, 90°, 180°, and 270° between the board correction image and the reference board image, that is, the board correction image is most similar to the reference board image in one of the four rotation conditions. Since the first similarity between the board correction image and the reference board image when the rotation angle is 0°, that is, when the board correction image is not rotated, has been calculated in step 2631, therefore, in this step, the board correction image is rotated 90° along the same direction each time with a rotation angle of 90°, and is rotated three times to correspond to the three rotation conditions of 90°, 180°, and 270°, respectively, wherein the embodiment of the rotation direction is not limited. Furthermore, after each rotation, the similarity between the rotated plate-corrected image and the reference plate-corrected image is calculated. In this embodiment, the similarity between the rotated plate-corrected image and the reference plate-corrected image is recorded as a second similarity. It is understood that when the plate-corrected image is a square image, there are three second similarities. The second similarities are calculated in the same manner as the first similarities.

[0109] Step 2633: If the board-corrected image is a long-direction image, rotate the board-corrected image by 180°, and calculate a second similarity between the board-corrected image and the reference board image after the rotation.

[0110] For example, for a rectangular image, its aspect ratio remains unchanged after each 180° rotation along the same direction. Therefore, if the corrected plate image is a rectangular image, this indicates that the corrected plate image and the reference plate image have two rotation conditions: 0° and 180°. That is, the corrected plate image is most similar to the reference plate image in one of the two rotation conditions. Since the first similarity between the corrected plate image and the reference plate image has already been calculated in step 2631 when the rotation angle is 0°, i.e., when the corrected plate image is not rotated, in this step, the corrected plate image is rotated 180° to correspond to the 180° rotation condition. The rotation direction is not limited in this embodiment. Furthermore, the similarity between the rotated corrected plate image and the reference plate image is calculated. In this embodiment, the similarity between the rotated corrected plate image and the reference plate image is recorded as a second similarity. It is understood that when the corrected plate image is a rectangular image, a second similarity exists. The second similarity is calculated in the same manner as the first similarity.

[0111] It is understood that in practical applications, after obtaining the board-calibrated image, the width and height of the board-calibrated image may be obtained first. When the board-calibrated image is determined to be a square image based on the width and height, the first similarity and the second similarity are calculated according to four rotation conditions: 0°, 90°, 180°, and 270°. When the board-calibrated image is determined to be a rectangular image based on the width and height, the first similarity and the second similarity are calculated according to two rotation conditions: 0° and 180°.

[0112] In one embodiment, since there are a first and a second similarity between the calibrated board image and the reference board image, i.e., multiple matching results exist between the calibrated board image and the reference board image, it is necessary to select the optimal result from the first and second similarities and determine the circuit board positioning result based on the optimal result. In this case, determining the circuit board positioning result on the conveyor belt based on the matching results includes: selecting the maximum similarity from the first and second similarities, and using the calibrated board image corresponding to the maximum similarity as the circuit board positioning result on the conveyor belt.

[0113] For example, a greater similarity indicates that the corrected board image, in its current rotational state, is closer to the reference board image, meaning that the placement of the circuit boards in the corrected board image is identical to that of the reference board. Therefore, in this embodiment, the maximum similarity is selected between the first and second similarities. It is understood that for a rectangular corrected board image, the maximum similarity is selected between one first similarity and one second similarity. For a square corrected board image, the maximum similarity is selected between one first similarity and three second similarities.

[0114] Furthermore, the board calibration image at the time of maximum similarity is used as the positioning result of the circuit board. That is, the rotation state corresponding to the maximum similarity is determined, and the board calibration image at this rotation state is used as the positioning result. Then, the rotation state and the board calibration image are used as the selected image. At this time, it can be determined that the circuit board is accurately located from the target image, and the located circuit board can be directly detected for subsequent defects.

[0115] The above-mentioned technical means solves the technical problem of low positioning accuracy when using feature points to locate complex circuit boards in the related art. In the above-mentioned positioning process, only the board mask of the circuit board needs to be identified to extract the corresponding circuit board from the target image. That is, even if the background of the circuit board is cluttered or there are more wires than necessary, the identification of the board mask will not be affected. This improves the accuracy of subsequent matching with the reference circuit image. In addition, the above-mentioned process does not require feature extraction and matching of repeated textures on the circuit board, further improving the accuracy of subsequent matching with the reference circuit image and enhancing the robustness of subsequent detection. Furthermore, by using the convex hull of the board outline to determine the bounding rectangle, we can accurately locate skewed circuit boards in the target image, effectively obtaining the skewed bounding rectangle. Furthermore, even if the target image doesn't contain a complete circuit board, the true bounding rectangle can still be determined, ensuring accurate subsequent circuit board positioning. Furthermore, the board image extracted using the bounding rectangle has a smaller background portion, and correction and matching based on the board image's length and width are performed, making the correction and matching process relatively simple and reducing the template matching error rate.

[0116] Figure 6 This is a schematic diagram of a circuit board positioning device provided in an embodiment of the present application, with reference to Figure 6 The circuit board positioning device includes an image acquisition module 301 , a mask determination module 302 , an area determination module 303 and a positioning determination module 304 .

[0117] Among them, the image acquisition module is used to obtain the target image obtained after photographing the conveyor belt, which is used to transport circuit boards; the mask determination module is used to segment the board mask corresponding to the circuit board from the target image, and each circuit board corresponds to a board mask; the area determination module is used to determine the circumscribed rectangular area of ​​the circuit board in the target image based on the board mask; and the positioning determination module is used to determine the positioning result of the circuit board based on the circumscribed rectangular area.

[0118] Based on the above embodiment, the area determination module includes: a contour determination unit, which is used to obtain the board contour of the circuit board according to the board mask; and a rectangular area determination unit, which is used to determine the circumscribed rectangular area of ​​the circuit board in the target image according to the convex hull of the board contour.

[0119] On the basis of the above embodiment, the rectangular area determination unit includes: a convex hull determination subunit, which is used to obtain the convex hull of the board outline; an alternative rectangle determination subunit, which is used to determine the alternative circumscribed rectangle corresponding to each side in the convex hull, where the alternative circumscribed rectangle is the minimum circumscribed rectangle containing the board outline, and one side in the alternative circumscribed rectangle is collinear with the corresponding side in the convex hull; an area calculation subunit, which is used to calculate the area of ​​each alternative circumscribed rectangle; a rectangle selection subunit, which is used to select the alternative circumscribed rectangle with the smallest area; and a circumscribed rectangle area determination subunit, which is used to use the display area of ​​the alternative circumscribed rectangle with the smallest area in the target image as the circumscribed rectangle area of ​​the circuit board in the target image.

[0120] On the basis of the above embodiment, the alternative rectangle determination subunit includes: a first angle determination grandchild unit, used to determine the first angle corresponding to each side in the convex hull, the first angle being the angle between the corresponding side and the set coordinate axis, and the set coordinate axis being the coordinate axis used in the coordinate system where the board outline is located; a convex hull rotation grandchild unit, used to reversely rotate each point in the convex hull according to the first angle, so that the side corresponding to the first angle is parallel to the set coordinate axis after the reverse rotation; a minimum enclosing rectangle determination grandchild unit, used to determine the minimum enclosing rectangle of the convex hull after the reverse rotation, and the side corresponding to the first angle is collinear with one of the sides in the minimum enclosing rectangle after the reverse rotation; a first rectangle rotation grandchild unit, used to positively rotate the minimum enclosing rectangle according to the first angle, and use the minimum enclosing rectangle after the positive rotation as the alternative enclosing rectangle for the side corresponding to the first angle.

[0121] Based on the above embodiment, the area calculation subunit is further used, when calculating the area of ​​each of the alternative circumscribed rectangles: if a partial area of ​​the alternative circumscribed rectangle exceeds the boundary of the target image, then the area of ​​the intersection area of ​​the alternative circumscribed rectangle and the target image is used as the area of ​​the alternative circumscribed rectangle.

[0122] Based on the above embodiment, the positioning determination module includes: a board image extraction unit, which is used to extract the board image of the circuit board in the target image based on the circumscribed rectangular area; an image matching unit, which is used to match the board image with the reference board image and determine the positioning result of the circuit board based on the matching result.

[0123] Based on the above embodiment, the image matching unit includes: a reference image acquisition subunit, which is used to acquire a reference board image of the circuit board; a correction subunit, which is used to correct the board image according to the reference board image to obtain a board correction image; a matching subunit, which is used to match the board correction image with the reference board image; and a positioning result determination subunit, which is used to determine the positioning result of the circuit board based on the matching result.

[0124] Based on the above embodiment, the correction sub-unit includes: a homography matrix determination sub-unit, which is used to determine the homography matrix between the reference plate image and the circumscribed rectangular area when the aspect ratio of the reference plate image and the aspect ratio of the circumscribed rectangular area satisfy a similarity relationship; and a processing sub-unit, which is used to process the plate image using the homography matrix to obtain a plate-corrected image.

[0125] On the basis of the above embodiment, it also includes: a second rectangle rotation module, which is used to rotate the circumscribed rectangular area according to a second angle if the aspect ratio of the reference board image and the aspect ratio of the circumscribed rectangular area do not satisfy a similar relationship, so that the aspect ratio of the rotated circumscribed rectangular area satisfies a similar relationship with the aspect ratio of the reference board image.

[0126] Based on the above embodiment, the matching subunit includes: a first similarity calculation subunit, which is used to calculate the first similarity between the board correction image and the reference board image; a first image rotation subunit, which is used to rotate the board correction image three times and 90° each time if the board correction image is a square image, and calculate the second similarity between the board correction image and the reference board image after each rotation; a second image rotation subunit, which is used to rotate the board correction image 180° if the board correction image is a positive direction image, and calculate the second similarity between the board correction image and the reference board image after the rotation.

[0127] Based on the above embodiment, the positioning result determination subunit is specifically configured to: select the maximum similarity from the first similarity and the second similarity, and use the board correction image corresponding to the maximum similarity as the positioning result of the circuit board.

[0128] Based on the above embodiment, it further includes: a pixel point acquisition module, which is used to obtain the number of pixel points with zero pixel value in the board card correction image before matching the board card correction image with the reference board card image, and the pixel points with zero pixel value are the pixels representing the background; and a number comparison module, which is used to perform the operation of matching the board card correction image with the reference board card image when the number of pixel points is less than the number threshold.

[0129] Based on the above embodiment, the method further includes: a scaling module, which is used to scale the board correction image and the reference board image to a target size before matching the board correction image with the reference board image.

[0130] The circuit board positioning device provided above can be used to execute the circuit board positioning method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0131] It is worth noting that in the embodiment of the above-mentioned circuit board positioning device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0132] Figure 7 This is a schematic diagram of the structure of a circuit board positioning device provided in an embodiment of the present application. Figure 7 As shown, the circuit board positioning device includes a processor 40, a memory 41, an input device 42, and an output device 43; the number of processors 40 in the circuit board positioning device can be one or more. Figure 7 A processor 40 is used as an example. The processor 40, memory 41, input device 42, and output device 43 in the circuit board positioning device can be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0133] Memory 41, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the circuit board positioning method in the embodiments of the present application (e.g., the image acquisition module 301, mask determination module 302, region determination module 303, and positioning determination module 304 in the circuit board positioning device). Processor 40 executes the software programs, instructions, and modules stored in memory 41 to perform various functional applications and data processing of the circuit board positioning device, thereby implementing the aforementioned circuit board positioning method.

[0134] The memory 41 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the circuit board positioning device. Furthermore, the memory 41 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 41 may further include memory remotely located relative to the processor 40, and such remote memory may be connected to the circuit board positioning device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The input device 42 can be used to receive input digital or character information and generate key signal input related to user settings and function control of the circuit board positioning device. The output device 43 can include a display device such as a display screen. The circuit board positioning device can also include a communication device that can communicate with other devices, such as a camera, to obtain a target image.

[0136] The above-mentioned circuit board positioning device includes a circuit board positioning device, which can be used to execute any circuit board positioning method and has corresponding functions and beneficial effects.

[0137] In addition, an embodiment of the present application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform relevant operations in the circuit board positioning method provided in any embodiment of the present application, and have corresponding functions and beneficial effects.

[0138] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products.

[0139] Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0140] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-permanent storage in a computer-readable medium, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0141] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0142] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0143] Note that the above are only preferred embodiments of the present application and the technical principles employed. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the appended claims.

Claims

1. A circuit board positioning method, characterized in that: include: Acquiring a target image obtained by photographing a conveyor belt, wherein the conveyor belt is used to transport circuit boards; Segmenting a board mask corresponding to the circuit board from the target image, where each circuit board corresponds to one board mask; Determine a circumscribed rectangular area of ​​the circuit board in the target image according to the board mask; Determining a positioning result of the circuit board according to the circumscribed rectangular area; The determining of the positioning result of the circuit board according to the circumscribed rectangular area includes: Extracting a board image of the circuit board from the target image according to the circumscribed rectangular area, and if the board image is a tilted image, rotating the board image into a non-tilted board image and using the non-tilted board image as the final extracted board image; The board image is matched with a reference board image, and a positioning result of the circuit board is determined according to the matching result.

2. The circuit board positioning method according to claim 1, wherein: The determining, according to the board mask, a circumscribed rectangular area of ​​the circuit board in the target image comprises: Obtaining a board outline of the circuit board according to the board mask; A circumscribed rectangular area of ​​the circuit board in the target image is determined according to the convex hull of the board outline.

3. The circuit board positioning method according to claim 2, wherein: Determining the circumscribed rectangular area of ​​the circuit board in the target image according to the convex hull of the board outline includes: Obtaining the convex hull of the board outline; Determine a candidate circumscribed rectangle corresponding to each side of the convex hull, where the candidate circumscribed rectangle is a minimum circumscribed rectangle containing the outline of the board, and one side of the candidate circumscribed rectangle is collinear with the corresponding side of the convex hull; Calculating the area of ​​each candidate circumscribed rectangle; Select the alternative bounding rectangle with the smallest area; The display area of ​​the candidate circumscribed rectangle with the smallest area in the target image is used as the circumscribed rectangular area of ​​the circuit board in the target image.

4. The circuit board positioning method according to claim 3, characterized in that: Determining the candidate circumscribed rectangle corresponding to each edge in the convex hull includes: Determine a first angle corresponding to each edge in the convex hull, where the first angle is the angle between the corresponding edge and a set coordinate axis, where the set coordinate axis is a coordinate axis used in a coordinate system where the board outline is located; Reversely rotate each point in the convex hull according to the first angle so that the side corresponding to the first angle is parallel to the set coordinate axis after the reverse rotation; Determine a minimum circumscribed rectangle containing the outline of the board after reverse rotation, wherein the side corresponding to the first angle is collinear with a side of the minimum circumscribed rectangle after reverse rotation; The minimum circumscribed rectangle is positively rotated according to the first angle, and the positively rotated minimum circumscribed rectangle is used as a candidate circumscribed rectangle of a side corresponding to the first angle.

5. The circuit board positioning method according to claim 3, wherein: The calculation of the area of ​​each candidate circumscribed rectangle further includes: If a portion of the candidate circumscribed rectangle exceeds the boundary of the target image, the area of ​​the intersection of the candidate circumscribed rectangle and the target image is used as the area of ​​the candidate circumscribed rectangle.

6. The circuit board positioning method according to claim 1, wherein: The matching of the board image with the reference board image comprises: Acquiring a reference board image of the circuit board; Correcting the board image according to the reference board image to obtain a board-corrected image; The board correction image is matched with the reference board image.

7. The circuit board positioning method according to claim 6, wherein: Correcting the board image according to the reference board image to obtain a board-corrected image includes: When the aspect ratio of the reference plate image and the aspect ratio of the circumscribed rectangular area satisfy a similarity relationship, determining a homography matrix between the reference plate image and the circumscribed rectangular area; The board image is processed using the homography matrix to obtain a board-corrected image.

8. The circuit board positioning method according to claim 7, characterized in that: Also includes: If the aspect ratio of the reference board image and the aspect ratio of the circumscribed rectangular area do not satisfy a similar relationship, the circumscribed rectangular area is rotated according to a second angle so that the aspect ratio of the rotated circumscribed rectangular area and the aspect ratio of the reference board image satisfy a similar relationship.

9. The circuit board positioning method according to claim 6, wherein: The matching of the board correction image with the reference board image comprises: Calculating a first similarity between the board correction image and the reference board image; If the board-corrected image is a square image, rotating the board-corrected image three times by 90° each time, and calculating a second similarity between the board-corrected image and the reference board image after each rotation; If the board-corrected image is a long-direction image, the board-corrected image is rotated 180°, and after the rotation, a second similarity between the board-corrected image and the reference board image is calculated.

10. The circuit board positioning method according to claim 9, wherein: Determining the positioning result of the circuit board according to the matching result includes: A maximum similarity is selected between the first similarity and the second similarity, and the board correction image corresponding to the maximum similarity is used as the positioning result of the circuit board.

11. The circuit board positioning method according to claim 6, wherein: Before matching the board correction image with the reference board image, the method further includes: Obtaining the number of pixels with zero pixel values ​​in the board correction image, where the pixels with zero pixel values ​​are pixels representing the background; When the number of the pixel points is less than the number threshold, an operation of matching the plate correction image with the reference plate image is performed.

12. The circuit board positioning method according to claim 7, wherein: Before matching the board correction image with the reference board image, the method further includes: The board correction image and the reference board image are scaled to target sizes respectively.

13. A circuit board positioning device, characterized in that: include: An image acquisition module, configured to acquire a target image obtained by photographing a conveyor belt used to transport circuit boards; a mask determination module, configured to segment the board mask corresponding to the circuit board from the target image, wherein each circuit board corresponds to one board mask; an area determination module, configured to determine a circumscribed rectangular area of ​​the circuit board in the target image according to the board mask; A positioning determination module, configured to determine a positioning result of the circuit board according to the circumscribed rectangular area; The positioning determination module includes: a board image extraction unit, configured to extract a board image of the circuit board in the target image according to a circumscribed rectangular area, and if the board image is an inclined image, rotate the board image into a non-inclined board image and use the non-inclined board image as the final extracted board image; The image matching unit is used to match the board image with the reference board image and determine the positioning result of the circuit board according to the matching result.

14. A circuit board positioning device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the circuit board positioning method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the circuit board positioning method according to any one of claims 1 to 12 is implemented.

Images