Calibration pattern recognition method, calibration method, device and electronic equipment

Abstract

The present disclosure provides a calibration pattern recognition method, a calibration method, a device and an electronic device. The calibration pattern recognition method comprises: acquiring a calibration image, the calibration image being obtained by a camera shooting a calibration board, the calibration board comprising a plurality of first marks and second marks arranged in an array, the first marks being arranged at the middle position of the calibration board, and the plurality of first marks being arranged in a non-rotationally symmetric manner; determining a first matching relationship between a first pattern in the calibration image and the first marks according to the arrangement manner of the first marks, the contour of the first pattern conforming to a first preset contour; determining a second pattern in a suspected pattern according to the positional relationship between the second marks and the first marks in the calibration board and the first matching relationship, the contour of the suspected pattern conforming to a second preset contour, and the second pattern being a suspected pattern in the calibration image conforming to the arrangement manner of the second marks on the calibration board; and determining the first pattern and the second pattern as calibration patterns of the calibration board, so that the calibration patterns can be accurately determined.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a calibration pattern recognition method, calibration method, apparatus and electronic device. Background Technology

[0002] Camera calibration is a crucial technique in image processing. In image measurement and machine vision applications, to determine the spatial position of a point on the surface of a spatial object and its corresponding point in the image, and to reconstruct the spatial object in the objective world from the image, it is necessary to establish camera parameters for camera imaging. The process of determining these camera parameters is called camera calibration.

[0003] Currently, when using calibration boards for camera calibration, there is a problem of low robustness in calibration pattern recognition. Summary of the Invention

[0004] This disclosure provides a calibration pattern recognition method, calibration method, apparatus, and electronic device to address the current problem of low robustness in calibration pattern recognition.

[0005] A first aspect of this disclosure provides a calibration pattern recognition method, comprising: acquiring a calibration image, wherein the calibration image is obtained by a camera capturing a calibration board, the calibration board including a plurality of first marks and second marks arranged in an array, the first marks being disposed at the center of the calibration board, and the plurality of first marks being arranged in a non-rotationally symmetrical manner; determining a first matching relationship between a first pattern in the calibration image and the first marks based on the arrangement of the first marks, wherein the outline of the first pattern conforms to a first preset outline; determining a second pattern among suspected patterns based on the positional relationship between the second marks and the first marks in the calibration board, and the first matching relationship, wherein the outline of the suspected pattern conforms to a second preset outline, and the second pattern being a suspected pattern in the calibration image that conforms to the arrangement of the second marks on the calibration board; and determining the first pattern and the second pattern as calibration patterns of the calibration board.

[0006] A second aspect of this disclosure provides a calibration method, including: a calibration pattern recognition method of the first aspect, which, after determining that a first pattern and a second pattern are calibration patterns of a calibration board, further includes: calibrating camera parameters using marks and calibration patterns on the calibration board.

[0007] A third aspect of this disclosure provides a calibration pattern recognition device for performing the calibration pattern recognition method of the first aspect, including:

[0008] The acquisition module is used to acquire calibration images, which are obtained by the camera taking pictures of the calibration board. The calibration board includes multiple first marks and second marks arranged in an array. The first marks are set in the middle of the calibration board, and the multiple first marks are arranged in a non-rotational symmetric manner.

[0009] The first determining module is used to determine the first matching relationship between the first pattern in the calibration image and the first mark according to the arrangement of the first mark, wherein the outline of the first pattern conforms to the first preset outline;

[0010] The second determining module is used to determine the second pattern in the suspected pattern according to the positional relationship between the second mark and the first mark in the calibration board and the first matching relationship. The outline of the suspected pattern conforms to the second preset outline. The second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second mark on the calibration board.

[0011] The third determining module is used to determine that the first pattern and the second pattern are the calibration patterns of the calibration plate.

[0012] A fourth aspect of this disclosure provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the calibration pattern recognition method of the first aspect.

[0013] The fifth aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the calibration pattern recognition method of the first aspect.

[0014] A sixth aspect of this disclosure provides a computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the calibration pattern recognition method of the first aspect.

[0015] This embodiment of the disclosure is applied in a camera parameter calibration scenario. A calibration image is acquired by the camera capturing a calibration board. The calibration board includes multiple first and second markers arranged in an array. The first markers are positioned in the center of the calibration board, and the multiple first markers are arranged in a non-rotationally symmetrical manner. Based on the arrangement of the first markers, a first matching relationship is determined between a first pattern in the calibration image and the first markers. The outline of the first pattern conforms to a first preset outline. Based on the positional relationship between the second and first markers in the calibration board, and the first matching relationship, a second pattern is determined from among the suspected patterns. The outline of the suspected pattern conforms to a second preset outline. The second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second markers on the calibration board. By determining the first and second patterns as the calibration patterns of the calibration board, the first pattern can be determined first, and the second pattern can be determined based on the first matching relationship between the first pattern and the first markers, thereby accurately determining the calibration patterns. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0017] Figure 1 An application scenario diagram of a calibration pattern recognition method provided for an exemplary embodiment of this disclosure;

[0018] Figure 2 A flowchart illustrating the steps of a calibration pattern recognition method provided for an exemplary embodiment of this disclosure;

[0019] Figure 3 A flowchart illustrating the steps of another calibration pattern recognition method provided as an exemplary embodiment of this disclosure;

[0020] Figure 4 A schematic diagram of a first pattern extraction process provided for an exemplary embodiment of this disclosure;

[0021] Figure 5 A schematic diagram of a second pattern extraction process provided for an exemplary embodiment of this disclosure;

[0022] Figure 6 A flowchart illustrating steps for determining a second image, provided as an exemplary embodiment of this disclosure;

[0023] Figure 7 Structural block diagram of a calibration pattern recognition device provided for exemplary embodiments of this disclosure;

[0024] Figure 8 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0026] During the camera parameter calibration process, a calibration image is obtained by photographing the calibration board. The calibration board has multiple marks, and the calibration image contains the corresponding calibration patterns for each mark. During calibration, it is necessary to identify the calibration pattern of each mark and establish a one-to-one matching relationship between the mark and the mark object in order to calibrate the camera parameters. However, the calibration image may contain patterns in the background of the calibration board. The presentation of these patterns in the calibration image can affect the identification of the calibration patterns, making the obtained calibration patterns inaccurate, which in turn affects the accuracy of the camera parameter calibration.

[0027] Based on the above problems, this embodiment of the present disclosure obtains a calibration image, which is obtained by a camera capturing a calibration board. The calibration board includes multiple first and second marks arranged in an array. The first marks are located in the middle of the calibration board, and the multiple first marks are arranged in a non-rotationally symmetrical manner. According to the arrangement of the first marks, a first matching relationship between a first pattern and the first marks in the calibration image is determined, and the outline of the first pattern conforms to a first preset outline. According to the positional relationship between the second and first marks in the calibration board and the first matching relationship, a second pattern is determined among the suspected patterns. The outline of the suspected pattern conforms to the second preset outline, and the second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second marks on the calibration board. By determining the first pattern and the second pattern as the calibration pattern of the calibration board, the first pattern can be determined first, and the second pattern can be determined according to the first matching relationship between the first pattern and the first marks, thereby accurately determining the calibration pattern.

[0028] In this embodiment, the calibration pattern recognition method can be a server-based calibration pattern recognition method. Furthermore, the server executing the calibration pattern recognition method can be a conventional server or server array, or a cloud server; no limitation is imposed here.

[0029] Furthermore, one application scenario of this disclosure embodiment is as follows: Figure 1 , Figure 1 The system includes a calibration plate 11 and a calibration image 12 obtained by a camera capturing images of the calibration plate 11. The calibration plate 11 is placed on an object 13. The calibration plate 11 has multiple first markers (r0 to r7) and second markers (S1 to S22). The object 13 has multiple patterns B1. The calibration image 12 includes first patterns (T0 to T7) corresponding to the first markers (r0 to r7), second patterns (C1 to C22) corresponding to the second markers, and a background pattern B2 corresponding to the patterns B1. In this disclosure, it is necessary to identify the first and second patterns in the calibration image 12, and to determine the matching relationship between each first pattern and the first marker, as well as the matching relationship between the second pattern and the second marker, to facilitate the calibration of camera parameters.

[0030] in, Figure 1 This is merely one exemplary application scenario, and the embodiments disclosed herein can be applied to any calibration of camera parameters. The embodiments disclosed herein do not limit the specific application scenarios.

[0031] Figure 2 A flowchart illustrating the steps of a calibration pattern recognition method provided for an exemplary embodiment of this disclosure. Specifically, it includes the following steps:

[0032] S201, Obtain the calibration image.

[0033] The calibration image is obtained by the camera taking pictures of the calibration board. The calibration board includes multiple first and second marks arranged in an array. The first marks are set in the middle of the calibration board, and the multiple first marks are arranged in a non-rotational symmetric manner.

[0034] Reference Figure 1 The calibration plate 11 includes multiple marks arranged in an array, and each mark includes a first mark and a second mark. Furthermore, multiple first marks are positioned in the center of the calibration plate, and a second mark surrounds the multiple first marks.

[0035] Furthermore, in this disclosure, the first mark and the second mark have different shapes, such as the first mark being a ring and the second mark being a circle.

[0036] Furthermore, the calibration image is obtained by photographing the calibration board with a camera. Due to the influence of the camera's shooting angle and camera parameters, the shapes of each pattern in the calibration image are distorted relative to the shapes of the marks on the calibration board. This disclosure requires determining the pattern that matches each mark on the calibration board in the calibration image, so as to calibrate the camera parameters using the determined target pattern.

[0037] S202, Based on the arrangement of the first mark, determine the first matching relationship between the first pattern in the calibration image and the first mark.

[0038] The arrangement of the first and second marks on the calibration plate is preset and is a known arrangement. Specifically, the arrangement of the first marks includes the number of first marks and the position of each first mark on the calibration plate. For example, refer to... Figure 1 It includes 8 first markers, where the first marker numbered r0 is located in the third row and third column, the first marker numbered r1 is located in the second row and third column, the first marker numbered r2 is located in the second row and fourth column, the first marker numbered r3 is located in the third row and fourth column, the first marker numbered r4 is located in the fourth row and fourth column, the first marker numbered r5 is located in the fourth row and third column, the first marker numbered r6 is located in the fourth row and second column, and the first marker numbered r7 is located in the third row and second column.

[0039] In this disclosure, the outline of the first pattern conforms to a first preset outline. The first preset outline is pre-set and is preset according to the shape of the first mark. If the shape of the first mark is a ring, then the first preset outline can be an elliptical ring or a ring, and the shape of the first pattern is an elliptical ring or a ring.

[0040] Furthermore, the first matching relationship is determined to be a one-to-one correspondence between the first pattern and the first symbol. (Refer to...) Figure 1 The first symbol r0 matches the first pattern T0, the first symbol r1 matches the first pattern T1, the first symbol r2 matches the first pattern T2, the first symbol r3 matches the first pattern T3, the first symbol r4 matches the first pattern T4, the first symbol r5 matches the first pattern T5, the first symbol r6 matches the first pattern T6, and the first symbol r7 matches the first pattern T7.

[0041] S203, based on the positional relationship between the second mark and the first mark in the calibration plate, and the first matching relationship, determine the second pattern among the suspected patterns.

[0042] The suspected pattern's outline conforms to a second preset outline, and the second pattern is a suspected pattern in the calibration image that matches the arrangement of the second marks on the calibration board. For example, if the second mark is circular, then the second preset outline is an ellipse or a circle. (Refer to...) Figure 1 In the calibrated image, a suspected pattern is extracted by the second preset contour. The suspected pattern includes the second pattern (C1 to C22) and the background pattern B2.

[0043] Furthermore, the arrangement of the second markers includes the number of second markers and the position of each second marker. For example, refer to... Figure 1 The calibration plate 11 has 22 second marks, and the position of each second mark is the row and column in which it is located.

[0044] Optionally, a smaller number of first flags and a larger number of second flags can be set, for example, in Figure 1 In this system, there are 8 first symbols, which are arranged asymmetrically. The number of second symbols is 22, which is greater than the number of first symbols. This allows for a faster and more efficient determination of the first matching relationship between the first pattern and the first symbols, and thus a more efficient determination of the second pattern.

[0045] In this disclosure, since the second mark surrounds the first mark and is positioned at the edge of the calibration plate, the second pattern will be adjacent to the background pattern in the obtained calibration image. Therefore, the suspected pattern includes both the second pattern and the background pattern, and it is necessary to identify the second pattern from the suspected patterns.

[0046] Furthermore, after determining the first matching relationship between the first mark and the first pattern, the second pattern can be determined from the suspected pattern based on the positional relationship between the second mark and the first mark, and the positional relationship between the suspected pattern and the first pattern. For example, refer to... Figure 1 The second marker S8 is positioned symmetrically to the first marker r4 with the first marker r0 as its center of symmetry. If the first marker r0 and the first pattern T0 are not matched, and the first marker r4 and the first pattern T4 are matched, then the suspected pattern C8 at the symmetrical position of the first pattern T4 with the first pattern T0 as its center of symmetry is determined to be the second pattern matched by the second marker S8. Alternatively, the second marker S3 is located on the extension of the line connecting the first marker r5, the first marker r0, and the first marker r1. If the first marker r5 matches the first pattern T5, the first marker r0 matches the first pattern T0, and the first marker r1 matches the first pattern T1, then the suspected pattern C3 on the extension of the line connecting the first patterns T5, T0, and T1 is the second pattern matched by the second marker S3. The extension line has a direction of extension; for example, on the calibration board, it points from the first marker r5 to the first marker r1, and on the calibration image, it points from the first pattern T5 to the first pattern T1. Using a similar method, multiple second patterns can be determined, and the matching relationship between the second patterns and the second markers can also be established.

[0047] S204, determine the first pattern and the second pattern as the calibration patterns of the calibration plate.

[0048] The determined calibration patterns include a first pattern and a second pattern, specifically the position information of the first pattern in the calibration image and the position information of the second pattern in the calibration image.

[0049] This embodiment of the present disclosure acquires a calibration image, which is obtained by a camera capturing a calibration board. The calibration board includes multiple first and second marks arranged in an array. The first marks are located in the middle of the calibration board, and the multiple first marks are arranged in a non-rotationally symmetrical manner. Based on the arrangement of the first marks, a first matching relationship between a first pattern in the calibration image and the first marks is determined, and the outline of the first pattern conforms to a first preset outline. Based on the positional relationship between the second and first marks in the calibration board, and the first matching relationship, a second pattern is determined from the suspected patterns. The outline of the suspected pattern conforms to the second preset outline, and the second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second marks on the calibration board. By determining the first pattern and the second pattern as the calibration pattern of the calibration board, the first pattern can be determined first, and the second pattern can be determined based on the first matching relationship between the first pattern and the first marks, thereby accurately determining the calibration pattern.

[0050] Figure 3 A flowchart illustrating another calibration pattern recognition method provided for an exemplary embodiment of this disclosure specifically includes the following steps:

[0051] S301, acquire calibration image.

[0052] The specific implementation process of this step is described in S201 and will not be repeated here.

[0053] S302, the calibration image is scaled according to a preset ratio to obtain a calibration image of a preset size.

[0054] The original calibration image captured by the camera is relatively large. Performing subsequent calculations on the original calibration image would increase the computational load on the computing device and reduce computational efficiency. Therefore, this disclosure scales the original calibration image to obtain a scaled calibration image. The scaled calibration image is smaller, which improves computational efficiency.

[0055] The preset ratio and preset size are preset. Specifically, the preset size is determined based on the computing resources of the computing device (such as a server) currently executing the present invention, so that the target image of the preset size is suitable for the computing resources. For example, if there are more computing resources, the preset size is larger, and if there are fewer computing resources, the preset size is smaller.

[0056] Furthermore, this disclosure also allows for subsequent calculations to be performed on the original calibration image captured by the camera, without limitation.

[0057] S303, extract the first pattern from the calibration image.

[0058] In this disclosure, the first pattern can be extracted from the calibration image before scaling or from the calibration image after scaling.

[0059] Furthermore, image segmentation and contour detection techniques are employed to extract a first pattern that conforms to a first preset contour. Specifically, the first pattern is extracted, and its coordinates in the target image are determined.

[0060] For example, refer to Figure 4 shrink Figure 1 The calibration image 12 is used to obtain a reduced calibration image 41. The first pattern, such as X0 to X7, is extracted from the calibration image 41. The image containing the first pattern is the intermediate image 42, wherein the size of the intermediate image 42 is the same as the size of the calibration image 41.

[0061] S304, Based on the arrangement of the first mark, establish a first matching relationship between the first pattern and the first mark.

[0062] The arrangement of the first markers is as follows: multiple first markers and at least one second marker array are arranged in a first square array, with at least one second marker positioned at a corner of the first square array. For example, refer to... Figure 1There are 8 first markers. The 8 first markers and 1 second marker form a first square array of 3 rows and 3 columns. The second marker is placed at the corner position of the first square array.

[0063] Furthermore, the arrangement of the first mark described above is a preferred embodiment. In this disclosure, the first mark may be arranged in other asymmetrical arrangements, which are not limited here.

[0064] S304 includes: among a plurality of first patterns, determining the first pattern located at the center position as the center pattern, determining the first pattern that is symmetrical with the center pattern as the center of symmetry and is collinear with the center pattern as the edge pattern; connecting the plurality of edge patterns to obtain a convex hull; determining the first pattern outside the convex hull as the target pattern; and determining the first matching relationship between the first pattern and the first mark according to the positional relationship between the target pattern and other first patterns and the arrangement of the first mark.

[0065] Reference Figure 4 The intermediate image 42 includes multiple extracted first patterns, with the first pattern X0 located at the center. Therefore, first pattern X0 is determined as the center pattern. Then, taking the center pattern X0 as its center of symmetry, the edge patterns collinear with and possessing symmetry with the center pattern X0 are first patterns X1, X2, X3, X5, X6, and X7. Connecting these edge patterns (X1, X2, X3, X5, X6, and X7) yields a convex hull; the first pattern X4 located outside the convex hull is determined as the target pattern. Furthermore, the unique center pattern X0 and... Figure 1 The first mark r0 on calibration plate 11 matches, and the unique target pattern X4 matches. Figure 1 The first mark r4 on the calibration plate 11 is matched. Starting from the target pattern X4 and the first mark r4, in a clockwise or counterclockwise direction, it can be determined that the first pattern X5 matches the first mark r5, the first pattern X6 matches the first mark r6, the first pattern X7 matches the first mark r7, the first pattern X1 matches the first mark r1, the first pattern X2 matches the first mark r2, and the first pattern X3 matches the first mark r3.

[0066] S305, based on the first matching relationship, the coordinates of the first mark in the calibration plate, and the coordinates of the first pattern in the calibration image, determine the homography matrix of the first mark mapped to the first pattern.

[0067] The positional relationship includes the coordinates of the first and second marks on the calibration plate.

[0068] Specifically, the homography matrix is ​​determined as matrix J, and the coordinates of the first mark r0 to the first mark r7 in the calibration plate are Y0 to Y7, where coordinates Y0 to Y7 can be the coordinates of the center of the first mark ring. The coordinates of the first pattern X0 to the first pattern X7 in the calibration image are Z0 to Z7, where coordinates Z0 to Z7 can be the coordinates of the center of the first pattern. Then, the homography matrix J can be determined according to the following formula (1).

[0069]

[0070] In formula (1), the homography matrix J can be a 3*3 matrix, S is any coefficient, and the coordinates of the first marker are expressed in homogeneous coordinates. The coordinates of the first pattern are represented using homogeneous coordinates. Where i ranges from 0 to 7, X i Y represents the x-coordinate of the i-th first marker on the calibration plate. i Let x represent the ordinate of the i-th first marker on the calibration plate. i Let y represent the x-coordinate of the i-th first marker in the calibration image. i Let J represent the ordinate of the i-th second marker in the calibration image. Furthermore, the homography matrix J can be estimated using a direct linear method, or optimized using a nonlinear method based on the linear solution; neither method is limited here.

[0071] S306, extract multiple suspected patterns from the calibration image according to the second preset contour.

[0072] If the second preset contour is elliptical, then the suspected patterns extracted from the calibration image using image segmentation and contour detection techniques will all be elliptical patterns.

[0073] Reference Figure 5 In the intermediate image 51, there are multiple suspected patterns (suspected patterns other than the first pattern).

[0074] S307, based on the homography matrix, the coordinates of the second mark in the calibration plate, and the coordinates of the suspected pattern in the calibration image, determine the suspected pattern that matches the second mark as the second pattern.

[0075] Among them, the determined second pattern reference Figure 5 The ellipse in the intermediate image 52. In an optional embodiment, the homography matrix is ​​multiplied by the coordinates of the second mark in the calibration plate to obtain the mapped coordinates of the second mark in the calibration image, and the suspected pattern corresponding to the coordinates whose mapped coordinates are less than a distance threshold is determined as the second pattern.

[0076] Specifically, refer to Figure 6 Step S307 includes repeatedly performing the following steps:

[0077] S601, Determine the mapping coordinates of the second mark based on the homography matrix and the coordinates of the second mark in the calibration plate.

[0078] The mapping coordinates of the second marker are determined using the following formula (2):

[0079]

[0080] In formula (2), E1 to En are the coordinates of the first to the nth second markers on the calibration plate, specifically the coordinates of the center of the second marker circle, where n is a positive integer. F1 to Fn are the mapped coordinates of the first to the nth second markers, corresponding one-to-one with E1 to En. For example, refer to... Figure 1 n is 22, E1 can be the coordinates of the second mark S1 on the calibration plate 11, and F1 is the mapped coordinates of the second mark S1 in the calibration image.

[0081] In formula (2), Where j ranges from 1 to n. X j Y represents the x-coordinate of the j-th second marker on the calibration plate. j Let x represent the ordinate of the j-th second marker on the calibration plate. j Let y represent the x-coordinate of the j-th second label mapped in the calibration image. j This represents the ordinate of the j-th second marker mapped in the calibration image.

[0082] S602, in the coordinates of the suspected pattern in the calibration image, determine the target coordinates that are less than the distance threshold from the mapped coordinates.

[0083] The distance threshold is preset. For example, if the distance threshold is 0.1, and the coordinates of a suspected pattern are (1.2, 1.5), the distance between the suspected pattern and the mapped coordinates (1.2, 1.48) is 0.02, which is less than the distance threshold, then the coordinates (1.2, 1.5) can be determined as the target coordinates, and the suspected pattern corresponding to these coordinates is the second pattern.

[0084] S603, the suspected pattern corresponding to the target coordinates is determined to be the second pattern.

[0085] This method can be used to determine the second pattern and the matching relationship between the second pattern and the second mark.

[0086] S604, determine whether the preset conditions are met.

[0087] The preset conditions include: the number of iterations is less than the preset number of iterations or a new second pattern is determined.

[0088] If yes, execute 605 and continue executing S601; otherwise, terminate the process.

[0089] S605, establish a second matching relationship between the second pattern and the second mark, and update the homography matrix according to the second matching relationship.

[0090] If the distance between the coordinates of one of the suspected patterns in the calibration image and the mapped coordinates of one of the second marks is less than a threshold, then the suspected pattern can be determined to be the second pattern and has a second matching relationship with the second mark.

[0091] Furthermore, updating the homography matrix involves using the second matching relationship, the coordinates of the second pattern in the calibration image, and the coordinates of the second mark in the calibration plate. Specifically, the new homography matrix is ​​determined by using the first matching relationship between the first pattern and the first mark, the second matching relationship between the detected second pattern and the second mark, and the second matching relationship between the newly detected second pattern and the second mark. For example, referring to formula (3), in formula (3), P1 and P2 are the coordinates of the second marks on the calibration plate that match the two newly determined second patterns, and Q1 and Q2 are the coordinates of the two newly determined second patterns in the calibration image. Specifically, the second mark corresponding to P1 and the second pattern corresponding to Q1 have a matching relationship. The second mark corresponding to Q1 and the second pattern corresponding to Q2 also have a matching relationship.

[0092]

[0093] Furthermore, the updated homography matrix is ​​J1, which can be a 3*3 matrix. Then, S601 to S605 are executed in a loop. For example, in the next round, the homography matrix J1 and formula (2) are used to continue to determine a new second pattern in the suspected pattern until there is no new second pattern or the number of loops from S601 to S605 is greater than the preset number.

[0094] Furthermore, when only a portion of the calibration board is captured in the calibration image, as long as all the first patterns are present in the calibration image, the second pattern in the calibration image can be determined using the scheme of this disclosure. This, in turn, determines the calibration patterns in the calibration image and the matching relationship between each calibration pattern and the first or second mark, thereby calibrating the camera parameters. It is evident that using the scheme of this disclosure, the camera can capture partial or complete images of the calibration board from any angle, and the resulting calibration images can be used for determining the calibration patterns and calibrating the camera parameters. This disclosure improves the robustness and ease of use of calibration pattern recognition.

[0095] In this disclosure, the above method can determine the second matching relationship between the second pattern and the second mark at the same time as determining the second pattern, which facilitates the calibration of camera parameters and can exclude background patterns.

[0096] S308, determine the first and second patterns in the calibration image before scaling as the calibration patterns of the calibration plate.

[0097] In this disclosure, if the first and second patterns determined above are from Figure 4 The determined values ​​in the scaled calibration image 41 shown will correspond to the return. Figure 1 The first and second patterns in the calibration image 12 before scaling are used as calibration patterns.

[0098] In addition, after determining the calibration pattern, subpixel contour detection and ellipse fitting techniques can be used to fit the calibration pattern to obtain the final calibration pattern used to calibrate camera parameters.

[0099] In summary, this disclosure can first determine the first pattern through contour detection technology, then determine the first matching relationship between the first pattern and the first mark by establishing the convex hull, and then determine the second pattern by homography mapping, thereby accurately determining the calibration pattern.

[0100] Furthermore, this disclosure also provides a calibration method, including the calibration pattern recognition method in the above embodiments, which, after determining the calibration pattern, further includes: calibrating camera parameters using the marks and calibration pattern on the calibration board.

[0101] The camera parameters can be calibrated based on the coordinates of the first and second marks on the calibration plate and the coordinates of the calibration pattern in the calibration image before scaling.

[0102] This disclosure can accurately determine the calibration pattern and the matching relationship between each calibration pattern and the mark on the calibration plate, thereby enabling accurate calibration of camera parameters.

[0103] In this embodiment of the disclosure, reference is made to Figure 7 In addition to providing a calibration pattern recognition method, the system also provides a calibration pattern recognition device 70 applied to the above-mentioned calibration pattern recognition method, including:

[0104] The acquisition module 71 is used to acquire a calibration image, which is obtained by the camera taking a picture of the calibration board. The calibration board includes multiple first marks and second marks arranged in an array. The first marks are set in the middle of the calibration board, and the multiple first marks are arranged in a non-rotational symmetric manner.

[0105] The first determining module 72 is used to determine the first matching relationship between the first pattern in the calibration image and the first mark according to the arrangement of the first mark, wherein the outline of the first pattern conforms to the first preset outline.

[0106] The second determining module 73 is used to determine the second pattern in the suspected pattern according to the positional relationship between the second mark and the first mark in the calibration plate and the first matching relationship. The outline of the suspected pattern conforms to the second preset outline. The second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second mark on the calibration plate.

[0107] The third determining module 74 is used to determine that the first pattern and the second pattern are the calibration patterns of the calibration plate.

[0108] In one optional embodiment, the first determining module 72 is specifically used to: extract a first pattern from a calibration image based on a first preset contour; and establish a first matching relationship between the first pattern and the first mark based on the arrangement of the first mark.

[0109] In one optional embodiment, the positional relationship includes the coordinates of the first mark and the second mark in the calibration plate, respectively. The second determining module 73 is specifically used to: determine the homography matrix of the first mark mapped to the first pattern based on the first matching relationship, the coordinates of the first mark in the calibration plate, and the coordinates of the first pattern in the calibration image; extract multiple suspected patterns in the calibration image according to the second preset contour; and determine the suspected pattern that matches the second mark as the second pattern based on the homography matrix, the coordinates of the second mark in the calibration plate, and the coordinates of the suspected pattern in the calibration image.

[0110] In one optional embodiment, when the second determining module 73 determines that the suspected pattern matching the second mark is the second pattern based on the homography matrix, the coordinates of the second mark in the calibration board, and the coordinates of the suspected pattern in the calibration image, it is specifically used to repeatedly execute the following steps: determining the mapped coordinates of the second mark based on the homography matrix and the coordinates of the second mark in the calibration board; determining the target coordinates whose distance from the mapped coordinates is less than a distance threshold among the coordinates of the suspected pattern in the calibration image; determining the suspected pattern corresponding to the target coordinates as the second pattern; and establishing a second matching relationship between the second pattern and the second mark under the condition that a preset condition is met, and updating the homography matrix according to the second matching relationship. The preset condition includes: the number of iterations is less than a preset number of iterations or a new second pattern is determined.

[0111] In one optional embodiment, the arrangement of the first markers is as follows: multiple first markers and at least one second marker array are arranged in a first square array, and at least one second marker is set at the corner position of the first square array.

[0112] In one optional embodiment, when the first determining module 72 establishes a first matching relationship between the first pattern and the first mark according to the arrangement of the first mark, it is specifically used to: determine the first pattern located at the center position as the center pattern among multiple first patterns; determine the first pattern that is symmetrical with the center pattern as the center of symmetry and is collinear with the center pattern as the edge pattern; connect multiple edge patterns to obtain a convex hull; determine the first pattern outside the convex hull as the target pattern; and determine the first matching relationship between the first pattern and the first mark according to the positional relationship between the target pattern and other first patterns and the arrangement of the first mark.

[0113] In one optional embodiment, the calibration pattern recognition device 70 further includes: a scaling module (not shown), used to scale the calibration image according to a preset ratio to obtain a calibration image of a preset size before determining the first matching relationship between the first pattern and the first mark in the calibration image based on the arrangement of the first mark; then the third determining module 73 is used to determine that the first pattern and the second pattern in the calibration image before scaling are calibration patterns of the calibration plate.

[0114] The calibration pattern recognition device provided in this embodiment can first determine a first pattern by contour detection technology, then determine a first matching relationship between the first pattern and the first mark by establishing a convex hull, and then determine a second pattern by homography mapping, thereby accurately determining the calibration pattern.

[0115] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0116] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an example embodiment of this disclosure. For example... Figure 8 As shown, the electronic device 80 includes a processor 81 and a memory 82 communicatively connected to the processor 81, the memory 82 storing computer-executed instructions.

[0117] The processor executes computer execution instructions stored in the memory to implement the calibration pattern recognition method or calibration method provided in any of the above method embodiments. The specific functions and technical effects that can be achieved will not be elaborated here.

[0118] This disclosure also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the calibration pattern recognition method or calibration method provided in any of the above method embodiments.

[0119] This disclosure also provides a computer program product, which includes a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the calibration pattern recognition method or calibration method provided in any of the above method embodiments.

[0120] In the embodiments provided in this disclosure, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0123] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0125] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0126] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for recognizing calibration patterns, characterized in that, include: Acquire a calibration image, which is obtained by the camera taking a picture of the calibration board. The calibration board includes a plurality of first marks and second marks arranged in an array. The first marks are set in the middle position of the calibration board, and the plurality of first marks are arranged in a non-rotationally symmetric manner. Based on the arrangement of the first mark, a first matching relationship between the first pattern in the calibration image and the first mark is determined, wherein the outline of the first pattern conforms to a first preset outline; Based on the positional relationship between the second mark and the first mark in the calibration board, and the first matching relationship, a second pattern is determined from the suspected patterns. The outline of the suspected pattern conforms to the second preset outline. The second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second mark on the calibration board. The first pattern and the second pattern are determined to be the calibration patterns of the calibration plate; The positional relationship includes the coordinates of the first mark and the second mark in the calibration plate, respectively. The step of determining the second pattern in the suspected pattern based on the positional relationship of the second mark and the first mark in the calibration plate and the first matching relationship includes: determining the homography matrix of the first mark to the first pattern based on the first matching relationship, the coordinates of the first mark in the calibration plate and the coordinates of the first pattern in the calibration image. Extract multiple suspected patterns from the calibration image according to the second preset contour; Based on the homography matrix, the coordinates of the second mark in the calibration plate, and the coordinates of the suspected pattern in the calibration image, the suspected pattern that matches the second mark is determined to be the second pattern.

2. The calibration pattern recognition method according to claim 1, characterized in that, Determining the first matching relationship between the first pattern in the calibration image and the first mark based on the arrangement of the first mark includes: Based on the first preset contour, extract the first pattern from the calibration image; Based on the arrangement of the first mark, a first matching relationship is established between the first pattern and the first mark.

3. The calibration pattern recognition method according to claim 1, characterized in that, The step of determining the suspected pattern matching the second mark as the second pattern based on the homography matrix, the coordinates of the second mark in the calibration plate, and the coordinates of the suspected pattern in the calibration image includes repeatedly executing the following steps: Based on the homography matrix and the coordinates of the second mark in the calibration plate, determine the mapped coordinates of the second mark; In the coordinates of the suspected pattern in the calibration image, determine the target coordinates that are less than a distance threshold from the mapped coordinates; The suspected pattern corresponding to the target coordinates is determined to be the second pattern; Under the condition that the preset conditions are met, a second matching relationship between the second pattern and the second mark is established, and the homography matrix is ​​updated according to the second matching relationship. The preset conditions include: the number of iterations is less than a preset number or a new second pattern is determined.

4. The calibration pattern recognition method according to claim 2 or 3, characterized in that, The arrangement of the first marker is as follows: multiple first markers and at least one second marker array are arranged in a first square array, and at least one second marker is set at the corner position of the first square array.

5. The calibration pattern recognition method according to claim 4, characterized in that, The step of establishing a first matching relationship between the first pattern and the first logo based on the arrangement of the first logo includes: Among the multiple first patterns, the first pattern located at the center position is determined as the center pattern; The first pattern that is symmetrical with the central pattern as the center of symmetry and is collinear with the central pattern is identified as the edge pattern. Connecting multiple edge patterns yields a convex hull; The first pattern located outside the convex hull is identified as the target pattern. Based on the positional relationship between the target pattern and other first patterns, and the arrangement of the first mark, a first matching relationship between the first pattern and the first mark is determined.

6. The calibration pattern recognition method according to any one of claims 1 to 3, characterized in that, Before determining the first matching relationship between the first pattern in the calibration image and the first mark based on the arrangement of the first mark, the method further includes: The calibration image is scaled according to a preset ratio to obtain a calibration image of a preset size; The step of determining the first pattern and the second pattern as the calibration patterns of the calibration board includes: The first and second patterns in the calibration image before scaling are determined to be the calibration patterns of the calibration plate.

7. A calibration method, characterized in that, include: The calibration pattern recognition method according to any one of claims 1 to 6, after determining that the first pattern and the second pattern are calibration patterns of the calibration plate, further includes: The camera parameters are calibrated using the marks and calibration patterns on the calibration plate.

8. A calibration plate, characterized in that, The calibration plate is used to implement the calibration pattern recognition method according to any one of claims 1-6, comprising: a plurality of first marks and second marks arranged in an array, wherein the first marks are disposed at the middle position of the calibration plate, and the plurality of first marks are arranged in a non-rotationally symmetrical manner.

9. A calibration pattern recognition device, used to perform the calibration pattern recognition method according to any one of claims 1 to 6, and / or to recognize the calibration pattern corresponding to the mark on the calibration plate according to claim 8, characterized in that, The calibration pattern recognition device includes: The acquisition module is used to acquire a calibration image, which is obtained by the camera taking a picture of the calibration board. The calibration board includes a plurality of first marks and second marks arranged in an array. The first marks are set in the middle position of the calibration board, and the plurality of first marks are arranged in a non-rotational symmetric manner. The first determining module is used to determine a first matching relationship between the first pattern in the calibration image and the first mark based on the arrangement of the first mark, wherein the outline of the first pattern conforms to a first preset outline. The second determining module is used to determine a second pattern from the suspected patterns based on the positional relationship between the second mark and the first mark in the calibration board and the first matching relationship. The outline of the suspected pattern conforms to a second preset outline, and the second pattern is a suspected pattern in the calibration image that conforms to the arrangement of the second mark on the calibration board. The third determining module is used to determine that the first pattern and the second pattern are the calibration patterns of the calibration plate.

10. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the calibration pattern recognition method as described in any one of claims 1 to 6 or the calibration method as described in claim 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the calibration pattern recognition method according to any one of claims 1 to 6 or the calibration method according to claim 7.

Images