Alignment method and device for camera and test graphic card

Abstract

The invention provides an alignment method and device for a camera and a test graphic card, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first image obtained by shooting a first test graphic card by a camera, wherein the first test graphic card comprises a plurality of calibration graphs; controlling the imaging plane of the camera to be parallel to the plane of the first test graph card based on the respective positions of the plurality of calibration graphs in the first image; obtaining a second image obtained by shooting a second test graphic card by the camera, wherein the second test graphic card comprises a plurality of calibration graphs; controlling the center of the optical axis of the camera to coincide with the center of the second test chart card based on the position of the central pixel point of the second image and the respective positions of the plurality of calibration graphs in the second image; alignment of the camera and the test graphic cards comprises the steps that the imaging plane of the camera is parallel to the plane of the first test graphic card, and the center of the optical axis of the camera coincides with the center of the second test graphic card, so that the alignment efficiency of the camera and the test graphic cards can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method and device for aligning a camera with a test chart. Background Art

[0002] To meet the needs of image evaluation, a large number of high-quality images need to be obtained. When obtaining high-quality images, it is usually necessary to adjust the camera pose to align the camera with the test chart and perform shooting in the aligned state.

[0003] Among them, aligning the camera with the test chart requires that: the imaging plane of the camera is parallel to the plane of the test chart, and the optical axis center of the camera also coincides with the center of the test chart.

[0004] Currently, mainly through manual means, the camera pose is adjusted to align the camera with the test chart. However, using the manual method makes the alignment of the camera with the test chart time-consuming, resulting in low alignment efficiency of the camera with the test chart. Summary of the Invention

[0005] This application provides a method and device for aligning a camera with a test chart, which can effectively improve the alignment efficiency of the camera with the test chart.

[0006] This application provides a method for aligning a camera with a test chart, and the method for aligning the camera with the test chart may include:

[0007] Obtain a first image captured by the camera of a first test chart, where the first test chart includes a plurality of calibration patterns;

[0008] Based on the positions of the plurality of calibration patterns in the first image, control the imaging plane of the camera to be parallel to the plane of the first test chart;

[0009] Obtain a second image captured by the camera of a second test chart, where the second test chart includes the plurality of calibration patterns;

[0010] Based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image, control the optical axis center of the camera to coincide with the center of the second test chart;

[0011] Wherein, the alignment of the camera with the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the optical axis center of the camera coincides with the center of the second test chart.

[0012] An embodiment of this application further provides an alignment device for a camera with a test chart, and the alignment device for the camera with the test chart may include:

[0013] A first acquisition unit, configured to acquire a first image obtained by the camera photographing a first test chart, where the first test chart includes a plurality of calibration patterns;

[0014] A first control unit, configured to control the imaging plane of the camera to be parallel to the plane of the first test chart based on the positions of the plurality of calibration patterns in the first image;

[0015] A second acquisition unit, configured to acquire a second image obtained by the camera photographing a second test chart, where the second test chart includes the plurality of calibration patterns;

[0016] A second control unit, configured to control the optical axis center of the camera to coincide with the center of the second test chart based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image;

[0017] Wherein, the alignment of the camera with the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the optical axis center of the camera coincides with the center of the second test chart.

[0018] The present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the program, the alignment method of the camera with the test chart as described in any one of the above is implemented.

[0019] The present application further provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the alignment method of the camera with the test chart as described in any one of the above is implemented.

[0020] The present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the alignment method of the camera with the test chart as described in any one of the above is implemented.

[0021] The alignment method and device for a camera and a test chart provided in this application can obtain a first image captured by the camera of a first test chart when controlling the alignment of the camera and the test chart. The first test chart includes multiple calibration patterns. Based on the positions of the multiple calibration patterns in the first image, the imaging plane of the camera is controlled to be parallel to the plane of the first test chart. Then, a second image captured by the camera of a second test chart is obtained. The second test chart includes multiple calibration patterns. Based on the position of the central pixel point of the second image and the positions of the multiple calibration patterns in the second image, the optical axis center of the camera is controlled to coincide with the center of the second test chart. Among them, the alignment of the camera and the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the optical axis center of the camera coincides with the center of the second test chart. In this way, based on the positions of the multiple calibration patterns in the first image, the imaging plane of the camera is controlled to be parallel to the plane of the first test chart, and based on the position of the central pixel point of the second image and the positions of the multiple calibration patterns in the second image, the optical axis center of the camera is controlled to coincide with the center of the second test chart, realizing the automatic control of the alignment of the camera and the test chart, thereby effectively improving the alignment efficiency of the camera and the test chart. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0023] Figure 1 It is a schematic flowchart of an alignment method for a camera and a test chart provided by an embodiment of this application;

[0024] Figure 2 It is a schematic structural diagram of a first test chart including a square target provided by an embodiment of this application;

[0025] Figure 3 It is a schematic structural diagram of a calibration pattern provided by an embodiment of this application;

[0026] Figure 4 It is a schematic structural diagram of a second test chart including a square target provided by an embodiment of this application;

[0027] Figure 5 It is a schematic flowchart of a method for controlling the imaging plane of a camera to be parallel to the plane of a first test chart provided by an embodiment of this application;

[0028] Figure 6 It is a schematic diagram of the principle of vector inner product provided by an embodiment of this application;

[0029] Figure 7 Schematic diagram of a vector outer product principle provided by an embodiment of the present application;

[0030] Figure 8 Schematic diagram of a method for determining the tilt angle of a camera in the horizontal direction provided by an embodiment of the present application;

[0031] Figure 9 Schematic diagram of a method for determining the tilt angle of a camera in the vertical direction provided by an embodiment of the present application;

[0032] Figure 10 Schematic diagram of a method for determining the tilt angle of a camera in the vertical direction provided by an embodiment of the present application;

[0033] Figure 11 Schematic diagram of a first image when the imaging plane of a camera is parallel to the plane of a first test chart provided by an embodiment of the present application;

[0034] Figure 12 Schematic flowchart of a method for controlling the optical axis center of a camera to coincide with the center of a second test chart provided by an embodiment of the present application;

[0035] Figure 13 Schematic diagram of a second test chart and its central pixel point provided by an embodiment of the present application at this time;

[0036] Figure 14 Schematic diagram of a second image when the optical axis center of a camera coincides with the center of a second test chart provided by an embodiment of the present application;

[0037] Figure 15 Schematic diagram of the structure of an alignment device for a camera and a test chart provided by an embodiment of the present application;

[0038] Figure 16 Schematic diagram of the physical structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0039] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings in the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0040] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In the written description of the present application, the character " / " generally represents an "or" relationship between the front and back associated objects.

[0041] The technical solution provided by the embodiments of the present application can be applied to the automatic shooting scenario of a camera. Currently, mainly through manual means, the camera attitude is adjusted to align the camera with the test chart. However, using the manual method takes a long time to align the camera with the test chart, resulting in a low alignment efficiency between the camera and the test chart.

[0042] To improve the alignment efficiency between the camera and the test chart, the embodiments of the present application provide an alignment method between the camera and the test chart, which can be completed with the help of a mechanical device and a chart switcher. Before executing the alignment method between the camera and the test chart, the camera can be fixed on a mechanical device (robot arm, six-axis slide table) first, and it is determined that the field of view of the camera can cover the chart switcher. The chart switcher can switch to display different test charts to assist in implementing the alignment method between the camera and the test chart through the test chart displayed on the chart switcher, thereby controlling the alignment between the camera and the test chart.

[0043] When assisting in implementing the alignment method between the camera and the test chart through the test chart displayed on the chart switcher, the chart switcher can be controlled to display a first test image including a plurality of calibration patterns first, and based on the positions of the respective calibration patterns in the first image obtained by photographing the first test chart, the imaging plane of the camera is controlled to be parallel to the plane of the first test chart; then the chart switcher is controlled to display a second test image including a plurality of calibration patterns, and based on the positions of the respective calibration patterns in the second image obtained by photographing the second test chart, the optical axis center of the camera is controlled to coincide with the center of the second test chart; in this way, based on the positions of the respective calibration patterns in the first image, the imaging plane of the camera is controlled to be parallel to the plane of the first test chart, and based on the position of the central pixel point of the second image and the positions of the respective calibration patterns in the second image, the optical axis center of the camera is controlled to coincide with the center of the second test chart, realizing the automatic control of the alignment between the camera and the test chart, thereby effectively improving the alignment efficiency between the camera and the test chart.

[0044] Among them, for the alignment between the camera and the test chart to be satisfied, the imaging plane of the camera needs to be parallel to the plane of the test chart, and the optical axis center of the camera also needs to coincide with the center of the test chart. That is, only when both of these two conditions are met can the alignment between the camera and the test chart be achieved.

[0045] Next, the alignment method of the camera and the test chart provided by this application will be described in detail through the following specific embodiments. It can be understood that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0046] Figure 1 It is a schematic flowchart of an alignment method for a camera and a test chart provided by an embodiment of this application. This alignment method can be executed by software and / or hardware devices. For example, please refer to Figure 1 As shown, this alignment method for the camera and the test chart may include:

[0047] S101. Obtain a first image captured by the camera of a first test chart, where the first test chart includes multiple calibration patterns.

[0048] For example, in an embodiment of this application, the first test chart may be a checkerboard calibration board with a specification of 9 by 11, and multiple calibration patterns are set on the first test chart. Among them, the number of calibration patterns can be set according to actual needs.

[0049] Among them, the multiple calibration patterns in the first test chart are used to assist in controlling the imaging plane of the camera to be parallel to the plane of the first test chart. For example, taking the number of calibration patterns as 3 and the calibration pattern being a return label as an example, please refer to Figure 2 As shown, Figure 2 It is a schematic structural diagram of a first test chart including a return label provided by an embodiment of this application. Combining Figure 2 it can be seen that the image features of the return label are: it includes three contours, namely the inner black square contour, the white square contour, and the outer black square; the width ratio of the black and white frames of each "return label" is 1:1:3:1:1.

[0050] When obtaining the first image captured by the camera of the first test chart, the test chart switch can be controlled to display the first test chart first, and the camera can take a picture of the first test chart to obtain the first image. Since the first test chart includes multiple calibration patterns, correspondingly, the first image obtained by taking a picture of the first test chart will also include the multiple calibration patterns.

[0051] After obtaining the first image captured of the first test chart, the following S102 can be executed:

[0052] S102. Based on the positions of the multiple calibration patterns in the first image, control the imaging plane of the camera to be parallel to the plane of the first test chart.

[0053] It is not difficult to understand that before controlling the imaging plane of the camera to be parallel to the plane of the first test chart based on the positions of multiple calibration patterns in the first image, it is necessary to first identify the multiple calibration patterns in the first image.

[0054] Exemplarily, in the embodiments of the present application, the multiple calibration patterns in the first image can be identified by the OpenCV operator findContours. The specific process is as follows: The OpenCV operator findContours can be used to find the contours in the first image. Among them, the parent contour and the child contour in the contours are continuously sorted, that is, the contour after the parent contour is its child contour. All the contours are judged to determine whether there are two child contours inside the contour and whether the width ratio of the black and white frames meets 1:1:3:1:1. Exemplarily, reference can be made to Figure 3 as shown. Figure 3 FIG. is a schematic structural diagram of a calibration pattern provided by an embodiment of the present application. If the above conditions are met, it is determined as a calibration pattern, thereby identifying multiple calibration patterns in the first image.

[0055] It can be understood that after determining the contour that meets the above conditions, the central pixel coordinates of the contour are the central pixel coordinates of the calibration pattern. In the subsequent description, the central pixel coordinates of the calibration image are the pixel coordinates of the calibration image. The pixel coordinates (u, v) can be understood as the coordinates in the pixel coordinate system.

[0056] When controlling the imaging plane of the camera to be parallel to the plane of the first test chart based on the positions of multiple calibration patterns in the first image, the robotic arm of the mechanical device can be controlled to move to adjust the posture of the camera, so that the imaging plane of the camera is parallel to the plane of the first test chart, thereby meeting the first condition for aligning the camera with the test chart, that is, the imaging plane of the camera is parallel to the plane of the test chart.

[0057] Through the above S101 and S102, the imaging plane of the camera can be made parallel to the plane of the test chart. Next, through the following S103 and S104, the optical axis center of the camera is controlled to coincide with the center of the second test chart, thereby meeting the second condition for aligning the camera with the test chart.

[0058] S103. Obtain a second image obtained by the camera photographing the second test chart, and the second test chart includes multiple calibration patterns.

[0059] Exemplarily, in the embodiments of the present application, the second test chart can be one of a clarity test card, an SG color card, a standard 24-color card, or other test charts, which can be specifically set according to actual needs, and multiple calibration patterns are set on the second test chart. Among them, the number of calibration patterns can be set according to actual needs.

[0060] Among them, multiple calibration patterns in the second test card are used to assist in controlling the coincidence of the optical axis center of the camera and the center of the second test card. By way of example, taking the number of calibration patterns as 3 and the calibration pattern as a paperclip, reference can be made to Figure 4 as shown Figure 4 FIG. Figure 4 is a schematic structural diagram of a second test card including a paperclip provided by an embodiment of the present application. Combining Figure 4 it can also be seen that the image features of the paperclip are as follows: it includes three contours, namely an internal black square contour, a white square contour, and an external black square; the width ratio of the black and white frames of each "paperclip" is 1:1:3:1:1.

[0061] When obtaining the second image captured by the camera for the second test card, the card switcher can be first controlled to switch from the first test card to the second test card, and the second test card is displayed. The camera takes a picture of the second test card to obtain the second image. Since the second test card includes multiple calibration patterns, correspondingly, the second image obtained by taking a picture of the second test card will also include the multiple calibration patterns.

[0062] After the second image captured for the second test card, the following S104 can be executed:

[0063] S104: Based on the position of the central pixel point of the second image and the positions of the multiple calibration patterns in the second image, control the coincidence of the optical axis center of the camera and the center of the second test card.

[0064] Among them, the alignment of the camera and the test card includes that the imaging plane of the camera is parallel to the plane of the first test card, and the optical axis center of the camera coincides with the center of the second test card, so as to realize the alignment of the camera and the test card.

[0065] It is not difficult to understand that before controlling the coincidence of the optical axis center of the camera and the center of the second test card based on the positions of the multiple calibration patterns in the second image, it is also necessary to first identify the multiple calibration patterns in the second image.

[0066] By way of example, in the embodiment of the present application, the multiple calibration patterns in the second image can be identified by the OpenCV operator findContours. Its specific implementation is similar to the implementation of identifying the multiple calibration patterns in the second image by the OpenCV operator findContours above. Reference can be made to the relevant description of the above implementation. Here, the embodiment of the present application will not be elaborated further, so as to identify the multiple calibration patterns in the second image.

[0067] When controlling the optical axis center of the camera to coincide with the center of the second test card based on the position of the central pixel point of the second image and the positions of multiple calibration patterns in the second image, the robotic arm of the mechanical device can be controlled to move to adjust the pose of the camera, so that the optical axis center of the camera coincides with the center of the second test card, thereby meeting the second condition for aligning the camera with the test card.

[0068] It can be seen that in the embodiments of the present application, when controlling the alignment of the camera with the test card, by obtaining the first image captured by the camera for the first test card, where the first test card includes multiple calibration patterns; and based on the positions of the multiple calibration patterns in the first image, controlling the imaging plane of the camera to be parallel to the plane of the first test card; by obtaining the second image captured by the camera for the second test card, where the second test card includes multiple calibration patterns; and based on the position of the central pixel point of the second image and the positions of the multiple calibration patterns in the second image, controlling the optical axis center of the camera to coincide with the center of the second test card. In this way, based on the positions of the multiple calibration patterns in the first image, controlling the imaging plane of the camera to be parallel to the plane of the first test card, and based on the position of the central pixel point of the second image and the positions of the multiple calibration patterns in the second image, controlling the optical axis center of the camera to coincide with the center of the second test card, realizing the automatic control of the alignment of the camera with the test card, thereby effectively improving the alignment efficiency of the camera with the test card.

[0069] Based on the above Figure 1 shown embodiments, in order to facilitate understanding of how to control the imaging plane of the camera to be parallel to the plane of the first test card in S102 above, below, it will be described in detail through Figure 5 the shown embodiments.

[0070] Figure 5 It is a schematic flowchart of a method for controlling the imaging plane of the camera to be parallel to the plane of the first test card provided by the embodiments of the present application, and this method can also be executed by software and / or hardware devices. For example, please refer to Figure 5 shown, the method for controlling the imaging plane of the camera to be parallel to the plane of the first test card may include:

[0071] S501. Determine the tilt angle of the camera in the camera coordinate system based on the positions of the multiple calibration patterns in the first image.

[0072] Among them, the tilt angle of the camera in the camera coordinate system includes: the tilt angle of the camera in the horizontal direction, the tilt angle in the vertical direction, and the tilt angle in the vertical direction. For example, in the embodiments of the present application, the tilt angle of the camera in the horizontal direction can be denoted as Rx, the tilt angle of the camera in the vertical direction can be denoted as Ry, and the tilt angle of the camera in the vertical direction can be denoted as Rz.

[0073] Taking the number of multiple calibration patterns as 3, that is, taking the example that the first image includes three calibration patterns, for example, in the embodiment of the present application, when determining the tilt angle of the camera in the camera coordinate system based on the positions of the multiple calibration patterns in the first image, the first calibration pattern, the second calibration pattern, and the third calibration pattern can be determined from the three calibration patterns based on the positions of the three calibration patterns in the first image; among them, the straight line where the first calibration pattern and the second calibration pattern are located is perpendicular to the straight line where the first calibration pattern and the third calibration pattern are located, and on the first test chart, the ordinate of the first calibration pattern is the same as that of the second calibration pattern, and the abscissa of the first calibration pattern and the third calibration pattern is the same, that is, the first calibration pattern is the calibration pattern located in the upper left corner of the first image, the second calibration pattern is the calibration pattern located in the upper right corner of the first image, and the third calibration pattern is the calibration pattern located in the lower left corner of the first image. Then, based on the positions of the first calibration pattern and the second calibration pattern in the camera coordinate system respectively, the tilt angle of the camera in the vertical direction and the tilt angle in the vertical direction are determined; based on the positions of the first calibration pattern and the third calibration pattern in the camera coordinate system respectively, the tilt angle of the camera in the horizontal direction is determined.

[0074] Combined with the above description, through the OpenCV operator findContours, only the number of calibration patterns included in the first image and the pixel coordinates of each calibration pattern can be recognized, but it is impossible to determine which one of the three calibration patterns is the first calibration pattern, which one is the second calibration pattern, and which one is the third calibration pattern. Therefore, it is necessary to further determine the first calibration pattern, the second calibration pattern, and the third calibration pattern from the three calibration patterns based on the positions of the three calibration patterns respectively.

[0075] For example, when determining the first calibration pattern, the second calibration pattern, and the third calibration pattern from the three calibration patterns based on the positions of the three calibration patterns respectively, considering that the included angle between the calibration pattern located in the upper left corner and the other two calibration patterns is 90 degrees, therefore, the "vector inner product principle" can be used for judgment. For example, see Figure 6 as shown Figure 6 is a schematic diagram of a vector inner product principle provided by an embodiment of the present application. If the vector a=(x1,y1) and b=(x2,y2), the necessary and sufficient condition for the vector a to be perpendicular to the vector b is a·b = 0 (the vector inner product is equal to 0), that is, (x1x2 + y1y2) = 0.

[0076] Based on the above vector inner product principle, the first calibration pattern can be determined from the three calibration images. Specifically: vectors corresponding to each pair of calibration patterns can be constructed based on the positions of the three calibration patterns; and based on the inner product of the vectors corresponding to each pair of calibration patterns, the first calibration pattern can be determined from the three calibration patterns. For example, the pixel coordinates of any one calibration pattern are subtracted from the pixel coordinates of the other two calibration patterns respectively to obtain the corresponding vectors a and b. If a·b = 0, it indicates that this calibration pattern is the calibration pattern located at the upper left corner of the first image, that is, the first calibration pattern.

[0077] After the first calibration pattern is determined from the three calibration patterns based on the vector inner product principle, the second calibration pattern and the third calibration pattern can be determined from the other two calibration patterns based on the outer product between the two vectors constructed by the positions of the first calibration pattern and the positions of the other two calibration patterns respectively.

[0078] Exemplarily, when determining the second calibration pattern and the third calibration pattern from the other two calibration patterns based on the outer product between the two vectors constructed by the positions of the first calibration pattern and the positions of the other two calibration patterns respectively, assuming the pixel coordinates of the first calibration pattern are (u1, v1), and the pixel coordinates of the other two calibration patterns are (u2, v2) and (u3, v3) respectively, then the pixel coordinates (u1, v1) of the first calibration pattern can be subtracted from the pixel coordinates (u2, v2) of one calibration pattern to obtain vector c, and the pixel coordinates (u1, v1) of the first calibration pattern can be subtracted from the pixel coordinates (u3, v3) of the other calibration pattern to obtain vector d. Exemplarily, reference can be made to Figure 7 as shown Figure 7 which is a schematic diagram of a vector outer product principle provided by an embodiment of the present application. If c×d > 0, that is, the vector outer product is greater than 0, it indicates that the ordinate in the pixel coordinates of one calibration pattern (u2, v2) is the same as the ordinate in the pixel coordinates of the first calibration pattern, and the abscissa in the pixel coordinates of the other calibration pattern (u3, v3) is the same as the abscissa in the pixel coordinates of the first calibration pattern; conversely, if c×d < 0, that is, the vector outer product is less than 0, it indicates that the abscissa in the pixel coordinates of one calibration pattern (u2, v2) is the same as the abscissa in the pixel coordinates of the first calibration pattern, and the ordinate in the pixel coordinates of the other calibration pattern (u3, v3) is the same as the ordinate in the pixel coordinates of the first calibration pattern. In this way, the second calibration pattern and the third calibration pattern can be determined.

[0079] Among them, the second calibration pattern is the calibration pattern located at the upper right corner of the first image, and the third calibration pattern is the calibration pattern located at the lower left corner of the first image.

[0080] After respectively determining the first calibration pattern located in the upper left corner, the second calibration pattern located in the upper right corner, and the third calibration pattern located in the lower left corner in the first image, considering that the pixel coordinates of the first calibration pattern, the second calibration pattern, and the third calibration pattern are all coordinates in the pixel coordinate system, to determine the tilt angle of the camera in the camera coordinate system, it is also necessary to further perform coordinate transformation on the pixel coordinates of the first calibration pattern, the second calibration pattern, and the third calibration pattern respectively to obtain the positions of the first calibration pattern, the second calibration pattern, and the third calibration pattern in the camera coordinate system respectively; only in this way can the tilt angle of the camera in the camera coordinate system be determined based on the positions of the first calibration pattern, the second calibration pattern, and the third calibration pattern in the camera coordinate system respectively.

[0081] Exemplarily, in the embodiments of the present application, when obtaining the positions of the first calibration pattern, the second calibration pattern, and the third calibration pattern in the camera coordinate system respectively, considering that the methods for obtaining the positions of each calibration image in the camera coordinate system are similar, therefore, to avoid repetition, any one of the first calibration pattern, the second calibration pattern, and the third calibration pattern will be taken as an example to illustrate how to obtain the positions of the first calibration pattern, the second calibration pattern, and the third calibration pattern in the camera coordinate system respectively.

[0082] Assume that the camera is a monocular camera, and it is stated that the pixel positions obtained in the pixel coordinate system do not include the depth of the calibration pattern. Then, when obtaining the position of any calibration pattern in the camera coordinate system, the depth of the calibration pattern can be determined first based on the pixel coordinates of the calibration pattern, the camera coordinate system, and the world coordinate system; and after obtaining the depth of the calibration pattern, the position of the calibration pattern in the camera coordinate system can be jointly determined based on the pixel coordinates of the calibration pattern, the depth of the calibration pattern, and the internal parameter matrix of the camera.

[0083] Exemplarily, when determining the depth of the calibration pattern based on the pixel coordinates of the calibration pattern, the camera coordinate system, and the world coordinate system, the camera calibration can be completed first through the OpenCV operator calibrateCamera. During the camera calibration process, the internal parameter matrix and the external parameter matrix of the camera can be obtained. Among them, the internal parameter matrix of the camera is used to represent the conversion relationship between the "pixel coordinate system" and the "camera coordinate system", and the external parameter matrix of the camera is used to represent the conversion relationship between the "camera coordinate system" and the "world coordinate system", then the depth of the calibration pattern can be determined through the following formula 1.

[0084]

[0085] Among them, s represents the depth of the calibration pattern, (u, v) represents the pixel coordinates of the calibration pattern, represents the internal parameter matrix of the camera, represents the external parameter matrix of the camera, Indicates the coordinates of the calibration pattern in the world coordinate system.

[0086] As shown in the above formula 1, the external parameter matrix of the camera includes a rotation matrix and a translation matrix. Then, the conversion relationship between the pixel coordinates and the world coordinates in the above formula 1 can be simplified as shown in the following formula 2:

[0087]

[0088] Where M represents the internal parameter matrix of the camera, R represents the rotation matrix, t represents the translation matrix, Zconst is the height of the world coordinate system and can be set to 0. Then, performing matrix transformation on the above formula 2, the following formula 3 can be obtained:

[0089]

[0090] Substituting the pixel coordinates (u, v) of the calibration pattern into the above formula 3, the depth s of the calibration pattern can be calculated. After calculating the depth s of the calibration pattern, the pixel coordinates (u, v) and the depth s of the calibration pattern can be substituted into the formula to obtain the position (x c , y c , z c ) of the calibration pattern in the camera coordinate system.

[0091] Combined with the above description, the positions (Xc1, Yc1, Zc1) of the first calibration pattern, the positions (Xc2, Yc2, Zc2) of the second calibration pattern, and the positions (Xc3, Yc3, Zc3) of the third calibration pattern in the camera coordinate system can be obtained respectively.

[0092] It should be noted that in the embodiments of the present application, if the camera is a binocular camera or a depth camera, the depth of the calibration pattern can be directly obtained through the camera. In this case, there is no need to calculate the depth of the calibration pattern in the above manner. Instead, the pixel coordinates (u, v) and the depth s of the calibration pattern can be directly substituted into the formula to obtain the position (x c , y c , z c ) of the calibration pattern in the camera coordinate system.

[0093] After obtaining the positions (Xc1, Yc1, Zc1) of the first calibration pattern, (Xc2, Yc2, Zc2) of the second calibration pattern, and (Xc3, Yc3, Zc3) of the third calibration pattern in the camera coordinate system respectively, the tilt angle of the camera in the horizontal direction can be determined based on the positions of the first calibration pattern and the third calibration pattern in the camera coordinate system respectively; the tilt angle of the camera in the vertical direction and the tilt angle in the vertical direction can be determined based on the positions of the first calibration pattern and the second calibration pattern in the camera coordinate system respectively.

[0094] When determining the tilt angle of the camera in the horizontal direction based on the positions of the first calibration pattern and the third calibration pattern in the camera coordinate system respectively, by way of example, reference can be made to Figure 8 as shown Figure 8 which is a schematic diagram for determining the tilt angle of the camera in the horizontal direction provided by an embodiment of the present application. Assume that the center point of the first calibration pattern is H1, the center point of the second calibration pattern is H2, and the center point of the third calibration pattern is H3. The center point H3 of the third calibration pattern can be projected onto the plane passing through the center point H1 of the first calibration pattern and parallel to the camera X-Y axis along the direction of the camera Z axis, and the intersection point is denoted as point P2. Connect points H1, P2, and H3 to form a right triangle. Calculate and determine the tilt angle Rx of the camera in the horizontal direction. Reference can be made to the formula where ∠P2H1H3 is the tilt angle Rx of the camera in the horizontal direction, thereby determining the tilt angle Rx of the camera in the horizontal direction.

[0095] When determining the tilt angle of the camera in the vertical direction based on the positions of the first calibration pattern and the second calibration pattern in the camera coordinate system respectively, by way of example, reference can be made to Figure 9 as shown Figure 9 which is a schematic diagram for determining the tilt angle of the camera in the vertical direction provided by an embodiment of the present application. Assume that the center point of the first calibration pattern is H1, the center point of the second calibration pattern is H2, and the center point of the third calibration pattern is H3. The center point H1 of the first calibration pattern can be projected onto the plane passing through the center point H2 of the second calibration pattern and parallel to the camera X-Y axis along the direction of the camera Z axis, and the intersection point is denoted as point P3. Connect points H1, P3, and H2 to form a right triangle. Determine the tilt angle Ry of the camera in the vertical direction by calculation. Reference can be made to the formula where ∠P3H2H1 is the tilt angle Ry of the camera in the vertical direction, thereby determining the tilt angle Ry of the camera in the vertical direction.

[0096] When determining the tilt angle of the camera in the vertical direction based on the positions of the first calibration pattern and the second calibration pattern in the camera coordinate system respectively, by way of example, reference can be made to Figure 10 as shownFigure 10 This is a schematic diagram for determining the tilt angle of a camera in the vertical direction provided by an embodiment of the present application. Assume that the center point of the first calibration pattern is H1, the center point of the second calibration pattern is H2, and the center point of the third calibration pattern is H3. A straight line parallel to the X-axis of the camera can be drawn through the center H1 of the first calibration pattern, and then a perpendicular line to this straight line can be drawn through the center point H2 of the second calibration pattern, and the intersection point is denoted as intersection point P1. Finally, connecting point H1, point P1, and point H2 can form a right triangle. By calculating, the tilt angle Rz of the camera in the vertical direction can be determined. Refer to the formula where ∠P1H1H2 is the tilt angle Rz of the camera in the vertical direction, thereby determining the tilt angle Rz of the camera in the vertical direction.

[0097] Combined with the above description, the tilt angle Rx of the camera in the horizontal direction, the tilt angle Ry in the vertical direction, and the tilt angle Rz in the vertical direction can be determined. Then, it can be further determined whether the three obtained tilt angles are all less than a preset tilt angle threshold. If at least one of the three tilt angles is greater than or equal to the preset tilt angle threshold, then perform the following S502 and S503; if all three tilt angles are less than the preset tilt angle threshold, then directly perform the following S503:

[0098] S502. When at least one of the tilt angles is greater than or equal to the preset tilt angle threshold, adjust the pose of the camera based on the tilt angle, and re-acquire the image obtained by the camera shooting the first test chart based on the adjusted pose, and use the image as the new first image, and repeat the above operations until the tilt angles of the camera in the camera coordinate system are all less than the preset tilt angle threshold.

[0099] Among them, the value of the preset tilt angle threshold can be set according to actual needs. For example, in the embodiment of the present application, the preset tilt angle threshold can be set to 0.1 degree.

[0100] It can be understood that if at least one of the three tilt angles, namely the tilt angle Rx of the camera in the horizontal direction, the tilt angle Ry in the vertical direction, and the tilt angle Rz in the vertical direction, is greater than or equal to the preset tilt angle threshold, it means that the imaging plane of the camera is not parallel to the plane of the first test chart. In this case, the pose of the camera can be adjusted based on -Rx, -Ry, and -Rz, and the image obtained by the camera shooting the first test chart based on the adjusted pose can be re-acquired, and the image is used as the new first image, and the above steps are repeated until the tilt angle Rx of the camera in the horizontal direction, the tilt angle Ry in the vertical direction, and the tilt angle Rz in the vertical direction are all less than the preset tilt angle threshold.

[0101] S503. When the tilt angles are all smaller than a preset tilt angle threshold, determine that the imaging plane of the camera is parallel to the plane of the first test chart.

[0102] It can be understood that if the tilt angle Rx of the camera in the horizontal direction, the tilt angle Ry in the vertical direction, and the tilt angle Rz in the vertical direction are all smaller than the preset tilt angle threshold, it means that the imaging plane of the camera is parallel to the plane of the first test chart. By way of example, please refer to Figure 11 as shown in Figure 11 FIG. 7 is a schematic diagram of a first image when the imaging plane of the camera provided in an embodiment of the present application is parallel to the plane of the first test chart.

[0103] It can be seen that in the embodiment of the present application, when controlling the imaging plane of the camera to be parallel to the plane of the first test chart, the tilt angle of the camera in the camera coordinate system can be first determined based on the positions of multiple calibration patterns in the first image; and based on the tilt angle of the camera in the camera coordinate system, the imaging plane of the camera is controlled to be parallel to the plane of the first test chart, realizing automatic control of the imaging plane of the camera to be parallel to the plane of the first test chart, that is, satisfying the first condition for the alignment of the camera and the test chart.

[0104] To achieve the alignment of the camera and the test chart, in addition to controlling the imaging plane of the camera to be parallel to the plane of the first test chart, it is also necessary to control the optical axis center of the camera to coincide with the center of the second test chart, so as to achieve the alignment of the camera and the test chart.

[0105] Based on the above Figure 1 shown embodiment, for the convenience of understanding how to control the optical axis center of the camera to coincide with the center of the second test chart based on the position of the central pixel point of the second image and the positions of multiple calibration patterns in the second image in S104 above, below, Figure 12 the shown embodiment will be described in detail.

[0106] Figure 12 FIG. 8 is a schematic flowchart of a method for controlling the optical axis center of the camera to coincide with the center of the second test chart provided in an embodiment of the present application, and this method can also be executed by software and / or hardware devices. By way of example, please refer to Figure 12 as shown in FIG. 8, the method for controlling the optical axis center of the camera to coincide with the center of the second test chart may include:

[0107] S1201. Based on the positions of multiple calibration patterns in the second image, determine the position of the central pixel point of the second test chart.

[0108] Exemplarily, after obtaining the pixel coordinates of the first calibration pattern as H1(u1, v1), the pixel coordinates of the second calibration pattern as H2(u2, v2), and the pixel coordinates of the third calibration pattern as H3(u3, v3), the abscissa of the central pixel of the second test chart can be determined based on the respective position coordinates of the first calibration pattern and the second calibration pattern; the ordinate of the central pixel of the second test chart can be determined based on the respective positions of the first calibration pattern and the third calibration pattern.

[0109] Exemplarily, reference can be made to Figure 13 as shown in Figure 13 which is a schematic diagram of a second test chart and its central pixel provided by an embodiment of the present application. Among them, point P1(x1, y1) is the central pixel of the second test chart, and point P2(x2, y2) is the central pixel of the second image. Among them, the abscissa of the central pixel of the second test chart The ordinate of the central pixel of the second test chart The abscissa of the central pixel of the second image The ordinate of the central pixel of the second image

[0110] After respectively determining the positions of the central pixel of the second image and the central pixel of the second test chart, the following S1202 can be executed:

[0111] S1202: When the difference between the position of the central pixel of the second image and the position of the central pixel of the second test chart is greater than or equal to a preset difference, adjust the pose of the camera based on the difference, and re-obtain the image captured by the camera of the second test chart based on the adjusted pose, and use the image as the new second image, and repeat the above operation until the difference between the position of the central pixel of the second image and the position of the central pixel of the second test chart is less than the preset difference.

[0112] Exemplarily, the preset difference between the position of the central pixel of the second image and the position of the central pixel of the second test chart can be 2 pixels, and it can be specifically set according to actual requirements.

[0113] Exemplarily, the difference between the position of the central pixel of the second image and the position of the central pixel of the second test chart includes the abscissa difference and the ordinate difference; among them, the abscissa difference can be recorded as △x = x2 - x1, and the ordinate difference △y = y2 - y1, where the abscissa difference and the ordinate difference are the offset amounts of the optical axis center of the camera.

[0114] If there is a difference greater than or equal to a preset difference in the horizontal coordinate difference and the vertical coordinate difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart, it indicates that the optical axis center of the camera does not coincide with the center of the second test chart. In this case, the attitude of the camera can be adjusted based on -△x and -△y, and the image obtained by the camera shooting the second test chart is re-acquired based on the adjusted attitude, and the image is used as the new second image. Repeat the above steps until the differences in the horizontal coordinate difference and the vertical coordinate difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart are both less than the preset difference.

[0115] S1203. When the difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart is less than the preset difference, it is determined that the optical axis center of the camera coincides with the center of the second test chart.

[0116] It can be understood that if the differences in the horizontal coordinate difference and the vertical coordinate difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart are both less than the preset difference, it indicates that the optical axis center of the camera coincides with the center of the second test chart. By way of example, please refer to Figure 14 as shown Figure 14 This is a schematic diagram of the second image when the optical axis center of a camera provided in an embodiment of the present application coincides with the center of the second test chart.

[0117] It can be seen that in the embodiment of the present application, when controlling the coincidence of the optical axis center of the camera with the center of the second test chart, the position of the central pixel point of the second test chart can be determined based on the positions of multiple calibration patterns in the second image; and based on the difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart, the optical axis center of the camera is controlled to coincide with the center of the second test chart, realizing the automatic control of the coincidence of the optical axis center of the camera with the center of the second test chart, that is, meeting the second condition for the alignment of the camera and the test chart.

[0118] Combined with the above description, based on the tilt angle of the camera in the camera coordinate system, controlling the imaging plane of the camera to be parallel to the plane of the first test chart can realize the automatic control of the imaging plane of the camera to be parallel to the plane of the first test chart; based on the difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart, controlling the optical axis center of the camera to coincide with the center of the second test chart realizes the automatic control of the coincidence of the optical axis center of the camera with the center of the second test chart, thereby realizing the automatic control of the alignment of the camera and the test chart and improving the alignment efficiency of the camera and the test chart.

[0119] The following describes an alignment device for a camera and a test chart provided by the present application. The alignment device for a camera and a test chart described below can be correspondingly referred to the alignment method for a camera and a test chart described above.

[0120] Figure 15 FIG. is a schematic structural diagram of an alignment device for a camera and a test chart provided by an embodiment of the present application. For example, please refer to Figure 15 As shown, the alignment device 150 for a camera and a test chart may include:

[0121] A first acquisition unit 1501, configured to acquire a first image obtained by the camera photographing a first test chart, where the first test chart includes a plurality of calibration patterns.

[0122] A first control unit 1502, configured to control the imaging plane of the camera to be parallel to the plane of the first test chart based on the positions of the plurality of calibration patterns in the first image.

[0123] A second acquisition unit 1503, configured to acquire a second image obtained by the camera photographing a second test chart, where the second test chart includes a plurality of calibration patterns.

[0124] A second control unit 1504, configured to control the optical axis center of the camera to coincide with the center of the second test chart based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image.

[0125] Wherein, the alignment of the camera and the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the optical axis center of the camera coincides with the center of the second test chart.

[0126] For example, in an embodiment of the present application, the first control unit 1502 is specifically configured to:

[0127] Based on the positions of the plurality of calibration patterns in the first image, determine the tilt angle of the camera in the camera coordinate system; in the case that at least one of the tilt angles is greater than or equal to a preset tilt angle threshold, adjust the attitude of the camera based on the tilt angle, and re-acquire the image obtained by the camera photographing the first test chart based on the adjusted attitude, and use the image as a new first image, and repeat the above operations until the tilt angles of the camera in the camera coordinate system are all less than the preset tilt angle threshold; in the case that the tilt angles are all less than the preset tilt angle threshold, determine that the imaging plane of the camera is parallel to the plane of the first test chart.

[0128] For example, in an embodiment of the present application, the number of the plurality of calibration patterns is three, and the first control unit 1502 is specifically configured to:

[0129] Based on the positions of the three calibration patterns respectively, determine the first calibration pattern, the second calibration pattern, and the third calibration pattern from the three calibration patterns; wherein, the straight line where the first calibration pattern and the second calibration pattern are located is perpendicular to the straight line where the first calibration pattern and the third calibration pattern are located, and on the first test card, the ordinates of the first calibration pattern and the second calibration pattern are the same, and the abscissas of the first calibration pattern and the third calibration pattern are the same; Based on the positions of the first calibration pattern and the second calibration pattern respectively in the camera coordinate system, determine the tilt angle of the camera in the vertical direction and the tilt angle in the vertical direction; Based on the positions of the first calibration pattern and the third calibration pattern respectively in the camera coordinate system, determine the tilt angle of the camera in the horizontal direction; wherein, the tilt angle of the camera in the camera coordinate system includes the tilt angle of the camera in the horizontal direction, the tilt angle in the vertical direction, and the tilt angle in the vertical direction.

[0130] Exemplarily, in the embodiment of the present application, the first control unit 1502 is specifically configured to:

[0131] Based on the positions of the three calibration patterns respectively, construct vectors corresponding to pairwise calibration patterns; based on the inner product of the vectors corresponding to the pairwise calibration patterns, determine the first calibration pattern from the three calibration patterns; based on the outer product between the two vectors constructed from the position of the first calibration pattern and the positions of the other two calibration patterns respectively, determine the second calibration pattern and the third calibration pattern from the other two calibration patterns.

[0132] Exemplarily, in the embodiment of the present application, the second control unit 1504 is specifically configured to:

[0133] Based on the positions of the multiple calibration patterns in the second image respectively, determine the position of the central pixel point of the second test card; when the difference between the position of the central pixel point in the second image and the position of the central pixel point of the second test card is greater than or equal to a preset difference, adjust the pose of the camera based on the difference, and re-acquire the image obtained by the camera shooting the second test card based on the adjusted pose, and use the image as the new second image, and repeat the above operations until the difference between the position of the central pixel point in the second image and the position of the central pixel point of the second test card is less than the preset difference; when the difference between the position of the central pixel point in the second image and the position of the central pixel point of the second test card is less than the preset difference, determine that the optical axis center of the camera coincides with the center of the second test card.

[0134] Exemplarily, in the embodiment of the present application, the second control unit 1504 is specifically configured to:

[0135] Based on the positions of the first calibration pattern and the second calibration pattern respectively, determine the abscissa of the central pixel point of the second test card; based on the positions of the first calibration pattern and the third calibration pattern respectively, determine the ordinate of the central pixel point of the second test card.

[0136] Exemplarily, in an embodiment of the present application, when the camera is a monocular camera, the alignment device 150 between the camera and the test chart further includes a processing unit for:

[0137] For any one of the first calibration pattern, the second calibration pattern, and the third calibration pattern, based on the pixel coordinates of the calibration pattern, the camera coordinate system, and the world coordinate system, determine the depth of the calibration pattern; based on the pixel coordinates of the calibration pattern, the depth of the calibration pattern, and the internal parameter matrix of the camera, determine the position of the calibration pattern in the camera coordinate system.

[0138] The alignment device 150 between the camera and the test chart provided in the embodiment of the present application can execute the technical solutions of the alignment method between the camera and the test chart in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the alignment method between the camera and the test chart. For details, refer to the implementation principle and beneficial effects of the alignment method between the camera and the test chart, and will not be elaborated here.

[0139] Figure 16 FIG. is a schematic physical structure diagram of an electronic device provided in an embodiment of the present application, as Figure 16 shown. The electronic device may include: a processor 1610, a communication interface 1620, a memory 1630, and a communication bus 1640. Among them, the processor 1610, the communication interface 1620, and the memory 1630 communicate with each other through the communication bus 1640. The processor 1610 may call the logical instructions in the memory 1630 to execute the alignment method between the camera and the test chart. The method includes: obtaining a first image captured by the camera of a first test chart, where the first test chart includes a plurality of calibration patterns; based on the positions of the plurality of calibration patterns in the first image, controlling the imaging plane of the camera to be parallel to the plane of the first test chart; obtaining a second image captured by the camera of a second test chart, where the second test chart includes a plurality of calibration patterns; based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image, controlling the optical axis center of the camera to coincide with the center of the second test chart; where the alignment between the camera and the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the optical axis center of the camera coincides with the center of the second test chart.

[0140] In addition, when the logical instructions in the above-mentioned memory 1630 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0141] On the other hand, this application also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the alignment method of the camera and the test chart provided by the above-mentioned various methods. The method includes: obtaining a first image captured by the camera of a first test chart, where the first test chart includes multiple calibration patterns; based on the positions of the multiple calibration patterns in the first image, controlling the imaging plane of the camera to be parallel to the plane of the first test chart; obtaining a second image captured by the camera of a second test chart, where the second test chart includes multiple calibration patterns; based on the position of the central pixel point of the second image and the positions of the multiple calibration patterns in the second image, controlling the optical axis center of the camera to coincide with the center of the second test chart; where the alignment of the camera and the test chart includes the imaging plane of the camera being parallel to the plane of the first test chart and the optical axis center of the camera coinciding with the center of the second test chart.

[0142] In another aspect, the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the alignment method of the camera and the test chart provided by the above-mentioned various methods. The method includes: obtaining a first image captured by the camera of a first test chart, where the first test chart includes a plurality of calibration patterns; based on the positions of the plurality of calibration patterns in the first image, controlling the imaging plane of the camera to be parallel to the plane of the first test chart; obtaining a second image captured by the camera of a second test chart, where the second test chart includes a plurality of calibration patterns; based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image, controlling the optical axis center of the camera to coincide with the center of the second test chart; where the alignment of the camera and the test chart includes the imaging plane of the camera being parallel to the plane of the first test chart and the optical axis center of the camera coinciding with the center of the second test chart.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0144] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0145] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for aligning a camera with a test chart, characterized in that: include: Acquire a first image obtained by photographing a first test chart with a camera, wherein the first test chart includes a plurality of calibration patterns; Based on the respective positions of the plurality of calibration patterns in the first image, controlling the imaging plane of the camera to be parallel to the plane of the first test chart; Acquire a second image obtained by photographing a second test chart by the camera, wherein the second test chart includes the plurality of calibration patterns; Based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image, controlling the optical axis center of the camera to coincide with the center of the second test chart; The alignment of the camera and the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the center of the optical axis of the camera coincides with the center of the second test chart.

2. The method for aligning a camera with a test chart according to claim 1, characterized in that: The controlling the imaging plane of the camera to be parallel to the plane of the first test chart based on the respective positions of the plurality of calibration patterns in the first image comprises: Determining a tilt angle of the camera in a camera coordinate system based on respective positions of the plurality of calibration figures in the first image; In the case where at least one of the tilt angles is greater than or equal to a preset tilt angle threshold, adjusting the posture of the camera based on the tilt angle, and reacquiring an image of the first test chart photographed by the camera based on the adjusted posture, taking the image as a new first image, and repeatedly performing the above operation until the tilt angles of the camera in the camera coordinate system are all less than the preset tilt angle threshold; When the tilt angles are all smaller than the preset tilt angle threshold, it is determined that the imaging plane of the camera is parallel to the plane of the first test chart.

3. The method for aligning a camera with a test chart according to claim 2, characterized in that: The number of the plurality of calibration figures is three, and determining the tilt angle of the camera in a camera coordinate system based on respective positions of the plurality of calibration figures in the first image includes: Based on the positions of the three calibration figures, a first calibration figure, a second calibration figure and a third calibration figure are determined from the three calibration figures; wherein the straight line where the first calibration figure and the second calibration figure are located is perpendicular to the straight line where the first calibration figure and the third calibration figure are located, and on the first test chart, the vertical coordinates of the first calibration figure and the second calibration figure are the same, and the horizontal coordinates of the first calibration figure and the third calibration figure are the same; Determine the tilt angle of the camera in the vertical direction and the tilt angle in the vertical direction based on the positions of the first calibration figure and the second calibration figure in the camera coordinate system; Determine the tilt angle of the camera in the horizontal direction based on the positions of the first calibration figure and the third calibration figure in the camera coordinate system; The tilt angle of the camera in the camera coordinate system includes the tilt angle of the camera in the horizontal direction, the tilt angle in the vertical direction and the tilt angle in the vertical direction.

4. The method according to claim 3, characterized in that The determining of the first calibration figure, the second calibration figure and the third calibration figure from the three calibration figures based on respective positions of the three calibration figures comprises: Based on the positions of the three calibration figures, construct vectors corresponding to the two calibration figures; Determining the first calibration figure from the three calibration figures based on the inner product of the vectors corresponding to the two calibration figures; The second calibration figure and the third calibration figure are determined from the other two calibration figures based on the outer product between two vectors constructed from the position of the first calibration figure and the positions of the other two calibration figures respectively.

5. The method for aligning a camera with a test chart according to claim 3 or 4, characterized in that: The controlling the optical axis center of the camera to coincide with the center of the second test chart based on the position of the central pixel point of the second image and the respective positions of the plurality of calibration patterns in the second image comprises: Determine the position of the central pixel point of the second test chart based on the positions of the plurality of calibration patterns in the second image; In the case where the difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart is greater than or equal to a preset difference, adjusting the posture of the camera based on the difference, and reacquiring an image captured by the camera on the second test chart based on the adjusted posture, taking the image as a new second image, and repeatedly performing the above operation until the difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart is less than the preset difference; When the difference between the position of the central pixel point of the second image and the position of the central pixel point of the second test chart is less than the preset difference, it is determined that the center of the optical axis of the camera coincides with the center of the second test chart.

6. The method for aligning a camera with a test chart according to claim 5, characterized in that: The determining the position of the central pixel point of the second test chart based on the respective positions of the plurality of calibration patterns in the second image includes: Determine the horizontal coordinate of the central pixel point of the second test chart based on the respective positions of the first calibration pattern and the second calibration pattern; Based on the respective positions of the first calibration pattern and the third calibration pattern, the vertical coordinate of the central pixel point of the second test chart is determined.

7. The method according to claim 3, characterized in that In the case where the camera is a monocular camera, the method further includes: For any calibration figure among the first calibration figure, the second calibration figure and the third calibration figure, determining the depth of the calibration figure based on the pixel coordinates of the calibration figure, the camera coordinate system and the world coordinate system; Based on the pixel coordinates of the calibration figure, the depth of the calibration figure and the intrinsic parameter matrix of the camera, the position of the calibration figure in the camera coordinate system is determined.

8. A camera and test chart alignment device, characterized in that: include: A first acquisition unit, configured to acquire a first image obtained by photographing a first test chart with a camera, wherein the first test chart includes a plurality of calibration patterns; A first control unit, configured to control the imaging plane of the camera to be parallel to the plane of the first test chart based on the positions of the plurality of calibration patterns in the first image; A second acquisition unit, configured to acquire a second image obtained by photographing a second test chart by the camera, wherein the second test chart includes the plurality of calibration patterns; A second control unit, configured to control the center of the optical axis of the camera to coincide with the center of the second test chart based on the position of the central pixel point of the second image and the positions of the plurality of calibration patterns in the second image; The alignment of the camera and the test chart includes that the imaging plane of the camera is parallel to the plane of the first test chart, and the center of the optical axis of the camera coincides with the center of the second test chart.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for aligning a camera and a test chart as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for aligning a camera and a test chart as claimed in any one of claims 1 to 7 is implemented.

Images