Binocular imaging system and control method, device and storage medium thereof

Abstract

The present invention relates to the field of image processing technology, and provides a binocular imaging system and a control method, device and storage medium thereof. The method comprises: establishing a mapping table; controlling a wide-angle camera to perform imaging to obtain a calibration image; determining a pixel point from the calibration image as a calibration point, using the wide-angle image coordinates of the calibration point as an index, retrieving n control parameters from the mapping table, where n is a positive integer greater than 1; performing interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters; and controlling a telephoto camera to perform imaging according to the calibration control parameters. The embodiment of the present invention uses interpolation calculation to reduce the entries in the mapping table, reduce the burden of establishing the mapping table and improve the retrieval speed, and can firstly correct the control parameters in the mapping table to avoid the problem of inaccurate positioning caused by optical path distortion, thereby quickly and accurately driving the telephoto camera to aim at the calibration point.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a binocular imaging system and a control method, device and storage medium thereof. Background Art

[0002] In order to track and image the target of interest in a large scene with high resolution, a binocular imaging system can be formed by using two cameras, where the first camera is a wide-angle camera that images the entire scene to find and locate the target of interest, and the second camera is a telephoto high-resolution camera that aims at the target of interest to take photos or videos driven by the control system to obtain high-resolution photos or videos of the target. Such a design has a cost advantage over using only a single-lens ultra-high-resolution camera that can cover the entire large scene, because the relationship between the cost of the imaging system and the resolution is not linear, and when the resolution reaches a certain level, the cost of the system increases exponentially.

[0003] When the relative positions of the gimbal (the control system of the telephoto camera), the telephoto camera, and the wide-angle camera in the binocular imaging system are fixed, the telephoto camera can be driven to aim at the target by calculating the angle position parameters of the gimbal motor corresponding to the wide-angle image coordinates of the wide-angle camera.

[0004] However, due to the inevitable distortion of the optical path systems of wide-angle cameras and telephoto cameras, and the inevitable errors in the processing and assembly of mechanical parts, under the influence of the superposition of various errors, the gimbal motor angle position parameters calculated by the wide-angle image coordinates of the wide-angle camera are difficult to truly align the imaging center point of the telephoto camera with the position corresponding to the aforementioned wide-angle image coordinates.

[0005] Therefore, a major challenge faced by the binocular imaging system in the prior art is how to quickly and accurately drive the telephoto camera to aim at the target of interest after the target of interest is identified and located by the wide-angle camera. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a binocular imaging system and a control method, device and storage medium thereof to solve the technical problem that it is difficult to quickly and accurately drive a telephoto camera to aim at a target of interest in the prior art.

[0007] In a first aspect, an embodiment of the present invention provides a control method for a binocular imaging system, wherein the binocular imaging system includes a wide-angle camera and a telephoto camera, and the method includes:

[0008] S10: Establishing a mapping table; wherein the index of the mapping table is the wide-angle image coordinate of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera;

[0009] S20: Control the wide-angle camera to perform imaging to obtain a calibration image;

[0010] S30: determining a pixel point from the calibration image as a calibration point, using the wide-angle image coordinates of the calibration point as an index, and retrieving n control parameters from the mapping table, where n is a positive integer greater than 1;

[0011] S40: performing interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters;

[0012] S50: Controlling the telephoto camera to perform imaging according to the calibration control parameters.

[0013] The embodiment of the present invention establishes a mapping table by taking the wide-angle image coordinates of the wide-angle camera as an index and the control parameters of the telephoto camera as a return value. After the wide-angle camera is controlled to perform imaging to obtain a calibration image, n control parameters can be retrieved from the mapping table according to the calibration points in the calibration image. Then, interpolation calculation is performed based on the n control parameters, that is, the calibration control parameters are jointly determined by the n control parameters, so as to drive the telephoto camera to perform imaging. On the one hand, the entries of the mapping table can be reduced. When establishing the mapping table, it is not necessary to establish it for all reachable points of the telephoto camera, which greatly reduces the burden of establishing the mapping table and improves work efficiency. At the same time, due to the reduction of mapping table entries, the retrieval speed can be effectively improved. On the other hand, since each calibration control parameter is obtained from a preset stored mapping table, the calculation problem in the system operation process is avoided, and the work efficiency is improved. Moreover, since the control parameters in the mapping table are all pre-set, the problem of inaccurate positioning caused by optical path distortion can be avoided after pre-correction. Therefore, the embodiment of the present invention can quickly and accurately drive the telephoto camera to aim at the calibration point.

[0014] Preferably, in said S40, it includes:

[0015] S41: Determine a weight coefficient corresponding to each of the n control parameters; wherein the weight coefficient is inversely correlated with a distance of a wide-angle image coordinate corresponding to the weight coefficient relative to a wide-angle image coordinate of a calibration point;

[0016] S42: taking the sum of the products of the n control parameters and their corresponding weight coefficients as the calibration control parameter.

[0017] An embodiment of the present invention provides a distance-based interpolation algorithm, which determines the size of the weight coefficient by determining the distance of the wide-angle image coordinates corresponding to n control parameters (that is, the wide-angle image coordinates corresponding to the weight coefficient) relative to the wide-angle image coordinates of the calibration point, so that the control parameters corresponding to the wide-angle image coordinates closer to the calibration point account for a larger proportion in the calibration control parameters, thereby effectively improving the accuracy of telephoto camera positioning.

[0018] Preferably, in said S10, it includes:

[0019] S11: Control the binocular imaging system to image a calibration card printed with a two-dimensional code matrix to obtain a wide-angle image and a telephoto image; wherein each two-dimensional code encodes its own row and column coordinates in the two-dimensional code matrix;

[0020] S12: determining a correspondence between wide-angle image coordinates and row and column coordinates according to the QR code in the wide-angle image, and determining a correspondence between telephoto image coordinates and row and column coordinates according to the QR code in the telephoto image;

[0021] S13: determining the row and column coordinates corresponding to the central pixel of the telephoto image according to the correspondence between the telephoto image coordinates and the row and column coordinates;

[0022] S14: determining the wide-angle image coordinates corresponding to the central pixel of the telephoto image according to the row and column coordinates corresponding to the central pixel of the telephoto image and the correspondence between the wide-angle image coordinates and the row and column coordinates;

[0023] S15: Establishing the mapping table by taking the wide-angle image coordinates corresponding to the central pixel of the telephoto image as an index and the control parameters of the telephoto camera during imaging as a return value.

[0024] The embodiment of the present invention obtains a wide-angle image and a telephoto image by controlling the binocular imaging system to image a calibration card printed with a QR code matrix. Then, based on the QR code on the image, the correspondence between the wide-angle image coordinates and the row and column coordinates, and the correspondence between the telephoto image coordinates and the row and column coordinates are determined. Then, based on the aforementioned correspondence, the center pixel of the telephoto image (the imaging center point of the telephoto camera) and the wide-angle image coordinates can be unified into the same coordinate system (row and column coordinate system), thereby determining the control parameters corresponding to the wide-angle image coordinates, thereby eliminating the effects of optical distortion and structural component production and assembly errors, and effectively improving the accuracy of telephoto camera positioning.

[0025] Preferably, in S12, determining the correspondence between wide-angle image coordinates and row and column coordinates according to the two-dimensional code in the wide-angle image includes:

[0026] S121: Determine the row and column coordinates corresponding to each QR code based on the wide-angle image;

[0027] S122: determining a correspondence between wide-angle image coordinates and row and column coordinates according to the wide-angle image coordinates of the central pixel of the two-dimensional code in the wide-angle image and the row and column coordinates corresponding to the two-dimensional code;

[0028] Determining the correspondence between the telephoto image coordinates and the row and column coordinates according to the two-dimensional code in the telephoto image includes:

[0029] S123: Determine the row and column coordinates corresponding to each QR code based on the telephoto image;

[0030] S124: Determine a correspondence between the telephoto image coordinates and the row and column coordinates according to the telephoto image coordinates of the central pixel of the two-dimensional code in the telephoto image and the row and column coordinates corresponding to the two-dimensional code.

[0031] When establishing the correspondence between wide-angle image coordinates and row and column coordinates, the embodiment of the present invention uses the row and column coordinates corresponding to the QR code and the wide-angle image coordinates of the central pixel of the QR code to determine the correspondence, and the established correspondence is accurate, and the image offset and distortion are small when the coordinate system is transformed. Similarly, using the row and column coordinates corresponding to the QR code and the telephoto image coordinates of the central pixel of the QR code to establish the correspondence between telephoto image coordinates and row and column coordinates also has the aforementioned advantages.

[0032] Preferably, in said S121, it includes:

[0033] S1211: converting the wide-angle image into a grayscale image and performing binarization processing;

[0034] S1212: Performing connected domain analysis on the wide-angle image that has been binarized to obtain multiple color blocks;

[0035] S1213: Eliminate color blocks whose areas do not meet the preset standards;

[0036] S1214: Determine the color block whose center of gravity is closest to the central pixel of the wide-angle image as the seed color block, and set its row and column coordinates to be the same as the row and column coordinates encoded by the QR code at the center of the QR code matrix;

[0037] S1215: Based on the seed color block, corresponding row and column coordinates are set for each color block to determine the row and column coordinates corresponding to each two-dimensional code.

[0038] Due to the low imaging accuracy of wide-angle image coordinates, it may not be possible to obtain the row and column coordinates encoded by each QR code by parsing the QR code after imaging. Therefore, the embodiment of the present invention processes the QR code in the wide-angle image into a color block through connected domain analysis, and then screens the color blocks according to a preset standard, and determines the color block whose center of gravity is closest to the central pixel of the wide-angle image as the seed color block, and then sets the corresponding row and column coordinates for each color block based on the seed color block to determine the row and column coordinates corresponding to each QR code, thereby solving the technical problem that the row and column coordinates encoded by the QR code cannot be obtained due to the low imaging accuracy of the wide-angle camera and the inability to parse the QR code.

[0039] Preferably, in S1215, the following steps are included:

[0040] S12151: Using the ordinate of the centroid coordinate of the seed color block as a reference, searching for the next color block to the left / right, and each time a new color block is found, subtracting 1 or adding 1 to the row and column coordinates of the previous color block, and using the row and column coordinates of the new color block as a reference, and then continuing to search for the next color block to the left / right, until the wide-angle image boundary is reached;

[0041] S12152: For the color block with determined row coordinates, use the horizontal coordinates of its own centroid coordinates as a reference to search for the next color block upward / downward. Each time a new color block is found, the row coordinates of the row and column coordinates of the previous color block are reduced by 1 or increased by 1 accordingly, and used as the row and column coordinates of the new color block. Then, use the horizontal coordinates of the centroid coordinates of the new color block as a reference to continue searching for the next color block upward / downward until the wide-angle image boundary is reached.

[0042] In the embodiment of the present invention, the vertical coordinate of the centroid coordinate of the seed color block is used as a reference to search for the next color block to the left / right. After each new color block is found, the column coordinate of the row and column coordinate of the previous color block is reduced by 1 or increased by 1 accordingly, which is used as the row and column coordinate of the new color block. Then, the vertical coordinate of the centroid coordinate of the new color block is used as a reference to continue to search for the next color block to the left / right, instead of always using the vertical coordinate of the centroid coordinate of the seed color block as a reference to search for all color blocks in the same row. Therefore, since the vertical coordinate of the centroid coordinate of the color blocks in the adjacent columns of the same row of color blocks does not change much, even if the calibration card is slightly tilted and / or rotated relative to the wide-angle camera, and the wide-angle camera lens itself is slightly distorted, it is always possible to reliably find the next nearest color block in the horizontal direction from one color block. Similarly, since the horizontal coordinate of the centroid coordinate of the color blocks in the adjacent rows of the same column of color blocks does not change much, even if the calibration card is slightly tilted and / or rotated relative to the wide-angle camera, and the wide-angle camera lens itself is slightly distorted, it is always possible to reliably find the next nearest color block in the vertical direction from one color block.

[0043] Preferably, the two-dimensional code matrix includes a reference row and a reference column, and the color of the two-dimensional code in the reference row and the reference column is different from other two-dimensional codes in the two-dimensional code matrix. After S1215, the method further includes:

[0044] S1216: For color blocks with the same row coordinates, calculate the color average of the color blocks in the row, and for color blocks with the same column coordinates, calculate the color average of the color blocks in the column;

[0045] S1217: Determine a reference row and a reference column based on the color average value;

[0046] S1218: Determine a row coordinate correction value and a column coordinate correction value according to the difference between the row and column coordinates pre-encoded by the reference row and the reference column and the row and column coordinates set when searching through the seed color block;

[0047] S1219: Correct the row and column coordinates corresponding to each color block according to the row coordinate correction value and the column coordinate correction value.

[0048] Since the imaging center of the wide-angle camera is not necessarily aligned with the center of the calibration card in some cases, if the row and column coordinates set by the color block are not corrected, it may lead to inconsistency with the row and column coordinates obtained by parsing in the telephoto image. Therefore, the embodiment of the present invention sets the color of the QR code of the reference row and reference column in the calibration card to be different from other QR codes. Therefore, by calculating the color mean of each row and column of color blocks, it is possible to determine which row and column of color blocks correspond to the reference row and reference column, thereby determining the correction value of the row and column coordinates, and thereby correcting the row and column coordinates corresponding to each color block, so that the application range of the embodiment of the present invention is wider, and there is no need to ensure that the imaging center of the wide-angle camera is aligned with the center position of the calibration card.

[0049] Preferably, the correspondence between the wide-angle image coordinates and the row and column coordinates is an affine transformation matrix or a perspective transformation matrix; the correspondence between the telephoto image coordinates and the row and column coordinates is an affine transformation matrix or a perspective transformation matrix.

[0050] In a second aspect, an embodiment of the present invention provides a control device for a binocular imaging system, wherein the binocular imaging system includes a wide-angle camera and a telephoto camera, and the device includes:

[0051] A mapping establishment module, used to establish a mapping table; wherein the index of the mapping table is the wide-angle image coordinates of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera;

[0052] A wide-angle imaging control module, used to control the wide-angle camera to perform imaging to obtain a calibration image;

[0053] A retrieval module, used to determine a pixel point from the calibration image as a calibration point, and retrieve n control parameters from the mapping table using the wide-angle image coordinates of the calibration point as an index, where n is a positive integer greater than 1;

[0054] An interpolation calculation module, used to perform interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters;

[0055] A telephoto imaging control module is used to control the telephoto camera to perform imaging according to the calibration control parameters.

[0056] In a third aspect, an embodiment of the present invention provides a binocular imaging system, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0057] In a fourth aspect, an embodiment of the present invention provides a storage medium having computer program instructions stored thereon, which implements the method of the first aspect in the above-mentioned embodiment when the computer program instructions are executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0059] Figure 1 Schematic diagram of a binocular imaging system provided by an embodiment of the present invention.

[0060] Figure 2 It is a flow chart of a control method of a binocular imaging system provided by an embodiment of the present invention.

[0061] Figure 3 It is a flowchart of an interpolation calculation method provided by an embodiment of the present invention.

[0062] Figure 4 It is a schematic diagram of an interpolation calculation provided by an embodiment of the present invention.

[0063] Figure 5 It is a flowchart of a method for establishing a mapping table provided by an embodiment of the present invention.

[0064] Figure 6 It is a schematic diagram of a calibration card provided in an embodiment of the present invention.

[0065] Figure 7 is a schematic diagram of a wide-angle image provided by an embodiment of the present invention.

[0066] Figure 8 It is a flow chart of a method for determining row and column coordinates corresponding to a two-dimensional code provided by an embodiment of the present invention.

[0067] Fig. 9 is a schematic diagram of another wide-angle image provided by an embodiment of the present invention.

[0068] Fig.10 It is a flow chart of a method for setting row and column coordinates based on a seed color block provided by an embodiment of the present invention.

[0069] Fig.11 It is a flowchart of a method for correcting row and column coordinates provided by an embodiment of the present invention.

[0070] Fig.12It is a structural schematic diagram of a control device of a binocular imaging system provided by an embodiment of the present invention.

[0071] Fig.13 Schematic diagram of a binocular imaging system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0073] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. It should be particularly noted that the row coordinates and column coordinates and the horizontal coordinates and vertical coordinates referred to in the present invention are an exemplary description, and those skilled in the art can obtain the technical solution of the other based on the technical solution disclosed by one of the row coordinates and column coordinates or based on the technical solution disclosed by one of the horizontal coordinates and vertical coordinates through simple reasoning and substitution.

[0074] As mentioned above, in order to achieve tracking and high-resolution imaging of targets of interest within a large scene range, a binocular imaging system can be formed using two cameras, one of which is a wide-angle camera and the other is a telephoto camera.

[0075] For easier understanding, see Figure 1, is a schematic diagram of a binocular imaging system provided by an embodiment of the present invention. The binocular imaging system includes a wide-angle camera 10, a telephoto camera 20 and a control system 30. The control system 30 is used to control the movement of the telephoto camera 20, or to control the movement of the wide-angle camera 10 and the telephoto camera 20. The control system 30 can select a motor, a gimbal, a mechanical arm, a mechanical axis, etc., which is not specifically limited in the present invention. In a preferred embodiment of the present invention, a gimbal motor is used as a control system, and the gimbal motor drives the telephoto camera 20 to move through two angle position parameters.

[0076] In the binocular imaging system, the most preferred arrangement of the wide-angle camera and the telephoto camera is as follows: the imaging surface of the wide-angle camera is parallel to the target plane, the long side of the pixel matrix of the image sensor of the wide-angle camera is parallel to the long side of the rectangle of the target plane (the short side of the pixel matrix of the image sensor is also parallel to the short side of the rectangle of the target plane), and the center of the pixel matrix of the image sensor of the wide-angle camera is aimed at the center of the rectangle of the target plane; when the telephoto camera is at zero position, the long side of the pixel matrix of its image sensor is parallel to the long side of the rectangle of the target plane, and the short side of the pixel matrix of its image sensor is parallel to the short side of the rectangle of the target plane.

[0077] Theoretically, when the relative positions of the gimbal motor, telephoto camera, and wide-angle camera in the binocular imaging system are fixed, the gimbal motor angle position parameter A(α, β) corresponding to the wide-angle image coordinates of the wide-angle camera can be calculated, where α is the angle position of the gimbal first-axis motor, and β is the angle position of the gimbal second-axis motor. The motor angle position parameter A(α, β) can drive the telephoto camera to aim at the target. However, due to the inevitable distortion of the optical path system of the wide-angle camera and the telephoto camera, and the inevitable errors in the processing and assembly of mechanical parts, under the influence of the superposition of various errors, the gimbal motor angle position parameter calculated by the wide-angle image coordinates of the wide-angle camera is difficult to truly align the imaging center point of the telephoto camera with the position corresponding to the aforementioned wide-angle image coordinates.

[0078] In view of this, an embodiment of the present invention provides a binocular imaging system and a control method, device and storage medium thereof to solve the technical problem that it is difficult to quickly and accurately drive a telephoto camera to aim at a target of interest in the prior art.

[0079] See also Figure 2 , is a flow chart of a control method of a binocular imaging system provided by an embodiment of the present invention, the method comprising:

[0080] S10: Establishing a mapping table; wherein the index of the mapping table is the wide-angle image coordinate of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera;

[0081] S20: Control the wide-angle camera to perform imaging to obtain a calibration image;

[0082] S30: determining a pixel point from the calibration image as a calibration point, using the wide-angle image coordinates of the calibration point as an index, and retrieving n control parameters from the mapping table, where n is a positive integer greater than 1;

[0083] S40: performing interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters;

[0084] S50: Controlling the telephoto camera to perform imaging according to the calibration control parameters.

[0085] Specifically, the wide-angle image coordinates of the wide-angle camera are used as indexes, and the control parameters of the telephoto camera are used as return values ​​to establish a mapping table. The mapping table can be stored in a non-volatile memory and can be used repeatedly after being established only once. A LUT (Look-Up Table) can also be used to establish the mapping table, which is more suitable for circuits with higher integration.

[0086] On the premise of establishing the mapping table, first control the wide-angle camera to perform imaging to obtain a calibration image. In the calibration image, a pixel point can be determined as a calibration point. Specifically, the calibration point can be selected manually, or one of the pixels of a specific image can be determined as a calibration point from the calibration image based on machine vision or other recognition methods. Here, the coordinates of the calibration point are recorded as P(x, y).

[0087] Then, the mapping table can be searched based on the coordinates P (x, y) of the calibration point to determine n control parameters. Wherein n is a positive integer greater than 1. In one embodiment of the present invention, the value of n can be preset. When searching, the n wide-angle image coordinates closest to the calibration point are found from the index of the mapping table (i.e., the wide-angle image coordinates), thereby determining the n control parameters. In another embodiment of the present invention, a distance threshold can also be set to retrieve the wide-angle image coordinates whose distance from the coordinates P (x, y) of the calibration point is less than the threshold, thereby determining the control parameters corresponding to them. The aforementioned distance can be defined based on the x-coordinate and the y-coordinate. For example, if the wide-angle image coordinates are (x0, y0), the distance between them and the calibration point can be defined as:

[0088] After retrieving n control parameters, interpolation calculation can be performed to obtain calibration control parameters. Interpolation calculation can use a variety of algorithms, which are not specifically limited in the present invention. Figure 3 , is a preferred implementation of the present invention, in which S40 includes:

[0089] S41: Determine a weight coefficient corresponding to each of the n control parameters; wherein the weight coefficient is inversely correlated with a distance of a wide-angle image coordinate corresponding to the weight coefficient relative to a wide-angle image coordinate of a calibration point;

[0090] S42: taking the sum of the products of the n control parameters and their corresponding weight coefficients as the calibration control parameter.

[0091] Specifically, after determining n control parameters using the aforementioned method, first determine their corresponding weight coefficients. For each weight coefficient, the larger the distance between the wide-angle image coordinates corresponding to it and the wide-angle image coordinates of the calibration point, the smaller the weight coefficient. For ease of description, the n control parameters are denoted as A1~A n , their corresponding weight coefficients are K1~K n , then the calibration control parameter is equal to:

[0092] The present invention also provides a more preferred specific algorithm. For ease of understanding, please refer to Figure 4 According to the wide-angle image coordinates (x, y) of the calibration point, a total of 4 wide-angle image coordinates are retrieved from the mapping table, namely (x a ,y a ), (x b ,y b ), (x c ,y c ), (x d ,y d ), these four wide-angle image coordinates correspond to the control parameters (α a ,β a ), (α b ,β b ), (α c ,β c ),(α d ,β d ). First, calculate the distance between the four wide-angle image coordinates and the wide-angle image coordinates of the calibration point, and get:

[0093]

[0094]

[0095]

[0096]

[0097] Then, calculate the weight coefficient corresponding to each control parameter:

[0098]

[0099] Where sum = 1 / L a +1 / L b +1 / L c +1 / L d .

[0100] Based on the above algorithm, the calibration control parameters are:

[0101] α=α a *K a +α b *K b +α c *K c +α d *K d ;

[0102] β=β a *K a +β b *K b +β c *K c +β d *K d .

[0103] Therefore, after obtaining the calibration control parameters, the telephoto camera can be driven to perform imaging, thereby achieving high-resolution shooting of the calibration points. The interpolation calculation method of the embodiment of the present invention can effectively reduce the entries of the mapping table. When establishing the mapping table, it is not necessary to establish it for all the reachable points of the telephoto camera, which greatly reduces the burden of establishing the mapping table and improves work efficiency. At the same time, due to the reduction of mapping table entries, the retrieval speed can be effectively improved. In particular, in the LUT implementation method, the number of multiplexers (MUX) can be greatly reduced, the complexity of the integrated circuit can be reduced, and since the control parameters in the mapping table are all pre-set, the problem of inaccurate positioning caused by optical path distortion can be avoided after pre-correction. The embodiment of the present invention can be applied to the classroom, and the wide-angle image obtained by the wide-angle camera includes multiple desks, and then the documents (such as test papers) placed on the desks are obtained based on the wide-angle image recognition, and then one of the pixels (preferably the center pixel) of each document is used as the calibration point one by one, and multiple control parameters are found from the mapping table. The calibration control parameters are obtained after interpolation calculation, and the telephoto camera is driven to shoot each document one by one, thereby obtaining a high-definition image of the document.

[0104] See also Figure 5 , is a flow chart of a method for establishing a mapping table provided by an embodiment of the present invention, wherein S10 includes:

[0105] S11: Control the binocular imaging system to image a calibration card printed with a two-dimensional code matrix to obtain a wide-angle image and a telephoto image; wherein each two-dimensional code encodes its own row and column coordinates in the two-dimensional code matrix;

[0106] S12: determining a correspondence between wide-angle image coordinates and row and column coordinates according to the QR code in the wide-angle image, and determining a correspondence between telephoto image coordinates and row and column coordinates according to the QR code in the telephoto image;

[0107] S13: determining the row and column coordinates corresponding to the central pixel of the telephoto image according to the correspondence between the telephoto image coordinates and the row and column coordinates;

[0108] S14: determining the wide-angle image coordinates corresponding to the central pixel of the telephoto image according to the row and column coordinates corresponding to the central pixel of the telephoto image and the correspondence between the wide-angle image coordinates and the row and column coordinates;

[0109] S15: Establishing the mapping table by taking the wide-angle image coordinates corresponding to the central pixel of the telephoto image as an index and the control parameters of the telephoto camera during imaging as a return value.

[0110] For easier understanding, see Figure 6 The calibration card shown in FIG. 1 has a QR code matrix printed thereon, and each QR code encodes its own row and column coordinates in the QR code matrix. The type of the QR code is not specifically limited, and may be, for example, QR Code, Data Matrix, Maxi Code, etc. In one embodiment of the present invention, the QR code encoding the center of the QR code matrix (i.e., Figure 6 The row and column coordinates of the QR code at the 3rd row and 5th column shown in the figure are (0, 0), and then the row and column coordinates of other QR codes are encoded based on this QR code. Figure 6 The row and column coordinates of the QR code in the 3rd row and 4th column shown in FIG. 1 are (0, -1), and the QR code adjacent to the right of the QR code (i.e. Figure 6 The row and column coordinates of the QR code in the 3rd row and 6th column shown in the figure are (0, 1), encoding the QR code adjacent to it below (i.e. Figure 6 The row and column coordinates of the QR code in the 4th row and 5th column shown in FIG. 1 are (1, 0), and the QR code adjacent to it (i.e. Figure 6 The row and column coordinates of the QR code at the 2nd row and 5th column shown are (-1, 0), and so on.

[0111] Therefore, the row and column coordinates of the QR code can be determined in the telephoto image or wide-angle image obtained by imaging the calibration card. Then, the correspondence between the wide-angle image coordinates and the row and column coordinates, and the correspondence between the telephoto image coordinates and the row and column coordinates can be determined based on the QR code.

[0112] Specifically, determining the correspondence between the wide-angle image coordinates and the row and column coordinates according to the two-dimensional code in the wide-angle image includes:

[0113] S121: Determine the row and column coordinates corresponding to each QR code based on the wide-angle image;

[0114] S122: Determine a correspondence between wide-angle image coordinates and row and column coordinates according to the wide-angle image coordinates of the central pixel of the two-dimensional code in the wide-angle image and the row and column coordinates corresponding to the two-dimensional code.

[0115] The correspondence between the wide-angle image coordinates and the row and column coordinates is an affine transformation matrix or a perspective transformation matrix.

[0116] Specifically, determining the correspondence between the telephoto image coordinates and the row and column coordinates according to the QR code in the telephoto image includes:

[0117] S123: Determine the row and column coordinates corresponding to each QR code based on the telephoto image;

[0118] S124: Determine a correspondence between the telephoto image coordinates and the row and column coordinates according to the telephoto image coordinates of the central pixel of the two-dimensional code in the telephoto image and the row and column coordinates corresponding to the two-dimensional code.

[0119] The correspondence between the telephoto image coordinates and the row and column coordinates is an affine transformation matrix or a perspective transformation matrix.

[0120] Since the principles of establishing the correspondence between wide-angle image coordinates and row and column coordinates and establishing the correspondence between telephoto image coordinates and row and column coordinates are similar, the correspondence between wide-angle image coordinates and row and column coordinates is now taken as an example for explanation.

[0121] First, based on the wide-angle image, the row and column coordinates encoded by the two-dimensional code can be obtained by parsing the two-dimensional code. At the same time, the wide-angle image coordinates of any pixel point in the two-dimensional code are used as the corresponding coordinates to determine the m groups of corresponding coordinates. Specifically, when determining the m groups of corresponding coordinates, the wide-angle image coordinates corresponding to the row and column coordinates are preferably the wide-angle image coordinates of the central pixel point of the two-dimensional code. If the center of the two-dimensional code is between multiple pixels, the wide-angle image coordinates corresponding to the row and column coordinates are preferably the average coordinates of the multiple pixels, or the coordinates of one of the pixels are selected as the wide-angle image coordinates corresponding to the row and column coordinates.

[0122] If an affine transformation matrix is ​​to be established, m is a positive integer greater than or equal to 3, and if a perspective transformation matrix is ​​to be established, m is a positive integer greater than or equal to 4. Here, an affine transformation matrix is ​​used for illustration. When establishing an affine transformation matrix, the formula may be used:

[0123]

[0124] is equivalent to:

[0125]

[0126] Among them, x represents the horizontal coordinate of the wide-angle image coordinate, y represents the vertical coordinate of the wide-angle image coordinate, u represents the row coordinate of the row-column coordinate, and v represents the vertical coordinate of the row-column coordinate. According to the above formula, when establishing the affine transformation matrix, 6 unknowns need to be solved, and the solution can be obtained by substituting 3 sets of corresponding wide-angle image coordinates and row-column coordinates. Based on a similar process, the corresponding relationship between the telephoto image coordinates and the row-column coordinates can be established, which will not be repeated here.

[0127] After establishing the correspondence between the wide-angle image coordinates and the row and column coordinates, and the correspondence between the telephoto image coordinates and the row and column coordinates, the row and column coordinates corresponding to the central pixel of the telephoto image can be determined according to the correspondence between the telephoto image coordinates and the row and column coordinates. Specifically, the row and column coordinates corresponding to the central pixel of the telephoto image can be obtained by substituting the telephoto image coordinates of the central pixel of the telephoto image into the affine transformation matrix or perspective transformation matrix between the telephoto image coordinates and the row and column coordinates.

[0128] Furthermore, the row and column coordinates corresponding to the central pixel of the telephoto image are substituted into the affine transformation matrix or perspective transformation matrix between the wide-angle image coordinates and the row and column coordinates, so as to determine the wide-angle image coordinates corresponding to the central pixel of the telephoto image.

[0129] At this time, the control parameters of the telephoto camera when imaging are obtained as the return value in the mapping table, and the wide-angle image coordinates corresponding to the central pixel of the telephoto image are used as the index of the return value, so as to establish a pair of mapping relationships in the mapping table. If the next set of mapping relationships is to be established, the control parameters are changed and the above process is repeated.

[0130] In some scenes, due to the limited resolution capability of the wide-angle camera, the wide-angle image captured is Figure 7 As described above, the QR code cannot be parsed, resulting in the inability to obtain the row and column coordinates corresponding to the QR code (there is no need to worry about this problem for telephoto images). Therefore, an embodiment of the present invention also provides a method for obtaining the row and column coordinates corresponding to the QR code in a wide-angle image, see Figure 8 , in said S121, including:

[0131] S1211: converting the wide-angle image into a grayscale image and performing binarization processing;

[0132] S1212: Performing connected domain analysis on the wide-angle image that has been binarized to obtain multiple color blocks;

[0133] S1213: Eliminate color blocks whose areas do not meet the preset standards;

[0134] S1214: Determine the color block whose center of gravity is closest to the central pixel of the wide-angle image as the seed color block, and set its row and column coordinates to be the same as the row and column coordinates encoded by the QR code at the center of the QR code matrix;

[0135] S1215: Based on the seed color block, corresponding row and column coordinates are set for each color block to determine the row and column coordinates corresponding to each two-dimensional code.

[0136] Specifically, the following Figure 7 The wide-angle image shown is converted into a grayscale image, and then binarized and connected domain analyzed to obtain a wide-angle image including multiple color blocks. Then, the image with an area smaller than A is removed. min or area greater than A max The color block is obtained as Fig. 9 The wide-angle image shown. min and A max is the default value.

[0137] Based on the wide-angle image shown in Figure 9, the center of gravity of each color block is calculated, and the center of gravity of the color block is the mean of the x coordinates of all pixels of the color block and the mean of the y coordinates of all pixels. Then, find the color block whose center of gravity is closest to the central pixel of the wide-angle image, use it as the seed color block (i.e., the color block in the 3rd row and the 5th column), and set its row and column coordinates to be the same as the row and column coordinates encoded by the two-dimensional code at the center of the two-dimensional code matrix. For example, if the row and column coordinates encoded by the two-dimensional code at the center of the two-dimensional code matrix are (0, 0), then the row and column coordinates of the seed color block are set to (0, 0), and based on the seed color block, the corresponding row and column coordinates are set for each color block to determine the row and column coordinates corresponding to each two-dimensional code. Specifically, if the row and column coordinates of the seed color block are set to (0, 0), the row and column coordinates of the color block adjacent to its left (i.e. the color block in the 3rd row and 4th column) are set to (0, -1), the row and column coordinates of the color block adjacent to its right (i.e. the color block in the 3rd row and 6th column) are set to (0, 1), the row and column coordinates of the color block adjacent to its bottom (i.e. the color block in the 4th row and 5th column) are set to (1, 0), the row and column coordinates of the color block adjacent to its top (i.e. the color block in the 2nd row and 5th column) are set to (-1, 0), and so on.

[0138] In one embodiment of the present invention, the grayscale image may be filtered first and then binarized, preferably by low-pass filtering.

[0139] In one embodiment of the present invention, a morphological erosion operation (erode) may be performed on the binary image first, and then a connected domain analysis may be performed, so that thin and small-area interference targets may be eliminated without substantially affecting the center of gravity of the color block.

[0140] The embodiment of the present invention also provides a more accurate method for finding adjacent color blocks, see Fig.10, in S1215, including:

[0141] S12151: Using the ordinate of the centroid coordinate of the seed color block as a reference, searching for the next color block to the left / right, and each time a new color block is found, subtracting 1 or adding 1 to the row and column coordinates of the previous color block, and using the row and column coordinates of the new color block as a reference, and then continuing to search for the next color block to the left / right, until the wide-angle image boundary is reached;

[0142] S12152: For the color block with determined row coordinates, use the horizontal coordinates of its own centroid coordinates as a reference to search for the next color block upward / downward. Each time a new color block is found, the row coordinates of the row and column coordinates of the previous color block are reduced by 1 or increased by 1 accordingly, and used as the row and column coordinates of the new color block. Then, use the horizontal coordinates of the centroid coordinates of the new color block as a reference to continue searching for the next color block upward / downward until the wide-angle image boundary is reached.

[0143] Specifically, the scenario in which the row and column coordinates of the seed color block are (0, 0) is described, and the centroid coordinates of the seed color block are recorded as (x e ,y e ).

[0144] First, find all the color blocks whose row coordinates are 0: e Find the nearest color block on the horizontal line to the left, and its centroid coordinate is (x f ,y f ), set its row and column coordinates to (0, -1), and then at y = y f Continue to search for the next nearest color block to the left on the horizontal line, and recursively perform this process until the left edge of the image is reached, thereby finding all the color blocks in the left half that belong to the same row as the seed color block. Then, use the same method to find all the color blocks in the right half that belong to the same row as the seed color block, and thus obtain all the color blocks with row coordinates of 0. Since the y coordinates of the centroid coordinates of adjacent column color blocks in the same row of color blocks do not change much, even if the calibration card is slightly tilted and / or rotated relative to the wide-angle camera, and the wide-angle camera lens itself is slightly distorted, it is always possible to reliably find the next nearest color block in the horizontal direction from one color block.

[0145] Then, take each color block with row coordinate 0 (i.e., the color block with row and column coordinates determined in the previous step) as the starting color block, and find all the color blocks in the same column. Define the centroid coordinates of the starting color block (0, C) as (x g ,y g ), when x=x g Find the nearest color block on the vertical line upward, and its centroid coordinate is (x h ,y h ), set its row and column coordinates to (-1, C), and then at x = xh Continue to search for the nearest color block upward on the vertical line, and recursively perform this process until the upper boundary of the image is reached, thereby finding all the color blocks in the upper half that belong to the same column as the starting color block. Then, use the same method to find all the color blocks in the lower half that belong to the same column as the starting color block, and thus obtain all the color blocks with column coordinates C. Since the x-coordinates of the centroid coordinates of adjacent rows of color blocks in the same column of color blocks do not change much, even if the calibration card is slightly tilted and / or rotated relative to the wide-angle camera, and the wide-angle camera lens itself is slightly distorted, it is always possible to reliably find the next nearest color block in the vertical direction from one color block.

[0146] In some application scenarios, since the center of the wide-angle image may not correspond to the center of the calibration card, in these cases, if the wide-angle image uses the aforementioned method to set the row and column coordinates, and the telephoto image directly parses the QR code to obtain the row and column coordinates, the row and column coordinates will not match. Therefore, the embodiment of the present invention also provides a method for correcting the row and column coordinates, see Fig.11 After S1215, the method further includes:

[0147] S1216: For color blocks with the same row coordinates, calculate the color average of the color blocks in the row, and for color blocks with the same column coordinates, calculate the color average of the color blocks in the column;

[0148] S1217: Determine a reference row and a reference column based on the color average value;

[0149] S1218: Determine a row coordinate correction value and a column coordinate correction value according to the difference between the row and column coordinates pre-encoded by the reference row and the reference column and the row and column coordinates set when searching through the seed color block;

[0150] S1219: Correct the row and column coordinates corresponding to each color block according to the row coordinate correction value and the column coordinate correction value.

[0151] Specifically, the two-dimensional code matrix includes a reference row and a reference column, and the color of the two-dimensional code in the reference row and the reference column is different from other two-dimensional codes in the two-dimensional code matrix. Figure 6 As shown, the reference row is the middle row of the two-dimensional code matrix, and the reference column is the center column of the two-dimensional code matrix. It is obvious that the reference row can be set to any row of the two-dimensional code matrix, and the reference column can be set to any column of the two-dimensional code matrix. Alternatively, the two diagonals of the two-dimensional code matrix are used as the reference row and the reference column.

[0152] For the color blocks with the same row coordinates, the color average value of the color block in the row is calculated. For example, if the RGB color mode is used for calculation, the color values ​​of the R channel, G channel and B channel of each pixel in the color block are first determined, and then the color average value of the R channel, the color average value of the G channel and the color average value of the B channel are determined respectively in a manner that each channel is calculated independently. Since the color of the reference row is known in advance, it is possible to determine which row of color blocks corresponds to the reference row based on the color average value of each row. Similarly, it is possible to determine which column of color blocks corresponds to the reference column. It should be noted that the present invention is not limited to the color mode, and can also be a color mode such as CMYK or LAB.

[0153] For the reference row, the set row and column coordinates are known, and the encoded row and column coordinates are known. Therefore, the difference between them can be calculated. If the row coordinate set by the reference row is S, and the row coordinate encoded in the two-dimensional code matrix is ​​0, then the difference between them can be determined to be S, which is used as the row coordinate correction value. The row coordinate of each color block is subtracted from S to obtain the final row coordinate of each color block.

[0154] Similarly, for the reference column, the set row and column coordinates are known, and the encoded row and column coordinates are known. Therefore, the difference between them can be calculated. If the column coordinate set by the reference column is D, and its column coordinate encoded in the two-dimensional code matrix is ​​0, then the difference between them can be determined to be D, which is used as the column coordinate correction value. The column coordinate of each color block is subtracted from D to obtain the final column coordinate of each color block.

[0155] Therefore, the embodiments of the present invention can be applied to more scenarios without ensuring that the imaging center of the wide-angle camera is aligned with the calibration card.

[0156] Based on the combination Figures 7 to 11 The described technical solution can determine the row and column coordinates corresponding to each color block when the wide-angle image cannot resolve the QR code, and use the row and column coordinates of the color block as the row and column coordinates of the corresponding QR code, thereby establishing a correspondence between the wide-angle image coordinates and the row and column coordinates based on the wide-angle image. Based on the technical solution disclosed above, those skilled in the art can easily and simply infer another technical solution: arrange the QR codes on the calibration card in other ways, such as arranging multiple groups of QR codes in a ring, and the center points of each group of QR codes arranged in a ring form a circle, and these circles form concentric circles. At the same time, a polar coordinate system is used when encoding the QR code. Obviously, there is no essential difference between the technical solution disclosed in the embodiment of the present invention and the method of changing the arrangement of the QR code on the calibration card, and it is regarded as a simple transformation of the present invention.

[0157] See also Fig.12 The embodiment of the present invention further provides a control device for a binocular imaging system, the device comprising:

[0158] A mapping establishment module, used to establish a mapping table; wherein the index of the mapping table is the wide-angle image coordinates of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera;

[0159] A wide-angle imaging control module, used to control the wide-angle camera to perform imaging to obtain a calibration image;

[0160] A retrieval module, used to determine a pixel point from the calibration image as a calibration point, and retrieve n control parameters from the mapping table using the wide-angle image coordinates of the calibration point as an index, where n is a positive integer greater than 1;

[0161] An interpolation calculation module, used to perform interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters;

[0162] A telephoto imaging control module is used to control the telephoto camera to perform imaging according to the calibration control parameters.

[0163] In addition, combined Figure 2 The control method of the binocular imaging system according to the embodiment of the present invention described above can be implemented by the binocular imaging system. Fig.13 A schematic diagram of the hardware structure of a binocular imaging system provided by an embodiment of the present invention is shown.

[0164] The binocular imaging system may include a processor and a memory storing computer program instructions.

[0165] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0166] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0167] The processor implements any one of the control methods of the binocular imaging system in the above embodiments by reading and executing computer program instructions stored in the memory.

[0168] In one example, the binocular imaging system may further include a communication interface and a bus. Fig.13 As shown in FIG. 1 , the processor, memory, and communication interface are connected through a bus and communicate with each other.

[0169] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0170] The bus includes hardware, software or both, and the components of the binocular imaging system are coupled to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-end bus (FSB), a hypertransport (HT) interconnection, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnection, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. In suitable cases, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnection.

[0171] In addition, in combination with the control method of the binocular imaging system in the above embodiments, the embodiment of the present invention can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the control methods of the binocular imaging system in the above embodiments is implemented.

[0172] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0173] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0174] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0175] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A control method for a binocular imaging system, wherein the binocular imaging system comprises a wide-angle camera and a telephoto camera, characterized in that: The method comprises: S10: Establishing a mapping table; wherein the index of the mapping table is the wide-angle image coordinate of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera; S20: Control the wide-angle camera to perform imaging to obtain a calibration image; S30: determining a pixel point from the calibration image as a calibration point, using the wide-angle image coordinates of the calibration point as an index, and retrieving n control parameters from the mapping table, where n is a positive integer greater than 1; S40: performing interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters; S50: Controlling the telephoto camera to perform imaging according to the calibration control parameters; In the S10, it includes: S11: Control the binocular imaging system to image a calibration card printed with a two-dimensional code matrix to obtain a wide-angle image and a telephoto image; wherein each two-dimensional code encodes its own row and column coordinates in the two-dimensional code matrix; S12: determining a correspondence between wide-angle image coordinates and row and column coordinates according to the QR code in the wide-angle image, and determining a correspondence between telephoto image coordinates and row and column coordinates according to the QR code in the telephoto image; S13: determining the row and column coordinates corresponding to the central pixel of the telephoto image according to the correspondence between the telephoto image coordinates and the row and column coordinates; S14: determining the wide-angle image coordinates corresponding to the central pixel of the telephoto image according to the row and column coordinates corresponding to the central pixel of the telephoto image and the correspondence between the wide-angle image coordinates and the row and column coordinates; S15: Establishing the mapping table by taking the wide-angle image coordinates corresponding to the central pixel of the telephoto image as an index and the control parameters of the telephoto camera during imaging as a return value.

2. The method according to claim 1, characterized in that: In the S40, it includes: S41: Determine a weight coefficient corresponding to each of the n control parameters; wherein the weight coefficient is inversely correlated with a distance of a wide-angle image coordinate corresponding to the weight coefficient relative to a wide-angle image coordinate of a calibration point; S42: taking the sum of the products of the n control parameters and their corresponding weight coefficients as the calibration control parameter.

3. The method according to claim 1, characterized in that In S12, determining the correspondence between wide-angle image coordinates and row and column coordinates according to the two-dimensional code in the wide-angle image includes: S121: Determine the row and column coordinates corresponding to each QR code based on the wide-angle image; S122: determining a correspondence between wide-angle image coordinates and row and column coordinates according to the wide-angle image coordinates of the central pixel of the two-dimensional code in the wide-angle image and the row and column coordinates corresponding to the two-dimensional code; Determining the correspondence between the telephoto image coordinates and the row and column coordinates according to the two-dimensional code in the telephoto image includes: S123: Determine the row and column coordinates corresponding to each QR code based on the telephoto image; S124: Determine a correspondence between the telephoto image coordinates and the row and column coordinates according to the telephoto image coordinates of the central pixel of the two-dimensional code in the telephoto image and the row and column coordinates corresponding to the two-dimensional code.

4. The method according to claim 3, characterized in that In the S121, it includes: S1211: converting the wide-angle image into a grayscale image and performing binarization processing; S1212: Performing connected domain analysis on the wide-angle image that has been binarized to obtain multiple color blocks; S1213: Eliminate color blocks whose areas do not meet the preset standards; S1214: Determine the color block whose center of gravity is closest to the central pixel of the wide-angle image as the seed color block, and set its row and column coordinates to be the same as the row and column coordinates encoded by the QR code at the center of the QR code matrix; S1215: Based on the seed color block, corresponding row and column coordinates are set for each color block to determine the row and column coordinates corresponding to each two-dimensional code.

5. The method according to claim 4, characterized in that In S1215, it includes: S12151: Using the ordinate of the centroid coordinate of the seed color block as a reference, searching for the next color block to the left / right, and each time a new color block is found, subtracting 1 or adding 1 to the row and column coordinates of the previous color block, and using the row and column coordinates of the new color block as a reference, and then continuing to search for the next color block to the left / right, until the wide-angle image boundary is reached; S12152: For the color block with determined row coordinates, use the horizontal coordinates of its own centroid coordinates as a reference to search for the next color block upward / downward. Each time a new color block is found, the row coordinates of the row and column coordinates of the previous color block are reduced by 1 or increased by 1 accordingly, and used as the row and column coordinates of the new color block. Then, use the horizontal coordinates of the centroid coordinates of the new color block as a reference to continue searching for the next color block upward / downward until the wide-angle image boundary is reached.

6. The method according to claim 4, characterized in that The two-dimensional code matrix includes a reference row and a reference column, and the color of the two-dimensional code in the reference row and the reference column is different from other two-dimensional codes in the two-dimensional code matrix. After S1215, the method further includes: S1216: For color blocks with the same row coordinates, calculate the color average of the color blocks in the row, and for color blocks with the same column coordinates, calculate the color average of the color blocks in the column; S1217: Determine a reference row and a reference column based on the color average value; S1218: Determine a row coordinate correction value and a column coordinate correction value according to the difference between the row and column coordinates pre-encoded by the reference row and the reference column and the row and column coordinates set when searching through the seed color block; S1219: Correct the row and column coordinates corresponding to each color block according to the row coordinate correction value and the column coordinate correction value.

7. The method according to any one of claims 1, 3-6, characterized in that: The correspondence between the wide-angle image coordinates and the row and column coordinates is an affine transformation matrix or a perspective transformation matrix; the correspondence between the telephoto image coordinates and the row and column coordinates is an affine transformation matrix or a perspective transformation matrix.

8. A control device for a binocular imaging system, the binocular imaging system comprising a wide-angle camera and a telephoto camera, characterized in that: The device comprises: A mapping establishment module, used to establish a mapping table; wherein the index of the mapping table is the wide-angle image coordinates of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera; A wide-angle imaging control module, used to control the wide-angle camera to perform imaging to obtain a calibration image; A retrieval module, used to determine a pixel point from the calibration image as a calibration point, and retrieve n control parameters from the mapping table using the wide-angle image coordinates of the calibration point as an index, where n is a positive integer greater than 1; An interpolation calculation module, used to perform interpolation calculation based on the retrieved n control parameters and the wide-angle image coordinates corresponding to the n control parameters in the mapping table to obtain calibration control parameters; A telephoto imaging control module, used to control the telephoto camera to perform imaging according to the calibration control parameters; The mapping establishment module is used to establish a mapping table; wherein the index of the mapping table is the wide-angle image coordinates of the wide-angle camera, and the return value of the mapping table is the control parameter of the telephoto camera, including: Controlling the binocular imaging system to image a calibration card printed with a two-dimensional code matrix to obtain a wide-angle image and a telephoto image; wherein each two-dimensional code encodes its own row and column coordinates in the two-dimensional code matrix; Determining the correspondence between wide-angle image coordinates and row and column coordinates according to the QR code in the wide-angle image, and determining the correspondence between telephoto image coordinates and row and column coordinates according to the QR code in the telephoto image; Determine the row and column coordinates corresponding to the central pixel of the telephoto image according to the correspondence between the telephoto image coordinates and the row and column coordinates; Determine the wide-angle image coordinates corresponding to the central pixel of the telephoto image according to the row and column coordinates corresponding to the central pixel of the telephoto image and the correspondence between the wide-angle image coordinates and the row and column coordinates; The mapping table is established by taking the wide-angle image coordinates corresponding to the central pixel of the telephoto image as an index and taking the control parameters of the telephoto camera during imaging as a return value.

9. A binocular imaging system, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 7.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images