A camera calibration method, a distance measuring method, a device and a storage medium

Abstract

Embodiments of the present application provide a camera calibration method, a distance measuring method, a device and a storage medium. In each grid contained in the checkerboard, an affine matrix corresponding to the grid is calculated according to the coordinate change state of the grid between the world coordinate system and the pixel coordinate system corresponding to the calibration image, as the calibration of the monocular camera. In this way, the fine calibration of different imaging regions of the monocular camera can be realized, thereby effectively improving the calibration accuracy. On this basis, the calibrated monocular camera can be used for distance measurement. By determining which grid in the checkerboard each object to be measured falls into, the world coordinates of the object are calculated using the affine matrix of the corresponding grid, and then the distance between the objects is calculated. Since the relative position and world coordinates of the grid and other parameters are not affected by image distortion, the error caused by image distortion can be effectively reduced during distance measurement, and the distance measurement accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of camera calibration technology, and in particular to a camera calibration method, ranging method, device and storage medium. Background Technology

[0002] In driving schools, GPS positioning is required for vehicles during student tests or practice sessions to indicate whether the vehicle has correctly arrived at the designated location.

[0003] However, vehicles often experience GPS location lockout issues due to factors such as roof or tree obstruction, leading to incorrect vehicle location or even failure to locate. Summary of the Invention

[0004] This application provides a camera calibration method, a ranging method, an apparatus, and a storage medium to improve the accuracy of camera calibration and / or ranging accuracy.

[0005] This application provides a ranging method, including:

[0006] A real-world image containing a first object and a second object is captured using a monocular camera. The monocular camera is fixedly mounted on the first object, and the calibration parameters of the monocular camera include the affine matrix under each cell of the checkerboard used in the calibration process.

[0007] Calculate the pixel coordinates of the key points on the first object and the second object in the pixel coordinate system corresponding to the real scene image;

[0008] Based on the range of pixel coordinates corresponding to each cell in the chessboard in the pixel coordinate system and the pixel coordinates corresponding to the key points on the first object and the second object, determine the first cell in which the key point on the first object falls and the second cell in which the key point on the second object falls.

[0009] Calculate the world coordinates of the key points on the first object in the world coordinate system based on the affine matrix under the first grid and the pixel coordinates corresponding to the key points on the first object, wherein the world coordinate system is established according to the size of the chessboard grid;

[0010] Calculate the world coordinates of the key points on the second object in the world coordinate system based on the affine matrix under the second grid and the pixel coordinates corresponding to the key points on the second object;

[0011] The distance between the first object and the second object is calculated based on the world coordinates of the key points on the first object and the second object, respectively.

[0012] This application also provides a camera calibration method, including:

[0013] After the monocular camera is fixed, the monocular camera is used to take pictures of the chessboard placed within the camera's field of view to obtain a calibration image;

[0014] Establish a world coordinate system based on the dimensions of the chessboard grid;

[0015] For each cell in the chessboard, the affine matrix corresponding to the cell is calculated based on the coordinate change state of the cell between the world coordinate system and the pixel coordinate system corresponding to the calibration image.

[0016] The affine matrix corresponding to each cell in the chessboard is used as the calibration parameter of the monocular camera.

[0017] This application also provides a computing device, including a memory and a processor;

[0018] The memory is used to store one or more computer instructions;

[0019] The processor is coupled to the memory and is used to execute the one or more computer instructions for:

[0020] A real-world image containing a first object and a second object is captured using a monocular camera. The monocular camera is fixedly mounted on the first object, and the calibration parameters of the monocular camera include the affine matrix under each cell of the checkerboard used in the calibration process.

[0021] Calculate the pixel coordinates of the key points on the first object and the second object in the pixel coordinate system corresponding to the real scene image;

[0022] Based on the range of pixel coordinates corresponding to each cell in the chessboard in the pixel coordinate system and the pixel coordinates corresponding to the key points on the first object and the second object, determine the first cell in which the key point on the first object falls and the second cell in which the key point on the second object falls.

[0023] Calculate the world coordinates of the key points on the first object in the world coordinate system based on the affine matrix under the first grid and the pixel coordinates corresponding to the key points on the first object, wherein the world coordinate system is established according to the size of the chessboard grid;

[0024] Calculate the world coordinates of the key points on the second object in the world coordinate system based on the affine matrix under the second grid and the pixel coordinates corresponding to the key points on the second object;

[0025] Based on the world coordinates corresponding to the keypoints on the first object and the second object, the distance between the first object and the second object is calculated.

[0026] This application also provides a computing device, including a memory and a processor;

[0027] The memory is used to store one or more computer instructions;

[0028] The processor is coupled to the memory and is used to execute the one or more computer instructions for:

[0029] After the monocular camera is fixed, the monocular camera is used to take pictures of the chessboard placed within the camera's field of view to obtain a calibration image;

[0030] Establish a world coordinate system based on the dimensions of the chessboard grid;

[0031] For each cell in the chessboard, the affine matrix corresponding to the cell is calculated based on the coordinate change state of the cell between the world coordinate system and the pixel coordinate system corresponding to the calibration image.

[0032] The affine matrix corresponding to each cell in the chessboard is used as the calibration of the monocular camera.

[0033] This application also provides a computer-readable storage medium for storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned ranging method or camera calibration method.

[0034] This application proposes a novel monocular camera calibration scheme. A world coordinate system is established based on the size of the calibration checkerboard. For each cell within the checkerboard, the affine matrix corresponding to the cell is calculated based on the coordinate changes between the world coordinate system and the pixel coordinate system of the calibration image. The affine matrix corresponding to each cell of the checkerboard is then used as the calibration matrix for the monocular camera. This allows for refined calibration of different imaging areas of the monocular camera, effectively improving calibration accuracy. Based on this, the calibrated monocular camera can be used for distance measurement. During distance measurement, it can be determined which cell within the checkerboard each object falls into, and the affine matrix of the corresponding cell is used to calculate the world coordinates of the object, thereby calculating the distance between the objects. Since the relative position of the cells and world coordinates are unaffected by image distortion, the distance measurement process based on the camera calibration scheme provided in this embodiment effectively reduces errors caused by image distortion, improving distance measurement accuracy. Furthermore, the distance measurement function can also achieve precise positioning of a specified object without relying on GPS positioning equipment. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 A flowchart illustrating a ranging method provided for an exemplary embodiment of this application;

[0037] Figure 2 A schematic diagram of a real-scene image provided for an exemplary embodiment of this application;

[0038] Figure 3 A schematic diagram illustrating a world coordinate system construction scheme provided for an exemplary embodiment of this application;

[0039] Figure 4 A schematic flowchart of a camera calibration method provided for another exemplary embodiment of this application;

[0040] Figure 5 This is a schematic diagram of the structure of a computing device provided as another exemplary embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] Currently, vehicles frequently experience GPS positioning loss due to factors such as roof obstruction or tree cover, leading to positioning errors or even complete inability to locate. To address this, some embodiments of this application propose a novel monocular camera calibration scheme. This scheme establishes a world coordinate system based on the dimensions of a calibration checkerboard. For each cell within the checkerboard, the affine matrix corresponding to the cell is calculated based on the coordinate changes between the world coordinate system and the pixel coordinate system of the calibration image. The affine matrix corresponding to each cell of the checkerboard is then used as the calibration matrix for the monocular camera. This allows for refined calibration of different imaging areas of the monocular camera, effectively improving calibration accuracy. Based on this, the calibrated monocular camera can be used for distance measurement. During distance measurement, it can be determined which cell within the checkerboard each object falls into, allowing the calculation of the world coordinates using the corresponding cell's affine matrix, and ultimately, the distance between the objects. Since the relative position and world coordinates of the grid cells are unaffected by image distortion, the distance measurement process based on the camera calibration scheme provided in this embodiment can effectively reduce errors caused by image distortion and improve distance measurement accuracy. Furthermore, the distance measurement function can also achieve precise positioning of a specified object without relying on GPS positioning equipment.

[0043] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0044] Figure 1 This is a flowchart illustrating a ranging method provided for an exemplary embodiment of this application. The method can be executed by a data processing device, which can be implemented as a combination of software and / or hardware, and can be integrated into a computing device. (Reference) Figure 1 The method includes:

[0045] Step 100: Use a monocular camera to capture a real-world image containing the first object and the second object. The monocular camera is fixedly mounted on the first object, and the calibration parameters of the monocular camera include the affine matrix under each cell of the checkerboard used in the calibration process.

[0046] Step 101: Calculate the pixel coordinates of the key points on the first object and the second object in the pixel coordinate system corresponding to the real scene image, respectively;

[0047] Step 102: Based on the range of pixel coordinates of each cell in the chessboard grid in the pixel coordinate system and the pixel coordinates of the key points on the first and second objects, determine the first cell in which the key point on the first object falls and the second cell in which the key point on the second object falls.

[0048] Step 103: Calculate the world coordinates of the key points on the first object in the world coordinate system according to the affine matrix under the first grid and the pixel coordinates corresponding to the key points on the first object. The world coordinate system is established according to the size of the chessboard grid.

[0049] Step 104: Calculate the world coordinates of the key points on the second object in the world coordinate system according to the affine matrix under the second grid and the pixel coordinates corresponding to the key points on the second object;

[0050] Step 105: Calculate the distance between the first object and the second object based on the world coordinates of the key points on the first object and the second object respectively.

[0051] The ranging method provided in this embodiment can be applied to various scenarios requiring ranging. For example, autonomous driving, driving school exams, etc. This embodiment does not specifically limit the application scenario. In different application scenarios, the object being measured can be diverse. For example, in an autonomous driving scenario, the object being measured can be the vehicle itself and adjacent vehicles; as another example, in a driving school exam scenario, the object being measured can be the vehicle itself and a white line on the ground. Of course, this is only an example, and this embodiment does not limit the attributes of the object being measured.

[0052] In this embodiment, the monocular camera can be fixedly mounted on the first object and calibrated. After calibration, the monocular camera remains stationary to ensure the validity of its calibration parameters. This embodiment can use a checkerboard pattern for camera calibration. Optionally, after the monocular camera is fixed, its field of view can be measured. A checkerboard pattern that perfectly covers the camera's field of view can be selected. The size and number of squares in the checkerboard can be set as needed; however, this is merely an example and this embodiment is not limited to this.

[0053] Based on this, a monocular camera can be used to photograph the checkerboard to obtain a calibration image. Then, the monocular camera can be calibrated according to the calibration object and the checkerboard. The specific calibration process will be detailed later. In this embodiment, during camera calibration, an affine matrix can be calculated for each cell in the checkerboard, and the affine matrix under each cell is used as the calibration parameter of the monocular camera. The affine matrix is ​​used to characterize the affine relationship between the pixel coordinate system and the world coordinate system. The pixel coordinate system can be established based on the size of the image captured by the monocular camera, while the world coordinate system can be established according to the size of the checkerboard.

[0054] After camera calibration is completed, the monocular camera can move synchronously with the first object, and can capture real-world images containing both the first and second objects. The second object can be any object in the real-world scene that needs to be measured in distance from the first object. For example, in an autonomous driving scenario, the first object could be the vehicle itself, and the second object could be the vehicle in front; similarly, in a driving school test scenario, the first object could be the vehicle itself, and the second object could be a white line on the ground. Figure 2 This is a schematic diagram of a real-scene image provided for an exemplary embodiment of this application. (Reference) Figure 2 In this exemplary real-world image, the first object is a vehicle, and the second object is a white line on the ground.

[0055] In this embodiment, key points can be selected on the first object and the second object respectively. These key points can represent the objects they belong to; for example, the position of a key point can represent the position of the object it belongs to. Thus, distance measurement between the first and second objects can be achieved by calculating the distance between the key points on the first and second objects. In this embodiment, the selection position of the key points can be customized as needed. For example, the center point of the object can be selected as a key point, or a specified point or random point on the object can be selected as a key point. This embodiment does not impose any limitations on this.

[0056] refer to Figure 1 In step 101, the pixel coordinates of the key points of the first object and the second object in the pixel coordinate system corresponding to the real-scene image can be calculated respectively. In this embodiment, the pixel coordinates can be obtained from the real-scene image using methods such as OpenCV. Of course, this is only an example. In this embodiment, any technical solution for obtaining pixel coordinates in an image that is proposed now or in the future can be used to calculate the pixel coordinates of the key points of the first object and the second object in the pixel coordinate system corresponding to the real-scene image. It will not be described in detail here.

[0057] For key points on the first object, preferably, the pixel coordinates corresponding to the key points on the first object can be calculated and stored after the monocular camera is fixedly assembled. The pre-calculation process may include: using the monocular camera to photograph the key points on the first object to obtain a test image; calculating and storing the pixel coordinates of the key points on the first object in the pixel coordinate system in the test image. During the ranging process, since the relative position between the monocular camera and the first object remains unchanged, the pixel coordinates corresponding to the key points on the first object remain unchanged in any real-world image captured by the monocular camera. Thus, the pre-stored pixel coordinates of the key points on the first object in the pixel coordinate system can be directly obtained and used during the ranging process.

[0058] For the key points on the second object, since the second object is flexible and variable, real-time calculation is still used to obtain the pixel coordinates of the key points in the pixel coordinate system. To further improve the accuracy of the pixel coordinates of the key points on the second object, in this embodiment, object detection can be performed on the real-scene image to outline the location region of the second object in the real-scene image; within the location region, the pixel coordinates of the key points on the second object are calculated. The object detection process on the real-scene image can be performed using a deep learning model, which can employ various object detection algorithms proposed now or in the future; this embodiment does not limit this approach. For example, if the second object is a white line on the ground, a deep learning model can first outline the white line region in the real-scene image. Then, within the white line region, methods such as OpenCV are used to calculate the pixel coordinates of the key points on the white line. Adding object detection processing can effectively improve the accuracy of the pixel coordinates of the key points.

[0059] As mentioned earlier, during camera calibration, a calibration image can be obtained. From this image, the pixel coordinate range of each cell in the checkerboard pattern can be determined using the pixel coordinate acquisition method described above. For example, for a single cell, the pixel coordinates of its four corner points can be calculated, and the range of pixel coordinates covered by that cell can be determined based on these corner coordinates.

[0060] Based on this, refer to Figure 1 In step 102, the first cell in which the key point on the first object falls and the second cell in which the key point on the second object falls can be determined based on the pixel coordinate range of each cell in the checkerboard grid and the pixel coordinates of the key points on the first and second objects, respectively. In practical applications, each cell in the checkerboard grid can be assigned a unique identifier, and the relative positional relationships between cells can be recorded. This allows for accurate determination of the first cell in which the key point on the first object falls and the second cell in which the key point on the second object falls within the checkerboard grid.

[0061] refer to Figure 1 Once the first grid cell where the key point on the first object falls and the second grid cell where the key point on the second object falls are determined, the affine matrices corresponding to the first and second grid cells can be obtained from the calibration parameters of the monocular camera.

[0062] In steps 103 and 104, the world coordinates of the keypoints on the first object are calculated in the world coordinate system based on the affine matrix under the first grid and the pixel coordinates corresponding to the keypoints on the first object; similarly, the world coordinates of the keypoints on the second object are calculated based on the affine matrix under the second grid and the pixel coordinates corresponding to the keypoints on the second object. This yields the world coordinates corresponding to the keypoints on both the first and second objects. Since the affine matrix under the grid where the keypoint falls is used in calculating the world coordinates, a suitable affine matrix can be used to calculate the world coordinates even when the keypoints are located in different areas of the real-world image, resulting in more accurate world coordinates.

[0063] refer to Figure 1 In step 105, the distance between the first object and the second object can be calculated based on the world coordinates corresponding to the keypoints on the first object and the second object, respectively. Here, the distance between two keypoints can be calculated based on the world coordinates corresponding to the keypoints on the first object and the second object, and this distance can be used as the distance between the first object and the second object.

[0064] In one optional implementation, the distance between the keypoints on the first object and the X and Y axes of the world coordinate system can be calculated based on the world coordinates corresponding to the keypoints on the first object; the distance between the keypoints on the second object and the X and Y axes of the world coordinate system can be calculated based on the world coordinates corresponding to the keypoints on the second object; and the Euclidean distance between the keypoints on the first object and the second object can be calculated based on the distance between the keypoints on the first object and the X and Y axes of the world coordinate system and the distance between the keypoints on the second object and the X and Y axes of the world coordinate system, which is then used as the distance between the first object and the second object.

[0065] In this implementation, during the calculation of the Euclidean distance between two key points, the distances from each key point to the X-axis and Y-axis in the world coordinate system are calculated. Taking a key point on the second object as an example, in one exemplary scheme, the X-axis in the world coordinate system can be set to coincide with a specified edge of a specified cell in the chessboard. Thus, during the calculation of the distance from the key point to the X-axis in the world coordinate system: a first distance can be calculated between the key point on the second object and the first edge of the second cell it falls into that is closest to the X-axis in the world coordinate system; a second distance can be obtained between the first edge and the X-axis in the world coordinate system; and the sum of the first and second distances can be used as the distance between the key point on the second object and the X-axis in the world coordinate system. In this exemplary scheme, the second distance between the first side and the X-axis in the world coordinate system can be obtained by multiplying the number of grids between the first side and the X-axis by the side length of a single grid. Since the values ​​of the number of grids and the side length of the grid do not change due to image distortion, the second distance between the first side and the X-axis in the world coordinate system will not have an error, regardless of whether there is distortion in the real-world image. Thus, for the distances between keypoints on the second object and the X-axis in the world coordinate system, only the first distance may cause an error, while the second distance will not. This greatly reduces the overall calculation error, especially when there is distortion in the real-world image, effectively improving the ranging error caused by distortion. This effectively overcomes the technical difficulty that existing camera calibration schemes such as Zhang's calibration cannot handle tilt angles and image distortion. In addition, in this exemplary scheme, the distances from keypoints on the second object to the Y-axis in the world coordinate system, as well as the distances from keypoints on the first object to the X-axis and Y-axis in the world coordinate system, can also be calculated in the same way, which will not be repeated here. With more accurate distances to the coordinate axes, the accuracy of the calculated Euclidean distance can be effectively improved, thereby enhancing the accuracy of distance measurement.

[0066] Accordingly, in this embodiment, the affine matrix of each cell in the chessboard used for calibration can be used as the calibration parameters of the monocular camera. Based on this, the calibrated monocular camera can be used for ranging. During the ranging process, it can be determined which cell in the chessboard each of the two objects being measured falls into, and then the affine matrix of the corresponding cell can be used to calculate the world coordinates, thereby calculating the distance between the objects. Since the relative position of the cells and parameters such as world coordinates are not affected by image distortion, the ranging scheme provided in this embodiment can effectively reduce errors caused by image distortion and improve ranging accuracy.

[0067] In the above or below embodiments, based on the above ranging scheme, GPS data corresponding to key points on the second object can also be obtained; based on the GPS data corresponding to key points on the second object and the distance between the first object and the second object, GPS data corresponding to key points on the first object is calculated to locate the first object.

[0068] This embodiment proposes a positioning scheme based on the aforementioned ranging scheme. In this embodiment, after determining the distance between the first object and the second object, the GPS data of the second object can be obtained. Given the GPS data of one object and the distance between them, the GPS data of the other object can be calculated quickly and accurately. For example, the first object can be a vehicle, and the second object can be a white line on the ground. The GPS data of the white line can be known in advance. Based on the known GPS data of the white line and the distance between the vehicle and the white line, the GPS data of the vehicle can be calculated, thereby achieving vehicle positioning.

[0069] To improve positioning accuracy, this embodiment also proposes to fix at least two monocular cameras on the first object. The at least two monocular cameras can be mounted at different locations on the first object. For example, if the first object is a vehicle, the at least two monocular cameras are distributed in front of, behind, and / or to the side of the vehicle.

[0070] Based on this, GPS data corresponding to key points on the second object captured by at least two monocular cameras can be obtained respectively, and the second objects captured by different monocular cameras are not completely the same; GPS data corresponding to key points on the first object can be calculated based on the GPS data corresponding to key points on the second object captured by at least two monocular cameras and the distance between the first object and the second object captured by at least two monocular cameras.

[0071] To address this, the ranging scheme proposed in the above embodiments can be used under each monocular camera to calculate the distance between the first object and the second object captured by the monocular camera. It is worth noting that the second objects captured by different monocular cameras may not be entirely identical. For example, a monocular camera can be mounted at the front and rear of the vehicle and calibrated. Based on this, the monocular camera mounted at the front of the vehicle can capture images of the front of the vehicle and the white line in front, while the monocular camera mounted at the rear of the vehicle can capture images of the rear of the vehicle and the white line behind. In this way, the distance between the front of the vehicle and the white line in front, and the distance between the rear of the vehicle and the white line behind, can be calculated separately under the two monocular cameras. Since the GPS data of the white lines in front and behind are known, the GPS data of the front and rear of the vehicle can be calculated quickly and accurately. The GPS data of the front and rear of the vehicle are cross-calibrated and fused to accurately determine the vehicle's GPS data.

[0072] In this embodiment, the ranging function proposed in the above embodiments can be used to achieve accurate positioning of a specified object without relying on a GPS positioning device.

[0073] In the above or following embodiments, the monocular camera can be calibrated in advance. One optional calibration process may be:

[0074] After the monocular camera is fixed, the monocular camera is used to take pictures of the checkerboard pattern placed within the camera's field of view to obtain a calibration image;

[0075] Establish a world coordinate system based on the dimensions of the chessboard grid;

[0076] For each cell in the chessboard, the affine matrix corresponding to the cell is calculated based on the coordinate change state of the cell between the world coordinate system and the pixel coordinate system corresponding to the calibration image.

[0077] The affine matrix corresponding to each cell in the chessboard is used as the calibration parameter for the monocular camera.

[0078] In establishing the world coordinate system, the X-axis can be set to coincide with a specified edge of a specified cell in the chessboard. This ensures that the coordinate axes of the world coordinate system lie on the edges of the relevant cells in the chessboard. Figure 3 This is a schematic diagram illustrating a world coordinate system construction scheme provided for an exemplary embodiment of this application. For example... Figure 3 As an example, the top-left corner of the top-left cell in the chessboard grid can be used as the origin, with the X-axis coinciding with the top edge of that cell and the Y-axis coinciding with the left edge of that cell, thus constructing a world coordinate system. In this world coordinate system, based on the size of each cell, the world coordinates of each corner point, the number of cells it spans relative to the coordinate axes, the range of world coordinates it covers, and other parameters that may be used in the ranging scheme can be determined. In this embodiment, these parameters can be recorded and directly called in the aforementioned ranging scheme.

[0079] In this embodiment, a pixel coordinate system can also be established based on the size of the captured calibration image. Images are composed of pixels, and pixel coordinates can be used to represent the position of a pixel within the image. Based on the pixel coordinate system, techniques such as target detection and OpenCV mentioned earlier can be used to determine the pixel coordinates of the corner points in each square of the checkerboard pattern, the range of pixel coordinates covered, and other parameters that may be used in the ranging scheme. In this embodiment, these parameters can be recorded and directly called in the aforementioned ranging scheme.

[0080] Based on this, using the grid as the unit, the coordinate transformation state of each grid in the world coordinate system and the pixel coordinate system of the calibration image can be characterized according to the various parameters of each grid in the world coordinate system and the various parameters of each grid in the pixel coordinate system. Then, the affine matrix corresponding to the grid can be calculated. Taking one target grid as an example, the world coordinates of the corner point A of the target grid are (M1, N1). In the calibration image, the target grid is flipped, and the pixel coordinates of the corner point A are (x1, y1). Then, the following matrix relationship must exist:

[0081]

[0082] Given the world coordinates and pixel coordinates of corner point A, the affine matrix of the target grid can be accurately calculated.

[0083] Of course, the target grid may rotate not only in two-dimensional space on the labeled image, but also in three-dimensional space. In this case, the dimension can be added to the affine matrix. The matrix relationship between the world coordinates and pixel coordinates of corner point A becomes:

[0084]

[0085] In this matrix, the third element of the pixel coordinate matrix [x1, y1, x1*y1, 1] is not limited to x1*y1. This element can be any value that satisfies the requirement of matrix non-invertibility (e.g., any value that is not 0, not equal to x1 or y1, and not a multiple of x1 or y1). Furthermore, in practical applications, by repeatedly flipping corner point A, or by adding the flipping data of other corner points of the target cell, the affine matrix of the target cell can be calculated as follows.

[0086]

[0087] In the same way, the affine matrices of other squares in the chessboard can be calculated, and the affine matrices of each square can be used as calibration parameters for the monocular camera.

[0088] Accordingly, this embodiment proposes a novel monocular camera calibration scheme. A world coordinate system is established based on the size of the calibration checkerboard. For each cell within the checkerboard, the affine matrix corresponding to the cell is calculated based on the coordinate changes between the world coordinate system and the pixel coordinate system of the calibration image. The affine matrix corresponding to each cell of the checkerboard is then used as the calibration matrix for the monocular camera. This allows for refined calibration of different imaging regions of the monocular camera, effectively improving calibration accuracy.

[0089] Figure 4This is a schematic flowchart illustrating a camera calibration method as provided in another exemplary embodiment of this application. The method can be executed by a data processing device, which can be implemented as a combination of software and / or hardware, and can be integrated into a computing device. (Reference) Figure 4 The method includes:

[0090] Step 400: After the monocular camera is fixed, take a picture of the checkerboard pattern placed within the camera's field of view using the monocular camera to obtain a calibration image.

[0091] Step 401: Establish a world coordinate system based on the dimensions of the chessboard grid;

[0092] Step 402: For each cell in the chessboard, calculate the affine matrix corresponding to the cell based on the coordinate change state between the world coordinate system and the pixel coordinate system corresponding to the calibration image.

[0093] Step 403: Use the affine matrix corresponding to each cell in the chessboard as the calibration parameters for the monocular camera.

[0094] The camera calibration scheme provided in this embodiment can be applied to scenarios where a monocular camera needs to be calibrated. This embodiment does not limit the application scenario.

[0095] The camera calibration scheme provided in this embodiment can serve as the basis for monocular cameras to participate in other applications. The applications that monocular cameras can participate in include, but are not limited to, the ranging and positioning work mentioned in the above embodiments. This embodiment does not limit the application fields after the camera calibration scheme.

[0096] In one alternative embodiment, the process of calculating the affine matrix may include:

[0097] Under the target grid, calculate the pixel coordinates of at least one target corner point in the pixel coordinate system corresponding to the calibration image;

[0098] In the world coordinate system, determine the world coordinates corresponding to at least one target corner point;

[0099] Calculate the affine matrix corresponding to the target cell based on the world coordinates and pixel coordinates of at least one target corner cell.

[0100] In an alternative embodiment, the coordinate axes of the world coordinate system can be set to coincide with the specified edges of specified cells in the chessboard.

[0101] It is worth noting that the technical details of the above-mentioned camera calibration scheme embodiments can be found in the relevant descriptions of the camera calibration process in the aforementioned range measurement scheme embodiments. To save space, these details will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0102] Accordingly, this embodiment proposes a novel monocular camera calibration scheme. A world coordinate system is established based on the size of the calibration checkerboard. For each cell within the checkerboard, the affine matrix corresponding to the cell is calculated based on the coordinate changes between the world coordinate system and the pixel coordinate system of the calibration image. The affine matrix corresponding to each cell of the checkerboard is then used as the calibration matrix for the monocular camera. This allows for refined calibration of different imaging regions of the monocular camera, effectively improving calibration accuracy.

[0103] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 100 to 105 can be device A; or the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0104] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different key points, grids, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0105] Figure 5 This is a schematic diagram of the structure of a computing device provided as another exemplary embodiment of this application. For example... Figure 5 As shown, the computing device includes a memory 50 and a processor 51.

[0106] Processor 51, coupled to memory 50, is used to execute computer programs in memory 50 for:

[0107] A real-world image containing a first object and a second object is captured using a monocular camera. The monocular camera is fixedly mounted on the first object, and the calibration parameters of the monocular camera include the affine matrix under each cell of the checkerboard used in the calibration process.

[0108] Calculate the pixel coordinates of the key points on the first and second objects in the pixel coordinate system corresponding to the real scene image, respectively;

[0109] Based on the range of pixel coordinates corresponding to each cell in the checkerboard grid in the pixel coordinate system and the pixel coordinates corresponding to the key points on the first and second objects, determine the first cell in which the key point on the first object falls and the second cell in which the key point on the second object falls.

[0110] Calculate the world coordinates of the key points on the first object in the world coordinate system based on the affine matrix under the first grid and the pixel coordinates corresponding to the key points on the first object. The world coordinate system is established according to the size of the chessboard grid.

[0111] Calculate the world coordinates of the key points on the second object in the world coordinate system based on the affine matrix under the second grid and the pixel coordinates corresponding to the key points on the second object.

[0112] Calculate the distance between the first object and the second object based on the world coordinates of the key points on the first object and the second object respectively.

[0113] In an alternative embodiment, processor 51 may also be used for:

[0114] Obtain GPS data corresponding to key points on the second object;

[0115] Based on the GPS data corresponding to the key points on the second object and the distance between the first and second objects, the GPS data corresponding to the key points on the first object is calculated to locate the first object.

[0116] In an optional embodiment, at least two monocular cameras are fixedly mounted on the first object. During the process of calculating the GPS data corresponding to the key points on the first object based on the GPS data corresponding to key points on the second object and the distance between the first and second objects, the processor 51 can be used for:

[0117] Obtain GPS data corresponding to key points on the second object captured by at least two monocular cameras, respectively, as the second objects captured by different monocular cameras are not exactly the same;

[0118] The GPS data corresponding to the key points on the first object is calculated based on the GPS data corresponding to the key points on the second object captured by at least two monocular cameras and the distance between the first object and the second object captured by at least two monocular cameras.

[0119] In one alternative embodiment, the first object is a vehicle, and at least two monocular cameras are distributed in front of, behind and / or to the side of the vehicle.

[0120] In an optional embodiment, the processor 51, during the process of calculating the pixel coordinates of key points on the second object in the pixel coordinate system, can be used to:

[0121] Target detection is performed on the real-world image to outline the location region of the second object in the real-world image;

[0122] Within the location region, calculate the pixel coordinates of key points on the second object.

[0123] In an optional embodiment, the processor 51, during the process of calculating the pixel coordinates of key points on the first object in the pixel coordinate system, may be used to:

[0124] Get the pixel coordinates of the key points on the pre-stored first object in the pixel coordinate system;

[0125] The process of pre-determining the pixel coordinates of key points on the first object in the pixel coordinate system includes:

[0126] Select key points on the first object;

[0127] Use a monocular camera to capture key points on the first object to obtain a test image;

[0128] In the test image, calculate and store the pixel coordinates of key points on the first object in the pixel coordinate system.

[0129] In an optional embodiment, the processor 51 may be used during the calibration of the monocular camera to:

[0130] After the monocular camera is fixed, the monocular camera is used to take pictures of the checkerboard pattern placed within the camera's field of view to obtain a calibration image;

[0131] Establish a world coordinate system based on the dimensions of the chessboard grid;

[0132] For each cell in the chessboard, the affine matrix corresponding to the cell is calculated based on the coordinate change state of the cell between the world coordinate system and the pixel coordinate system corresponding to the calibration image.

[0133] The affine matrix corresponding to each cell in the chessboard is used as the calibration parameter for the monocular camera.

[0134] In an optional embodiment, during the process of establishing a world coordinate system based on the dimensions of the chessboard, the processor 51 can be used to:

[0135] Set the coordinate axes in the world coordinate system to coincide with the specified edges of specified cells in the chessboard.

[0136] In an optional embodiment, the processor 51, in calculating the distance between the first object and the second object based on the world coordinates corresponding to key points on the first object and the second object, can be used for:

[0137] Based on the world coordinates corresponding to the key points on the first object, calculate the distances between the key points on the first object and the X-axis and Y-axis in the world coordinate system;

[0138] Based on the world coordinates corresponding to the key points on the second object, calculate the distances between the key points on the second object and the X-axis and Y-axis in the world coordinate system;

[0139] Based on the distance between keypoints on the first object and the X and Y axes in the world coordinate system, and the distance between keypoints on the second object and the X and Y axes in the world coordinate system, the Euclidean distance between keypoints on the first object and the second object is calculated, and this distance is taken as the distance between the first object and the second object.

[0140] In an optional embodiment, the X-axis in the world coordinate system is set to coincide with a specified edge of a specified cell in the chessboard. During the calculation of the distance between key points on the second object and the X-axis in the world coordinate system, the processor 51 can be used to:

[0141] Calculate the first distance between the key point on the second object and the first edge of the second grid that it falls into that is closest to the X-axis in the world coordinate system;

[0142] Obtain the second distance between the first edge and the X-axis in the world coordinate system;

[0143] The sum of the first and second distances is used as the distance between the key point on the second object and the X-axis in the world coordinate system.

[0144] Furthermore, such as Figure 5 As shown, the computing device also includes other components such as a communication component 52, a display 53, a power supply component 54, and an audio component 55. Figure 5 The diagram only shows some components and does not mean that the computing device includes only these components. Figure 5 The components shown.

[0145] It is worth noting that the technical details of the above-mentioned embodiments of the computing device can be referred to the relevant descriptions in the aforementioned ranging scheme embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0146] based on Figure 5 In another exemplary embodiment of this application, the aforementioned camera calibration scheme can also be implemented based on the computing device, in addition to the structure of the computing device provided.

[0147] In this embodiment, processor 51 can be used for:

[0148] After the monocular camera is fixed, the monocular camera is used to take pictures of the checkerboard pattern placed within the camera's field of view to obtain a calibration image;

[0149] Establish a world coordinate system based on the dimensions of the chessboard grid;

[0150] For each cell in the chessboard, the affine matrix corresponding to the cell is calculated based on the coordinate change state of the cell between the world coordinate system and the pixel coordinate system corresponding to the calibration image.

[0151] The affine matrix corresponding to each cell in the chessboard is used as the calibration parameter for the monocular camera.

[0152] Optionally, the processor 51 can be used in the process of calculating the affine matrix to:

[0153] Under the target grid, calculate the pixel coordinates of at least one target corner point in the pixel coordinate system corresponding to the calibration image;

[0154] In the world coordinate system, determine the world coordinates corresponding to at least one target corner point;

[0155] Calculate the affine matrix corresponding to the target cell based on the world coordinates and pixel coordinates of at least one target corner cell.

[0156] Optionally, the processor 51 can be used to: establish a world coordinate system based on the dimensions of the chessboard grid.

[0157] Set the coordinate axes in the world coordinate system to coincide with the specified edges of specified cells in the chessboard.

[0158] It is worth noting that the technical details of the above-mentioned embodiments of the computing device can be referred to the relevant descriptions in the aforementioned camera calibration scheme embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0159] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by a computing device in the above method embodiments.

[0160] The above Figure 5 The memory in a computer is used to store computer programs and can be configured to store various other data to support operation on a computing platform. Examples of this data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disks, or optical disks.

[0161] The above Figure 5 The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0162] The above Figure 5 The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0163] The above Figure 5 The power supply component provides power to various components within the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0164] The above Figure 5 The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0165] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0169] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0170] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0171] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0172] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method of ranging, characterized by, The method comprises the following steps: capturing a real scene image containing a first object and a second object by using a monocular camera fixedly arranged on the first object, and the calibration parameters of the monocular camera include an affine matrix of each grid in a calibration process; calculating the pixel coordinates of the key points on the first object and the second object in a pixel coordinate system respectively; determining the first grid in which the key points on the first object fall and the second grid in which the key points on the second object fall according to the pixel coordinate range corresponding to each grid in the pixel coordinate system and the pixel coordinates corresponding to the key points on the first object and the second object respectively; calculating the world coordinates of the key points on the first object in a world coordinate system according to the affine matrix of the first grid and the pixel coordinates corresponding to the key points on the first object, wherein the world coordinate system is established according to the size of the chessboard; calculating the world coordinates of the key points on the second object in the world coordinate system according to the affine matrix of the second grid and the pixel coordinates corresponding to the key points on the second object; calculating the distance between the first object and the second object based on the world coordinates corresponding to the key points on the first object and the second object respectively.

2. The method of claim 1, wherein, Further comprising: obtaining the GPS data corresponding to the key points on the second object; calculating the GPS data corresponding to the key points on the first object based on the GPS data corresponding to the key points on the second object and the distance between the first object and the second object, so as to position the first object.

3. The method of claim 2, wherein, The first object is fixedly arranged with at least two monocular cameras, and the calculation of the GPS data corresponding to the key points on the first object based on the GPS data corresponding to the key points on the second object and the distance between the first object and the second object comprises: obtaining the GPS data corresponding to the key points on the second object respectively photographed by the at least two monocular cameras, and the second object photographed by different monocular cameras is not completely the same; calculating the GPS data corresponding to the key points on the first object based on the GPS data corresponding to the key points on the second object respectively photographed by the at least two monocular cameras and the distance between the first object and the second object respectively photographed by the at least two monocular cameras.

4. The method of claim 3, wherein, The first object is a vehicle, and the at least two monocular cameras are distributed in front, rear and / or side of the vehicle.

5. The method of claim 1, wherein, The calculation of the pixel coordinates of the key points on the second object in the pixel coordinate system comprises: performing target detection on the real scene image to frame the position area of the second object in the real scene image; calculating the pixel coordinates of the key points on the second object in the position area.

6. The method of claim 1, wherein, The calculation of the pixel coordinates of the key points on the first object in the pixel coordinate system comprises: obtaining the pixel coordinates of the key points on the first object in the pixel coordinate system stored in advance; wherein the process of determining the pixel coordinates of the key points on the first object in the pixel coordinate system in advance comprises: Selecting key points on the first object; Taking pictures of the key points on the first object by using the monocular camera to obtain a test image; In the test image, calculating and storing the pixel coordinates of the key points on the first object in the pixel coordinate system.

7. The method of claim 1, wherein, The process of calibrating the monocular camera includes: After the monocular camera is fixed, taking pictures of a checkerboard placed in the field of view of the camera by using the monocular camera to obtain a calibration image; Establishing the world coordinate system according to the size of the checkerboard; Under each cell contained in the checkerboard, calculating an affine matrix corresponding to the cell according to the coordinate change state of the cell between the world coordinate system and the pixel coordinate system corresponding to the calibration image; Taking the respective affine matrix of each cell contained in the checkerboard as the calibration parameter of the monocular camera.

8. The method of claim 7, wherein, The process of establishing the world coordinate system according to the size of the checkerboard includes: Respectively setting the coordinate axes in the world coordinate system to coincide with the specified edges of the specified cells in the checkerboard.

9. The method of claim 7, wherein, The process of calculating the affine matrix includes: Under a target cell, calculating the pixel coordinates of at least one target corner point of the target cell in the pixel coordinate system corresponding to the calibration image; In the world coordinate system, determining the respective world coordinates of the at least one target corner point; Based on the world coordinates and pixel coordinates of the at least one target corner point cell, calculating the affine matrix corresponding to the target cell.

10. The method of claim 1, wherein, The process of calculating the distance between the first object and the second object based on the respective world coordinates of the key points on the first object and the second object includes: According to the world coordinates of the key points on the first object, calculating the distances between the key points on the first object and the X-axis and the Y-axis in the world coordinate system; According to the world coordinates of the key points on the second object, calculating the distances between the key points on the second object and the X-axis and the Y-axis in the world coordinate system; Based on the distances between the key points on the first object and the X-axis and the Y-axis in the world coordinate system and the distances between the key points on the second object and the X-axis and the Y-axis in the world coordinate system, calculating the Euclidean distance between the key points on the first object and the second object as the distance between the first object and the second object.

11. The method of claim 10, wherein, The process of calculating the distance between the key points on the second object and the X-axis in the world coordinate system includes: Calculating a first distance between the key point on the second object and the first edge of the second cell into which the key point falls and which is closest to the X-axis in the world coordinate system; Obtaining a second distance between the first edge and the X-axis in the world coordinate system; Taking the sum of the first distance and the second distance as the distance between the key point on the second object and the X-axis in the world coordinate system.

12. A computing device, comprising: The device includes a memory and a processor; The memory is configured to store one or more computer instructions; The processor is coupled to the memory and is configured to execute the one or more computer instructions to: An image of a real scene containing a first object and a second object is captured by a monocular camera fixedly mounted on the first object, and a calibration parameter of the monocular camera includes an affine matrix of each cell in a calibration process used checkerboard; Pixel coordinates of key points on the first object and the second object in a pixel coordinate system corresponding to the image are calculated respectively; A first cell in which the key points on the first object fall and a second cell in which the key points on the second object fall are determined according to a pixel coordinate range corresponding to each cell in the checkerboard and the pixel coordinates corresponding to the key points on the first object and the second object respectively; World coordinates of the key points on the first object in a world coordinate system are calculated according to the affine matrix of the first cell and the pixel coordinates corresponding to the key points on the first object, wherein the world coordinate system is established according to the size of the checkerboard; World coordinates of the key points on the second object in the world coordinate system are calculated according to the affine matrix of the second cell and the pixel coordinates corresponding to the key points on the second object; A distance between the first object and the second object is calculated based on the world coordinates corresponding to the key points on the first object and the second object respectively.

13. A computer readable storage medium storing computer instructions, wherein, The computer instructions, when executed by one or more processors, cause the one or more processors to perform the ranging method of any one of claims 1-11.

Images