Camera calibration method and device

Abstract

The invention discloses a camera calibration method and device which are applied to the field of image processing, and the method comprises the steps: obtaining the real feature angular point distribution of a current frame image shot by a vehicle-mounted camera and the real feature angular point distribution of a next frame image, according to the initial camera pose matrix of the vehicle-mounted camera and the real feature angular point distribution of the current frame of image, determining the predicted feature angular point distribution of the next frame of image, and according to the pixel error between the real feature angular point distribution of the next frame of image and the predicted feature angular point distribution, carrying out iterative tuning on the initial camera pose matrix; and in each adjustment and optimization process, according to the camera pose matrix after current adjustment and optimization, updating a pixel error until the pixel error satisfies a preset error condition or the iterative cost function satisfies an iteration condition, and determining that the corresponding camera pose matrix is a target camera pose matrix. Therefore, the problem that manual on-site operation needs to be influenced by the environment of the calibration site can be avoided. The camera calibration process can be simplified, and the camera calibration cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a camera calibration method and device. Background Art

[0002] The Surround View Monitor System (SVM) is an advanced driver assistance system that provides a panoramic view of the vehicle's surroundings by installing cameras around the vehicle. This system can significantly improve the driver's awareness of the vehicle's surroundings, especially when parking, reversing or driving in a narrow space.

[0003] At present, vehicles are generally equipped with an on-board surround view camera system. In some cases, such as when the vehicle encounters a collision, scratch, or bump during use, the posture of the surround view camera changes, and the factory camera calibration file is no longer applicable, so the surround view camera needs to be recalibrated.

[0004] Existing calibration methods generally require professional technicians to carry professional calibration equipment for on-site operations, which are affected by the environment of the calibration site and the entire camera calibration process is relatively cumbersome. The cost of manual camera calibration is relatively high. Summary of the invention

[0005] The present application provides a camera calibration method and device to prevent camera calibration from being affected by the environment of the calibration site, simplify the camera calibration process, and reduce the camera calibration cost.

[0006] In a first aspect, the present application provides a camera calibration method, comprising:

[0007] Obtain the real feature corner point distribution of the current frame image taken by the vehicle-mounted camera and the real feature corner point distribution of the next frame image;

[0008] Determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner point distribution of the current frame image;

[0009] According to the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, the initial camera pose matrix is ​​iteratively tuned, and in each tuning process, the pixel error is updated according to the camera pose matrix after this tuning until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

[0010] Optionally, iteratively tuning the initial camera pose matrix according to the pixel error includes:

[0011] According to the pixel error, construct a graph model, the graph model including a first type of node, a second type of node and an edge, the first type of node is used to represent a camera pose matrix, the second type of node is used to represent the coordinates of feature points detected in a three-dimensional space, the first type of node and the second type of node are connected by the edge, and are used to represent the pixel position projected from the three-dimensional feature point to the two-dimensional image plane and reflect the relationship between the displacement and rotation of the camera pose matrix and the feature point projection error;

[0012] Iteratively optimize the first type of nodes in the graph model.

[0013] Optionally, the pixel error satisfies a preset error condition, including:

[0014] When the pixel error value between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is less than a pixel error threshold, or the proportion of the pixel error value in the pixel value of the actual feature corner point distribution of the next frame image is less than a proportion threshold, it is determined that the pixel error meets a preset error condition.

[0015] Optionally, when the pixel error value is greater than or equal to a pixel error threshold, or the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is greater than or equal to a proportion threshold, the initial camera pose matrix is ​​iteratively optimized based on the pixel error between the real feature corner point distribution of the next frame image and the predicted feature corner point distribution.

[0016] Optionally, the iterative cost function satisfies an iteration condition, including:

[0017] When the number of iterations meets a preset number of iterations or the cost function reaches a minimum value, it is determined that the iterative cost function meets the iteration condition.

[0018] Optionally, the pixel error between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is determined in the following manner:

[0019] Sort the pixels in the real feature corner point distribution and the predicted feature corner point distribution according to the pixel coordinates;

[0020] After the pixels are sorted, the pixel error value between each mapping point pair between the real feature corner point distribution and the predicted feature corner point distribution is calculated;

[0021] The mean of the error values ​​is used as the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image.

[0022] Optionally, the method further includes:

[0023] Get the current frame image and the next frame image taken by the vehicle camera;

[0024] Scale-invariant feature transformation detection is performed on the current frame image and the next frame image respectively to obtain the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image.

[0025] In a second aspect, the present application further provides a camera calibration device, the device comprising:

[0026] A first acquisition unit is used to acquire the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image taken by the vehicle-mounted camera;

[0027] A first determining unit, configured to determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner point distribution of the current frame image;

[0028] The second determination unit is used to iteratively tune the initial camera pose matrix according to the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, and update the pixel error according to the camera pose matrix after the current tuning in each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

[0029] Optionally, the second determining unit is specifically configured to:

[0030] According to the pixel error, construct a graph model, the graph model including a first type of node, a second type of node and an edge, the first type of node is used to represent a camera pose matrix, the second type of node is used to represent the coordinates of feature points detected in a three-dimensional space, the first type of node and the second type of node are connected by the edge, and are used to represent the pixel position projected from the three-dimensional feature point to the two-dimensional image plane and reflect the relationship between the displacement and rotation of the camera pose matrix and the feature point projection error;

[0031] Iteratively optimize the first type of nodes in the graph model.

[0032] Optionally, the second determining unit is specifically configured to:

[0033] When the pixel error value between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is less than a pixel error threshold, or the proportion of the pixel error value in the pixel value of the actual feature corner point distribution of the next frame image is less than a proportion threshold, it is determined that the pixel error meets a preset error condition.

[0034] Optionally, the second determining unit is specifically configured to:

[0035] When the pixel error value is greater than or equal to a pixel error threshold, or the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is greater than or equal to a proportion threshold, the initial camera pose matrix is ​​iteratively tuned according to the pixel error between the real feature corner point distribution of the next frame image and the predicted feature corner point distribution.

[0036] Optionally, the second determining unit is specifically configured to:

[0037] When the number of iterations meets a preset number of iterations or the cost function reaches a minimum value, it is determined that the iterative cost function meets the iteration condition.

[0038] Optionally, the second determining unit is specifically configured to:

[0039] Sort the pixels in the real feature corner point distribution and the predicted feature corner point distribution according to the pixel coordinates;

[0040] After the pixels are sorted, the pixel error value between each mapping point pair between the real feature corner point distribution and the predicted feature corner point distribution is calculated;

[0041] The mean of the error values ​​is used as the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image.

[0042] Optionally, the device further comprises:

[0043] A second acquisition unit, used to acquire a current frame image and a next frame image taken by the vehicle-mounted camera;

[0044] The obtaining unit is used to perform scale-invariant feature transformation detection on the current frame image and the next frame image respectively to obtain the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image.

[0045] In a third aspect, an embodiment of the present application provides a device, comprising a memory and a processor, wherein the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes any method described in the first aspect.

[0046] In a fourth aspect, an embodiment of the present application provides a computer storage medium, in which a code is stored. When the code is executed, a device executing the code implements the method described in any one of the first aspects above.

[0047] Compared with the prior art, this application has the following beneficial effects:

[0048] In the embodiment of the present application, the real feature corner distribution of the current frame image taken by the vehicle-mounted camera and the real feature corner distribution of the next frame image can be obtained, and the predicted feature corner distribution of the next frame image is determined according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner distribution of the current frame image. According to the pixel error between the real feature corner distribution of the next frame image and the predicted feature corner distribution, the initial camera pose matrix is ​​iteratively tuned, and the pixel error is updated according to the camera pose matrix after this tuning in each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iterative condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix. In this way, the whole process can be completed automatically without human intervention, thereby avoiding the problem that manual on-site operations are affected by the environment of the calibration site. In addition, since there is no need for manual personnel to carry professional equipment to perform the camera calibration process, the camera calibration process can be simplified, and certain costs can be reduced, saving resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of a camera calibration method provided in an embodiment of the present application;

[0051] Figure 2 A schematic structural diagram of a specific implementation of the camera calibration device provided in this application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] It should be noted that the camera calibration method and device provided in this application are used in the field of image processing technology. The above is only an example and does not limit the application field of the method and device provided in this application.

[0054] Surround view camera system is an advanced driver assistance system that provides a panoramic view of the vehicle's surroundings by installing cameras around the vehicle. This system can significantly improve the driver's awareness of the vehicle's surroundings, especially when parking, reversing or driving in a narrow space.

[0055] At present, vehicles are generally equipped with an on-board surround view camera system. In some cases, such as when the vehicle encounters a collision, scratch, or bump during use, the posture of the surround view camera may change, and the factory camera calibration file is no longer applicable, so the surround view camera needs to be recalibrated.

[0056] Existing calibration methods generally require professional technicians to carry professional calibration equipment for on-site operations, which are affected by the environment of the calibration site and the entire camera calibration process is relatively cumbersome. The cost of manual camera calibration is relatively high.

[0057] In view of this, after research, the technical solution of this application is proposed. This application provides a camera calibration method and device, which can obtain the real feature corner point distribution of the current frame image taken by the vehicle-mounted camera and the real feature corner point distribution of the next frame image, determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner point distribution of the current frame image, and iteratively tune the initial camera pose matrix according to the pixel error between the real feature corner point distribution of the next frame image and the predicted feature corner point distribution. In each tuning process, the pixel error is updated according to the camera pose matrix after this tuning, until the pixel error meets the preset error condition or the iterative cost function meets the iterative condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix. In this way, the whole process can be completed automatically without manual participation, thereby avoiding the problem that manual on-site operations are affected by the environment of the calibration site. In addition, since there is no need for manual personnel to carry professional equipment for the camera calibration process, the camera calibration process can be simplified, and certain costs can be reduced, saving resources.

[0058] The method provided in the embodiment of the present application may be executed by software on a terminal device. The terminal device may be, for example, a mobile phone, a tablet computer, a computer, etc. The software may be, for example, system software.

[0059] In order to make the technical personnel in this field better understand the present application, the present application is further described in detail below in combination with the drawings and specific implementation methods. The method provided in the present application embodiment is described below by taking the first device as an example.

[0060] Figure 1 A flowchart of a camera calibration method provided in an embodiment of the present application.

[0061] like Figure 1 As shown, the method includes:

[0062] S101: Obtain the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image taken by the vehicle-mounted camera.

[0063] The first device obtains the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image taken by the vehicle-mounted camera.

[0064] Specifically, the first device obtains the current frame image and the next frame image taken by the vehicle-mounted camera, and then uses the Scale-Invariant Feature Transform (SIFT) feature detection algorithm to calculate the feature corner point distribution of the current frame image and the feature corner point distribution of the next frame image. Among them, SIFT feature detection is an algorithm for image feature detection, which is invariant to transformations such as illumination, scale, and rotation. Feature corner point distribution refers to the spatial distribution of key points (corner points) found in the image by feature detection algorithms (such as SIFT).

[0065] Furthermore, the specific process of calculating the characteristic corner point distribution of the image (the current frame image and the next frame image) by using SIFT feature detection can be as follows:

[0066] First, the scale space of the image is constructed, which is usually achieved through the Gaussian pyramid. The Gaussian pyramid creates different scale versions of the image through continuous Gaussian blurring and downsampling. Then the key points are detected. The Gaussian difference (DoG) is used on each scale image to approximate the Gaussian Laplacian operator to detect potential key points. Among them, the key points are points with local maximum or minimum responses, which are extreme values ​​at the same scale of the image and between the upper and lower scales. Then the key point localization is performed to accurately locate the detected key points. For example, the Taylor series expansion can be performed on the pixels in the neighborhood of each key point to determine the exact position and scale of the key point. Then the edge response is removed using the contrast threshold and edge threshold to remove low contrast and unstable edge response points. After that, the direction assignment is performed to assign a main direction to each key point, for example, by counting the gradient direction of the pixels in the neighborhood of the key point. The direction with the highest response is selected as the main direction of the key point. Then the feature descriptor is generated. For each key point, a 128-dimensional SIFT feature descriptor is generated. This usually involves measuring the local gradient of the image in the neighborhood around the key point and encoding this gradient information into a vector. Finally, by analyzing the distribution of feature points in the image, we can calculate the density of feature points, identify the area where feature points are concentrated, or calculate the spatial distribution pattern of feature points to obtain the feature corner point distribution. The calculated feature angle distribution can be called the true feature angle distribution. The true feature corner point distribution of the current frame image and the true feature corner point distribution of the next frame image can be determined in the above way.

[0067] S102: Determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the actual feature corner point distribution of the current frame image.

[0068] The first device can determine the predicted feature corner point distribution of the next frame image based on the initial camera pose matrix of the vehicle-mounted camera and the actual feature corner point distribution of the current frame image, wherein the initial camera pose matrix refers to the camera pose matrix calibrated when the vehicle-mounted camera leaves the factory, and the camera pose matrix refers to the matrix that describes the position and direction of the camera in three-dimensional space, which can include the rotation information and translation information of the camera.

[0069] Specifically, the depth information or plane hypothesis of the current frame image can be used to predict the positions of these corner points in 3D space. According to the change of camera pose, the same transformation is applied to the 3D corner point positions in the current frame image to predict their new positions in the next frame image. For each corner point P, its position P′ in the next frame can be calculated by the following formula:

[0070] P′=R·P+t

[0071] Among them, R is the rotation matrix, t is the translation vector, and then the camera intrinsic parameter matrix of the next frame image (which may include focal length and principal point coordinates) can be used to project the transformed three-dimensional point onto the two-dimensional image plane.

[0072] For example, the perspective projection model of the camera can be used, and the formula can be as follows:

[0073] [u,v]=K·[R·P+t] / w

[0074] Among them, [u, v] is the coordinate on the image plane of the next frame, K is the intrinsic parameter matrix of the camera, and w is the depth information in homogeneous coordinates.

[0075] In some possible implementations, the scale and rotation of feature points may change in the next frame due to camera zoom and translation. If the camera pose matrix includes these changes, the SIFT feature description needs to be adjusted accordingly.

[0076] Therefore, the predicted feature corner point distribution of the next frame image can be determined by using the above method. Here, the predicted feature corner point distribution of the next frame image can be called predicted corner point distribution.

[0077] S103: According to the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, the initial camera pose matrix is ​​iteratively tuned, and in each tuning process, the pixel error is updated according to the camera pose matrix after this tuning, until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

[0078] After determining the predicted feature corner point distribution, the first device may determine a pixel error between the actual feature corner point distribution and the predicted feature corner point distribution of the next frame image.

[0079] Exemplarily, the pixels in the real feature corner point distribution and the predicted feature corner point distribution can be sorted according to the pixel coordinates to ensure one-to-one correspondence between the mapping point pairs, and then the pixel error value between each mapping point pair between the real feature corner point distribution and the predicted feature corner point distribution can be calculated. For example, if there are 100 mapping point pairs between the real feature corner point distribution and the predicted feature corner point distribution, the pixel error values ​​(also called pixel error values) between these 100 feature point pairs can be calculated respectively, and finally the average of these error values ​​is used as the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the final next frame image.

[0080] After determining the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, the initial camera pose matrix can be iteratively tuned according to the pixel error, and the pixel error is updated according to the camera pose matrix after the current tuning process in each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

[0081] Here, the pixel error satisfies the preset error condition when the pixel error value between the predicted feature corner point distribution and the real feature corner point distribution of the next frame image is less than the pixel error threshold, or the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is less than the proportion threshold, then it can be determined that the pixel error satisfies the preset error condition, otherwise it is determined that the pixel error does not satisfy the preset error condition, wherein the pixel value of the real feature corner point distribution can be the mean of the pixel points in the real feature corner point distribution. The iterative cost function satisfies the iteration condition when the number of iterations satisfies the preset number of iterations or the cost function reaches the minimum value, wherein the cost function is used to measure the difference between the current camera pose and the target camera pose, and the smaller the cost function is, the smaller the difference between the current camera pose and the target camera pose is.

[0082] In addition, it should be noted that the pixel error threshold, ratio threshold and preset number of iterations here can be set according to actual needs and are not limited here.

[0083] Exemplarily, the camera pose matrix can be optimized by a graph optimization method according to the pixel error value between the predicted feature corner point distribution and the actual feature corner point distribution.

[0084] Specifically, a graph model can be constructed by using global nearest neighbors or relaxed labels, etc. The graph model includes first-class nodes, second-class nodes, and edges. The first-class nodes are used to represent the camera pose matrix, and the second-class nodes are used to represent the coordinates of feature points detected in the three-dimensional space. The first-class nodes and the second-class nodes are connected by edges, which are used to represent the pixel positions projected from the three-dimensional feature points to the two-dimensional image plane, and reflect the relationship between the displacement and rotation of the camera pose matrix and the feature point projection error;

[0085] Then determine the appropriate graph optimization algorithm, such as gradient descent, nonlinear least squares method, etc., execute the optimization algorithm, iteratively optimize the first type of nodes in the graph model to adjust the camera pose matrix to minimize the residuals of all edges, that is, minimize the cost function, and update the pixel error according to the camera pose matrix after this tuning during each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the camera pose matrix corresponding to this iteration is indeed the target camera pose matrix.

[0086] In some possible implementations, before iteratively tuning the initial camera pose matrix, in order to save resources and improve the efficiency of camera calibration, an additional step can be performed to determine whether the pixel error meets a preset error condition, that is, to determine whether the pixel error value is less than a pixel error threshold, or whether the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is less than a proportion threshold. If the pixel error value is greater than or equal to the pixel error threshold, or the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is greater than or equal to the proportion threshold, it can be determined that the error is large. At this time, there is an inaccurate problem in the camera pose matrix, and then the step of iteratively tuning the initial camera pose matrix is ​​performed.

[0087] In the embodiment of the present application, the real feature corner distribution of the current frame image taken by the vehicle-mounted camera and the real feature corner distribution of the next frame image can be obtained, and the predicted feature corner distribution of the next frame image is determined according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner distribution of the current frame image. According to the pixel error between the real feature corner distribution of the next frame image and the predicted feature corner distribution, the initial camera pose matrix is ​​iteratively tuned, and the pixel error is updated according to the camera pose matrix after this tuning in each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iterative condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix. In this way, the whole process can be completed automatically without human intervention, thereby avoiding the problem that manual on-site operations are affected by the environment of the calibration site. In addition, since there is no need for manual personnel to carry professional equipment to perform the camera calibration process, the camera calibration process can be simplified, and certain costs can be reduced, saving resources.

[0088] The above are some specific implementations of the camera calibration method provided in the embodiments of the present application. Based on this, the present application also provides a corresponding device. The device provided in the embodiments of the present application will be introduced from the perspective of functional modularization. The device and the camera calibration method described above can be referred to in correspondence with each other.

[0089] Figure 2 The structural block diagram provided for the embodiment of the present invention is referred to as the specific implementation mode 2, and the Figure 2The device may include:

[0090] The first acquisition unit 210 is used to acquire the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image taken by the vehicle-mounted camera;

[0091] A first determining unit 220, configured to determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner point distribution of the current frame image;

[0092] The second determination unit 230 is used to iteratively tune the initial camera pose matrix according to the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, and update the pixel error according to the camera pose matrix after the current tuning in each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

[0093] Optionally, the second determining unit is specifically configured to:

[0094] According to the pixel error, construct a graph model, the graph model including a first type of node, a second type of node and an edge, the first type of node is used to represent a camera pose matrix, the second type of node is used to represent the coordinates of feature points detected in a three-dimensional space, the first type of node and the second type of node are connected by the edge, and are used to represent the pixel position projected from the three-dimensional feature point to the two-dimensional image plane and reflect the relationship between the displacement and rotation of the camera pose matrix and the feature point projection error;

[0095] Iteratively optimize the first type of nodes in the graph model.

[0096] Optionally, the second determining unit is specifically configured to:

[0097] When the pixel error value between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is less than a pixel error threshold, or the proportion of the pixel error value in the pixel value of the actual feature corner point distribution of the next frame image is less than a proportion threshold, it is determined that the pixel error meets a preset error condition.

[0098] Optionally, the second determining unit is specifically configured to:

[0099] When the pixel error value is greater than or equal to a pixel error threshold, or the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is greater than or equal to a proportion threshold, the initial camera pose matrix is ​​iteratively tuned according to the pixel error between the real feature corner point distribution of the next frame image and the predicted feature corner point distribution.

[0100] Optionally, the second determining unit is specifically configured to:

[0101] When the number of iterations meets a preset number of iterations or the cost function reaches a minimum value, it is determined that the iterative cost function meets the iteration condition.

[0102] Optionally, the second determining unit is specifically configured to:

[0103] Sort the pixels in the real feature corner point distribution and the predicted feature corner point distribution according to the pixel coordinates;

[0104] After the pixels are sorted, the pixel error value between each mapping point pair between the real feature corner point distribution and the predicted feature corner point distribution is calculated;

[0105] The mean of the error values ​​is used as the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image.

[0106] Optionally, the device further comprises:

[0107] A second acquisition unit, used to acquire a current frame image and a next frame image taken by the vehicle-mounted camera;

[0108] The obtaining unit is used to perform scale-invariant feature transformation detection on the current frame image and the next frame image respectively to obtain the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image.

[0109] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the solutions provided by the embodiments of the present application.

[0110] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the method described in any embodiment of the present application.

[0111] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the method described in any embodiment of the present application.

[0112] The "first" and "second" in the names such as "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.

[0113] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0114] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0115] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A camera calibration method, characterized in that: include: Obtain the real feature corner point distribution of the current frame image taken by the vehicle-mounted camera and the real feature corner point distribution of the next frame image; Determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner point distribution of the current frame image; According to the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, the initial camera pose matrix is ​​iteratively tuned, and in each tuning process, the pixel error is updated according to the camera pose matrix after this tuning until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

2. The method according to claim 1, characterized in that According to the pixel error, the initial camera pose matrix is ​​iteratively tuned, including: According to the pixel error, construct a graph model, the graph model including a first type of node, a second type of node and an edge, the first type of node is used to represent a camera pose matrix, the second type of node is used to represent the coordinates of feature points detected in a three-dimensional space, the first type of node and the second type of node are connected by the edge, and are used to represent the pixel position projected from the three-dimensional feature point to the two-dimensional image plane and reflect the relationship between the displacement and rotation of the camera pose matrix and the feature point projection error; Iteratively optimize the first type of nodes in the graph model.

3. The method according to claim 1, characterized in that The pixel error satisfies a preset error condition, including: When the pixel error value between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is less than a pixel error threshold, or the proportion of the pixel error value in the pixel value of the actual feature corner point distribution of the next frame image is less than a proportion threshold, it is determined that the pixel error meets a preset error condition.

4. The method according to claim 3, characterized in that When the pixel error value is greater than or equal to a pixel error threshold, or the proportion of the pixel error value in the pixel value of the real feature corner point distribution of the next frame image is greater than or equal to a proportion threshold, the initial camera pose matrix is ​​iteratively tuned according to the pixel error between the real feature corner point distribution of the next frame image and the predicted feature corner point distribution.

5. The method according to claim 1, characterized in that: The iterative cost function satisfies the iterative condition, including: When the number of iterations meets a preset number of iterations or the cost function reaches a minimum value, it is determined that the iterative cost function meets the iteration condition.

6. The method according to claim 1, characterized in that The pixel error between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is determined by: Sort the pixels in the real feature corner point distribution and the predicted feature corner point distribution according to the pixel coordinates; After the pixels are sorted, the pixel error value between each mapping point pair between the real feature corner point distribution and the predicted feature corner point distribution is calculated; The mean of the error values ​​is used as the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Get the current frame image and the next frame image taken by the vehicle camera; Scale-invariant feature transformation detection is performed on the current frame image and the next frame image respectively to obtain the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image.

8. A camera calibration device, characterized in that: include: A first acquisition unit is used to acquire the real feature corner point distribution of the current frame image and the real feature corner point distribution of the next frame image taken by the vehicle-mounted camera; A first determining unit, configured to determine the predicted feature corner point distribution of the next frame image according to the initial camera pose matrix of the vehicle-mounted camera and the real feature corner point distribution of the current frame image; The second determination unit is used to iteratively tune the initial camera pose matrix according to the pixel error between the real feature corner point distribution and the predicted feature corner point distribution of the next frame image, and update the pixel error according to the camera pose matrix after the current tuning in each tuning process until the pixel error meets the preset error condition or the iterative cost function meets the iteration condition, and the corresponding camera pose matrix is ​​indeed the target camera pose matrix.

9. The device according to claim 8, characterized in that The second determining unit is specifically configured to: According to the pixel error, construct a graph model, the graph model including a first type of node, a second type of node and an edge, the first type of node is used to represent a camera pose matrix, the second type of node is used to represent the coordinates of feature points detected in a three-dimensional space, the first type of node and the second type of node are connected by the edge, and are used to represent the pixel position projected from the three-dimensional feature point to the two-dimensional image plane and reflect the relationship between the displacement and rotation of the camera pose matrix and the feature point projection error; Iteratively optimize the first type of nodes in the graph model.

10. The device according to claim 8, characterized in that The second determining unit is specifically configured to: When the pixel error value between the predicted feature corner point distribution and the actual feature corner point distribution of the next frame image is less than a pixel error threshold, or the proportion of the pixel error value in the pixel value of the actual feature corner point distribution of the next frame image is less than a proportion threshold, it is determined that the pixel error meets a preset error condition.

Images