Camera extrinsic parameter determination method, device and equipment, and computer storage medium

Abstract

The application discloses a camera extrinsic parameter determination method, device and equipment, and a computer storage medium. The method comprises the following steps: acquiring a first pose information set of a camera arranged on a vehicle within a first preset time period; acquiring a warehouse location image captured by the camera within the first preset time period; determining prior information corresponding to the first pose information set according to the warehouse location image; acquiring a second pose information set of the vehicle within the first preset time period; and determining the extrinsic parameter of the camera by using the first pose information set, the prior information and the second pose information set. The method can improve the determination efficiency of the camera extrinsic parameter determination.

Description

Technical Field

[0001] This application belongs to the field of positioning technology, and in particular relates to a method, apparatus, device for determining camera extrinsic parameters, and computer storage medium. Background Technology

[0002] In technologies such as autonomous driving navigation and high-precision maps, surround-view cameras are widely used in vehicle mapping and localization because they can provide a 360-degree field of view around the vehicle. In order to effectively integrate the perception data of the cameras, it is necessary to provide a unified geometric benchmark for the perception measurement values ​​under different camera coordinate systems, so as to map the perception elements of different cameras into the 360-degree surround-view image.

[0003] In related technologies, a coordinate system is generally established with the center of the rear wheel of the vehicle as the origin. By accurately solving the extrinsic parameter relationships between different camera coordinate systems relative to the vehicle coordinate system, the geometric accuracy of vehicle camera perception fusion is ensured. Currently, the common method for calibrating camera extrinsic parameters is to utilize prior environmental information, pre-select a calibration site, manually mark multiple control points or create artificial target boards, and determine the pose relationships of different cameras relative to the calibration field based on the positions of the control points in the camera's field of view. The pose relationship results are related to the distribution of control points in the camera's field of view, thus requiring manual intervention during the operation, which brings considerable inconvenience to practical applications. The calibration method for camera extrinsic parameters is cumbersome and has low calibration efficiency. Summary of the Invention

[0004] This application provides an implementation scheme that differs from the prior art, in order to solve the technical problem of low determination efficiency in camera extrinsic parameter determination methods in related technologies.

[0005] In a first aspect, this application provides a method for determining camera extrinsic parameters, including:

[0006] Acquire the first pose information set of the camera set on the vehicle within the first preset time period;

[0007] Acquire the warehouse location images captured by the camera within the first preset time period;

[0008] Determine the prior information corresponding to the first pose information set based on the first pose information set and the library location image;

[0009] Obtain the second pose information set of the vehicle within the first preset time period;

[0010] The extrinsic parameters of the camera are determined using the first pose information set, the prior information, and the second pose information set.

[0011] Wherein, any first pose information in the first pose information set includes three-dimensional position information, as well as first pitch angle information, first yaw angle information, and first roll angle information corresponding to the three-dimensional position information; the prior information includes second pitch angle information, second roll angle information, and height information of the camera relative to the reference plane corresponding to the storage location image.

[0012] Secondly, this application provides a camera extrinsic parameter determination device, comprising:

[0013] The first acquisition module is used to acquire the first pose information set of the camera set on the vehicle within a first preset time period;

[0014] The second acquisition module is used to acquire images of the storage location captured by the camera within the first preset time period;

[0015] The first determining module is used to determine the prior information corresponding to the first pose information set based on the first pose information set and the library location image.

[0016] The third acquisition module is used to acquire the second pose information set of the vehicle within the first preset time period;

[0017] The second determining module is used to determine the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set.

[0018] Wherein, any first pose information in the first pose information set includes three-dimensional position information, and the first pitch angle information, first yaw angle information, and first roll angle information corresponding to the three-dimensional position information; the prior information includes second pitch angle information, second roll angle information, and the height information of the camera from the reference plane corresponding to the storage location image. Thirdly, this application provides an electronic device, including:

[0019] Processor; and

[0020] Memory for storing the executable instructions of the processor;

[0021] The processor is configured to execute the method described in the first aspect or any of the possible embodiments of the first aspect by executing the executable instructions.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or any of the possible implementations of the first aspect.

[0023] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect or any of the possible implementations of the first aspect.

[0024] This application improves the efficiency of determining camera extrinsic parameters by acquiring a first pose information set of a camera installed on a vehicle within a first preset time period; acquiring a warehouse location image captured by the camera within the first preset time period; determining prior information corresponding to the first pose information set based on the first pose information set and the warehouse location image; acquiring a second pose information set of the vehicle within the first preset time period; and determining the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set. Each first pose information set in the first pose information set includes three-dimensional position information, and the corresponding first pitch angle, first yaw angle, and first roll angle information. The prior information includes second pitch angle information, second roll angle information, and the height information of the camera from the reference plane corresponding to the warehouse location image. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0026] Figure 1a A schematic flowchart illustrating a camera extrinsic parameter determination method provided in an embodiment of this application;

[0027] Figure 1b This is a schematic diagram illustrating the principle of determining the second pitch angle information and the second roll angle information according to an embodiment of this application;

[0028] Figure 2 This is a schematic diagram of the structure of a camera extrinsic parameter determination device provided in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application are described in detail below, with examples of the embodiments illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0031] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] First, some terms used in the embodiments of this application will be explained below to facilitate understanding by those skilled in the art.

[0033] Decoupling: In mathematics, decoupling refers to transforming a mathematical equation containing multiple variables into a system of equations that can be represented by a single variable. This means that the variables no longer simultaneously and directly affect the result of a single equation, thus simplifying analysis and calculation. By appropriately selecting control quantities and employing coordinate transformations, a multivariate system is systematized into a mathematical model of multiple independent single-variable systems, effectively decoupling the variables.

[0034] This application mainly introduces an adaptive camera extrinsic parameter calibration method suitable for vehicle motion platforms. The entire process requires no manual intervention and makes full use of the geometric structured information perceived from the parking lot scene. It decouples the extrinsic parameters by considering the constraints on the camera extrinsic parameters based on the characteristics of vehicle motion, and establishes constraint cost functions accordingly. This method does not require additional control fields or calibration boards. Data acquisition and parameter calculation can be completed online automatically, which improves the efficiency of camera extrinsic parameter determination. The redundant observation of a large amount of data further increases the reliability of the calibration algorithm itself.

[0035] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0036] Figure 1a A flowchart illustrating a camera extrinsic parameter determination method provided as an exemplary embodiment of this application is shown. This method is applicable to cameras or other electronic devices and includes at least the following steps:

[0037] S101. Obtain the first pose information set of the camera set on the vehicle within a first preset time period;

[0038] S102. Obtain the warehouse location image captured by the camera within the first preset time period;

[0039] S103. Determine the prior information corresponding to the first pose information set based on the storage location image;

[0040] S104. Obtain the second pose information set of the vehicle within the first preset time period;

[0041] S105. Determine the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set;

[0042] Wherein, any first pose information in the first pose information set includes three-dimensional position information, as well as first pitch angle information, first yaw angle information, and first roll angle information corresponding to the three-dimensional position information; the prior information includes second pitch angle information, second roll angle information, and height information of the camera relative to the reference plane corresponding to the storage location image.

[0043] Specifically, the first preset time period can be the sampling time period of the camera sampling environment image. The first preset time period is divided into multiple first sub-time periods by each sampling time. The pose information of the camera at each sampling time within the first preset time period is the first pose information corresponding to the current sampling time. The first pose information corresponding to each sampling time constitutes the first pose information set of the camera within the first preset time period.

[0044] Furthermore, each first pose information may include three-dimensional position information, as well as the first roll angle information, the first pitch angle information, and the first yaw angle information corresponding to the three-dimensional position information; wherein, the three-dimensional position information is the three-dimensional coordinate information of the camera relative to the camera's own coordinate system, and the first roll angle information, the first pitch angle information, and the first yaw angle information are the attitude angle information of the camera relative to the camera's own coordinate system.

[0045] For example, the first pose information is: (x,y,z,roll,pitch,yaw), where x, y, and z are the three-dimensional coordinate information of the camera, roll is the first roll angle information, pitch is the first pitch angle information, and yaw is the first yaw angle information.

[0046] Furthermore, the first pose information is the pose information of the camera relative to its initial position, which is the initial time of the first preset time period, the position of the camera. When the camera is in its initial position, its first pose information is (0, 0, 0, 0, 0, 0).

[0047] Furthermore, the aforementioned first pose information can be received from a visual odometry system (which can be a monocular visual odometry system). Accordingly, the first pose information set of the camera installed on the vehicle within a first preset time period includes:

[0048] Acquire the set of environmental images captured by the camera within the first preset time period;

[0049] The environmental image set is input into the visual odometry, so that the visual odometry determines the first pose information set based on the environmental image set;

[0050] Receive the first pose information set.

[0051] Visual odometry determines the first pose information set of the camera in the first preset time period based on the environmental image set, specifically including:

[0052] Visual odometry calculates the relative positional relationship between adjacent image frames by measuring the positional changes of feature points in each scene image within a scene image set. It progressively infers the current position of the camera by matching and associating features from the camera's sequential images. Considering that the trajectory calculated by monocular visual odometry is free-scale, it generally needs to be used in conjunction with other scale-constrained sensors (such as inertial measurement units and LiDAR). In this scheme, the absolute scale is determined by the scale constraint of the wheel speed signal. Optionally, the classic ORB-SLAM2 algorithm (or other similar monocular visual odometry methods) can be used to correlate common-view features within a sliding window, progressively calculating the first pose information of each monocular camera in the surround-view system at each sampling time, thereby determining the first pose information set.

[0053] Furthermore, acquiring images of the storage location captured by the camera within the first preset time period includes:

[0054] Acquire the set of environmental images captured by the camera within the first preset time period;

[0055] Images of storage locations are selected from the set of environmental images.

[0056] Specifically, the environmental image set can be input into a preset screening model, and the storage location image can be determined based on the output of the screening model. The screening model can be a neural network model trained on multiple sets of sample data. In addition, the method of screening storage location images from the environmental image set can also be implemented based on other existing technologies, which are not limited in this application.

[0057] Among them, the storage location image is an image that contains storage location information, such as an image that contains the edge line information of the storage location.

[0058] Further, in the aforementioned step S103, determining the prior information corresponding to the first pose information set based on the library location image includes:

[0059] S1031. Determine the second pitch angle information and the second roll angle information of the camera relative to the reference plane corresponding to the storage location image based on the storage location image;

[0060] S1032. Determine the height information of the camera from the reference plane using the second pitch angle information, the second roll angle information, and the storage location image;

[0061] S1033. Determine the prior information using the second pitch angle information, the second roll angle information, and the altitude information.

[0062] Specifically, the number of the aforementioned storage location images can be one or more, and the reference plane corresponding to the storage location image is the plane where the storage location corresponding to the storage location image is located, i.e., the ground.

[0063] In some optional embodiments of this application, a storage location image can determine a second pitch angle and a second roll angle, as follows:

[0064] First, the acquired raw silo image is preprocessed to remove geometric distortion and extract the structured information of the silo elements. The attitude parameters are then solved using the geometric configuration of two parallel lines and a line perpendicular to and coplanar with these parallel lines. Based on the specific geometric constraints of the perceived silo structured elements, the attitude parameters are solved through the orthogonality of the hidden point and the attitude rotation matrix. Assuming the unit direction vector is... The vanishing point of a straight line in three-dimensional space is through the center of the camera and its direction is... The intersection point v of the ray and the image plane is expressed by the formula:

[0065]

[0066] Here, `proj` is the camera projection matrix, and the vanishing point is the image at infinity, unaffected by changes in camera position but affected by camera rotation. Therefore, for a moving camera, let the rotation matrix between the camera coordinate system and the world coordinate system at a certain moment be...

[0067] Then formula (1.1) can be rewritten as:

[0068]

[0069] Combination Figure 1b It can be seen that the unit vector of the exit marker line L1 of the storage location and the marker lines L2 and L3 at both ends of the storage location is denoted as... and

[0070] in:

[0071]

[0072] Correspondingly, the vanishing point expressions for formula (1) are as follows:

[0073]

[0074] Further Expressed in column vector form, we get:

[0075]

[0076] The vanishing point v2 is the intersection of lines l2 and l3 in the image plane, i.e., v2 = l2 × l3. Combining formulas (1.4) and (1.5), we can obtain:

[0077]

[0078] Considering

[0079]

[0080] Combining formulas (1.6) and (1.7), we get:

[0081] v1 T (proj -T proj -1 v2 = 0 (1.8)

[0082] at the same time

[0083] v1·l1=0 (1.9)

[0084] Combining formulas (1.8) and (1.9), we can derive:

[0085]

[0086] Therefore, substituting the vanishing points v1 and v2 into (1.5) allows us to calculate... and Based on the orthogonality property of rotation matrices, it can be calculated Further decomposition of the attitude matrix yields x roll x pitch and x yaw That is, to obtain the tilt information of the camera relative to the ground reference plane (x pitch x roll The tilt information includes a second pitch angle and a second roll angle.

[0087] Furthermore, when there are multiple selected images of the storage location, multiple second pitch angle information and multiple second roll angle information can be determined. Finally, the second pitch angle information and second roll angle information used to determine the height information of multiple cameras from the reference plane can be the average value of the aforementioned multiple second pitch angle information and the average value of the multiple second roll angle information, respectively.

[0088] Specifically, the aforementioned determination of the camera's height from the reference plane using the second pitch angle information, the second roll angle information, and the storage location image may specifically include:

[0089] The corresponding reference plane projection transformation matrix is ​​determined based on the second pitch angle information and the second roll angle information;

[0090] The height information of the camera from the reference plane is determined based on the reference plane projection transformation matrix and the location image.

[0091] The corresponding reference plane projection transformation matrix, determined based on the second pitch angle information and the second roll angle information, can be achieved using the following formula (2.1):

[0092] R xy =R y (x pitch )R x (x roll (2.1)

[0093] Among them, R xy R is the projection transformation matrix of the reference plane. y (x pitch R is the rotation matrix corresponding to the second pitch angle information. x (x roll ) is the rotation matrix corresponding to the second roll angle information.

[0094] Furthermore, the height information of the camera from the reference plane can be determined based on the reference plane projection transformation matrix and the location image using the following formulas (2.2) to (2.5):

[0095] x best =argminξ(h) (2.2)

[0096]

[0097]

[0098] t = [0, 0, h] T (2.5)

[0099] Where h is the height parameter of the height information to be determined, R is the projection transformation matrix of the reference plane, xbest is the height information to be determined, mj is the 3D scene coordinates of the j-th feature point in the location image relative to the starting frame position (calculated by the ORB-SLAM2 algorithm), and η j (h) represents the normal distance from the j-th feature point in the storage location image to the reference ground plane, w j The weights represent the corresponding points. The weights can be determined using the Huber kernel function strategy based on the error magnitude. n represents the normal vector of the reference plane, and D represents the distance from the feature point along the normal vector to the reference plane. n = (0, 0, 1) and D = 0. Let be the coordinates of the j-th feature point projected onto the reference plane. The number of feature points in the storage location image can be determined based on the recognition results of the storage location image; this application does not impose any limitation on this.

[0100] Specifically, when multiple storage location images are selected from the environmental image set, the storage location image used to determine the camera's height information from the reference plane is any one of the multiple storage location images. After determining the storage location image used to determine the camera's height information from the reference plane, the second pitch angle information and the second roll angle information are the second pitch angle information and the second roll angle information corresponding to that storage location image.

[0101] Further, obtaining the second pose information set of the vehicle within the first preset time period includes:

[0102] Obtain the rotation number of each of the two rear wheels of the vehicle in each of the second sub-time periods within the first preset time period;

[0103] By using a preset differential motion model and the rotation number information of each of the two rear wheels in each second sub-time period, the second pose information corresponding to the start and end times of each second sub-time period is determined, thus obtaining the second pose information set of the vehicle in the first preset time period.

[0104] The second pose information is the pose information of the vehicle relative to its own coordinate system. The second pose information is the pose information of the vehicle relative to its initial position (i.e., the position of the vehicle at the initial moment of the first preset time period). The second pose information may specifically include: two-dimensional coordinate information and vehicle yaw angle information, such as (x, y, yaw).

[0105] Furthermore, each of the aforementioned second sub-time periods is a sub-time period divided into the time of collecting the rotation number information of the two rear wheels within the first preset time period.

[0106] Further, step S105 above, determining the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set, includes:

[0107] S1051. Determine the first motion increment information corresponding to each first sub-time period of the camera within the first preset time period based on the first pose information set, and obtain the first motion increment information set corresponding to the camera within the first preset time period.

[0108] S1052. The reference motion increment information set is determined by using the reference plane projection transformation matrix corresponding to the prior information and the first motion increment information set.

[0109] S1053. Determine the second motion increment information set corresponding to the vehicle in the first preset time period based on the second pose information set;

[0110] S1054. Determine the extrinsic parameters of the camera based on the second motion increment information set corresponding to the reference motion increment information set and the second pose information set;

[0111] The first sub-time period is the time period between adjacent sampling times corresponding to the first pose information set.

[0112] Furthermore, the first motion increment information corresponding to each first sub-time period of the camera within the first preset time period is determined based on the first pose information set. The first motion increment information set corresponding to the camera within the first preset time period can be obtained through the following formula (2.6):

[0113]

[0114] Where, q k The first pose information at sampling time k, and q k-1 p is the first pose information of the sampling time preceding time k. k The first motion increment information refers to the first sampling time k. The value of k corresponds one-to-one with the sampling time within the first preset time period. After obtaining the first motion increment information corresponding to each first sampling time, the first motion increment information set corresponding to the first preset time period can be obtained. The first motion increment information set is a set p composed of multiple first motion increment information sets. k (R0 k , t k )∈SE(3), where R0 k For the attitude angle component in the corresponding first motion increment information, t k These are the corresponding displacement components.

[0115] Furthermore, the aforementioned determination of the reference motion increment information set using the reference plane projection transformation matrix and the first motion increment information set can be achieved through the following formulas (2.7) and (2.8):

[0116]

[0117]

[0118] Where R is the projection transformation matrix of the reference plane, and R0 k For the attitude angle component in the corresponding first motion increment information, t k For the corresponding displacement component, i.e., the first motion increment information p k For (R0) k , t k ). The foregoing and Reference motion increment information The attitude angle components in, and displacement components in

[0119] The method of determining the second motion increment information set of the vehicle within the first preset time period based on the second pose information set is similar to the method of determining the first motion increment information set of the camera within the first preset time period based on the first pose information set, and will not be described in detail here.

[0120] Further, step S1054, which involves determining the camera's extrinsic parameters based on the reference motion increment information set and the second motion increment information set, includes:

[0121] The reference motion increment information set and the second motion increment information set are time-aligned to obtain the alignment result;

[0122] The alignment results are used to determine the extrinsic parameters of the camera.

[0123] Specifically, interpolation can be used to time-align the reference motion increment information set with the second motion increment information set, and the alignment result is the time-aligned reference motion increment information set. With the second motion increment information set The foregoing and These are the attitude angle increment information and displacement increment information in the second motion increment information set, respectively.

[0124] Furthermore, determining the camera's extrinsic parameters using the alignment results can specifically include:

[0125] The alignment results are used to determine the transformation relationship between the camera's coordinate space and the vehicle's coordinate space;

[0126] The extrinsic parameters of the camera are determined based on the prior information according to the transformation relationship.

[0127] Specifically, the transformation relationship between the camera's coordinate space and the vehicle's coordinate space can be determined using the following formula (2.9):

[0128]

[0129]

[0130] in, and These represent the rotation components of the environment image from the i-th frame to the (i+1)-th frame (as determined by...). Determine) and translation components (by) Sure), and These represent the rotational components of the vehicle at the corresponding time points (by...). Determine) and translation components (by) Sure), express The corresponding rotation matrix, I represents the 3×3 identity matrix, and sj represents the scaling factor. When the rotation matrix is ​​known... Using the above formula (2.9), the rotation component of the single camera relative to the vehicle body can be obtained. O q C Translation components O t C That is, the transformation relationship of a single camera. o x c And thus determine o x c =(x x ,x y ,yaw θ ,x scale )∈Sim(2), x x ,x y This refers to the relative position information between the camera and the vehicle body, yaw θ It is the relative attitude information between the camera and the vehicle body, x scale It is the scale factor corresponding to a single camera.

[0131] Furthermore, this application can also determine the extrinsic parameters of other multi-camera systems based on the aforementioned method.

[0132] Furthermore, after determining the transformation relationship between the camera's coordinate space and the vehicle's coordinate space using the alignment result, the initial values ​​of the camera's extrinsic parameters relative to the vehicle can be further calculated: (Xoc ,Y oc Z oc Roll oc Pitch oc Yaw oc )=(x scale *x x ,x scale *x y ,x scale *x z ,X roll X pitch X yaw ), where x x x y X yaw (i.e., yaw) θ ), and x scale From the aforementioned transformation relationship, x can be derived. z X pitch X roll These are the altitude information, the second pitch angle information, and the second roll angle information mentioned above, respectively.

[0133] Furthermore, considering that different motion trajectories have different information content for parameter estimation, especially different impacts on single-camera scale error, in order to improve the effectiveness of parameter estimation, data from multiple second preset time periods can be obtained to determine camera extrinsic parameters. Specifically, suitable road segments can be selected for overall nonlinear extrinsic parameter optimization. For example, n trajectory segments can be selected (each second preset time period corresponds to one trajectory segment), and each trajectory segment has m sampling time periods (i.e., m+1 sampling points). Overall, n equations can be established, and m initial values ​​of extrinsic parameters can be calculated. The RANSAC strategy is introduced, and the cost function is as follows:

[0134]

[0135] in, and These represent the rotation components of the camera's transformation from the i-th frame environment image to the (i+1)-th frame environment image (by...). Determine) and translation components (by) Sure), and These represent the rotational components of the vehicle at the corresponding time points (by...). Determine) and translation components (by) (Definition), where I represents the 3×3 identity matrix. express The corresponding rotation matrix; nj is the corresponding number of samples.

[0136] Based on the aforementioned error cost function, a Hessian matrix H is constructed. By quantitatively analyzing the information content of H, the optimal extrinsic parameter components can be selected. Specifically, this can be done by solving for the eigenvectors of H, assuming that the maximum value among a series of eigenvalues ​​corresponding to the eigenvectors is λ. max The minimum value is λ min Let the proportionality coefficient The amount of information in the trajectory segments to be filtered is defined as follows:

[0137]

[0138] Where m is the number of sampling time periods within the second preset time period. The greater the information content of the trajectory segment, the better the effect of parameter estimation based on the trajectory segment. The principle is to eliminate trajectory segments with too low information content and obtain the optimal external parameters from the camera to the vehicle.

[0139] This application improves the efficiency of determining camera extrinsic parameters by acquiring a first pose information set of a camera installed on a vehicle within a first preset time period; acquiring a warehouse location image captured by the camera within the first preset time period; determining prior information corresponding to the first pose information set based on the warehouse location image; acquiring a second pose information set of the vehicle within the first preset time period; and determining the camera's extrinsic parameters using the first pose information set, the prior information, and the second pose information set. Each first pose information set in the first pose information set includes three-dimensional position information, and the corresponding first pitch angle, first yaw angle, and first roll angle information. The prior information includes second pitch angle information, second roll angle information, and the height information of the camera from the reference plane corresponding to the warehouse location image.

[0140] Figure 2 A schematic diagram of a camera extrinsic parameter determination device provided for an exemplary embodiment of this application; wherein the device includes:

[0141] The first acquisition module 21 is used to acquire the first pose information set of the camera set on the vehicle within a first preset time period;

[0142] The second acquisition module 22 is used to acquire the warehouse location image captured by the camera within the first preset time period;

[0143] The first determining module 23 is used to determine the prior information corresponding to the first pose information set based on the storage location image;

[0144] The third acquisition module 24 is used to acquire the second pose information set of the vehicle within the first preset time period;

[0145] The second determining module 25 is used to determine the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set;

[0146] Wherein, any first pose information in the first pose information set includes three-dimensional position information, as well as first pitch angle information, first yaw angle information, and first roll angle information corresponding to the three-dimensional position information; the prior information includes second pitch angle information, second roll angle information, and height information of the camera relative to the reference plane corresponding to the storage location image.

[0147] Optionally, when the above-mentioned device is used to acquire the warehouse location image captured by the camera within the first preset time period, it is specifically used for:

[0148] Acquire the set of environmental images captured by the camera within the first preset time period;

[0149] Images of storage locations are selected from the set of environmental images.

[0150] Optionally, when the above-mentioned device is used to determine the prior information corresponding to the first pose information set based on the storage location image, it is specifically used for:

[0151] The second pitch angle information and the second roll angle information of the camera relative to the reference plane corresponding to the storage location image are determined based on the storage location image and the first pose information set.

[0152] The height information of the camera from the reference plane is determined using the second pitch angle information, the second roll angle information, and the storage location image;

[0153] The prior information is determined using the second pitch angle information, the second roll angle information, and the altitude information.

[0154] Optionally, when the above-mentioned device is used to determine the height information of the camera from the reference plane using the second pitch angle information, the second roll angle information, and the storage location image, it is specifically used for:

[0155] The corresponding reference plane projection transformation matrix is ​​determined based on the second pitch angle information and the second roll angle information;

[0156] The height information of the camera from the reference plane is determined based on the reference plane projection transformation matrix and the location image.

[0157] Optionally, the first preset time period includes multiple first sub-time periods. When the above-mentioned device is used to determine the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set, it is specifically used for:

[0158] Based on the first pose information set, the first motion increment information of the camera corresponding to each first sub-time period is determined, and the first motion increment information set of the camera corresponding to the first preset time period is obtained.

[0159] The reference motion increment information set is determined by using the reference plane projection transformation matrix corresponding to the prior information and the first motion increment information set;

[0160] The second motion increment information set corresponding to the vehicle within the first preset time period is determined based on the second pose information set.

[0161] The extrinsic parameters of the camera are determined based on the reference motion increment information set and the second motion increment information set;

[0162] The first sub-time period is the time period between adjacent sampling times corresponding to the first pose information set.

[0163] Optionally, when the above-mentioned device is used to acquire the first pose information set of the camera installed on the vehicle within a first preset time period, it is specifically used for:

[0164] Acquire the set of environmental images captured by the camera within the first preset time period;

[0165] The environmental image set is input into the visual odometry, so that the visual odometry determines the first pose information set based on the environmental image set;

[0166] Receive the first pose information set.

[0167] Optionally, when the above-mentioned device is used to acquire the second pose information set of the vehicle within the first preset time period, it is specifically used for:

[0168] Obtain the rotation number information of the two rear wheels of the vehicle in each of the second sub-time periods within the first preset time period;

[0169] By using a preset differential motion model and the rotation number information of each of the two rear wheels in each second sub-time period, the second pose information corresponding to the start and end times of each second sub-time period is determined, thus obtaining the second pose information set of the vehicle in the first preset time period.

[0170] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device can execute the above method embodiments, and the foregoing and other operations and / or functions of each module in the device correspond to the corresponding processes in the various methods in the above method embodiments, which will not be repeated here for the sake of brevity.

[0171] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0172] Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may include:

[0173] The system includes a memory 301 and a processor 302. The memory 301 stores computer programs and transfers the program code to the processor 302. In other words, the processor 302 can retrieve and run the computer programs from the memory 301 to implement the methods described in the embodiments of this application.

[0174] For example, the processor 302 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0175] In some embodiments of this application, the processor 302 may include, but is not limited to:

[0176] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0177] In some embodiments of this application, the memory 301 includes, but is not limited to:

[0178] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0179] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 301 and executed by the processor 302 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0180] like Figure 3 As shown, the electronic device may further include:

[0181] Transceiver 303, which can be connected to processor 302 or memory 301.

[0182] The processor 302 can control the transceiver 303 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 303 may include a transmitter and a receiver. The transceiver 303 may further include antennas, and the number of antennas may be one or more.

[0183] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0184] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0185] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0186] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0188] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0189] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining camera extrinsic parameters, characterized in that, include: Acquire the first pose information set of the camera set on the vehicle within the first preset time period; Acquire the warehouse location images captured by the camera within the first preset time period; Determine the prior information corresponding to the first pose information set based on the library location image; Obtain the second pose information set of the vehicle within the first preset time period; The extrinsic parameters of the camera are determined using the first pose information set, the prior information, and the second pose information set. Wherein, any first pose information in the first pose information set includes three-dimensional position information, as well as first pitch angle information, first yaw angle information, and first roll angle information corresponding to the three-dimensional position information; the prior information includes second pitch angle information, second roll angle information, and height information of the camera relative to the reference plane corresponding to the storage location image.

2. The method according to claim 1, characterized in that, Obtaining the warehouse location images captured by the camera within the first preset time period includes: Acquire the set of environmental images captured by the camera within the first preset time period; Images of storage locations are selected from the set of environmental images.

3. The method according to claim 2, characterized in that, The prior information corresponding to the first pose information set determined based on the storage location image includes: The second pitch angle information and the second roll angle information of the camera relative to the reference plane corresponding to the storage location image are determined based on the storage location image and the first pose information set. The height information of the camera from the reference plane is determined using the second pitch angle information, the second roll angle information, and the storage location image; The prior information is determined using the second pitch angle information, the second roll angle information, and the altitude information.

4. The method according to claim 3, characterized in that, Determining the height information of the camera from the reference plane using the second pitch angle information, the second roll angle information, and the storage location image includes: The corresponding reference plane projection transformation matrix is ​​determined based on the second pitch angle information and the second roll angle information; The height information of the camera from the reference plane is determined based on the reference plane projection transformation matrix and the location image.

5. The method according to claim 4, characterized in that, The first preset time period includes multiple first sub-time periods. Determining the camera's extrinsic parameters using the first pose information set, the prior information, and the second pose information set includes: Based on the first pose information set, the first motion increment information of the camera corresponding to each first sub-time period is determined, and the first motion increment information set of the camera corresponding to the first preset time period is obtained. The reference motion increment information set is determined by using the reference plane projection transformation matrix corresponding to the prior information and the first motion increment information set; The second motion increment information set corresponding to the vehicle within the first preset time period is determined based on the second pose information set. The extrinsic parameters of the camera are determined based on the reference motion increment information set and the second motion increment information set; The first sub-time period is the time period between adjacent sampling times corresponding to the first pose information set.

6. The method according to claim 1, characterized in that, The first pose information set of the camera installed on the vehicle during the first preset time period includes: Acquire the set of environmental images captured by the camera within the first preset time period; The environmental image set is input into the visual odometry, so that the visual odometry determines the first pose information set based on the environmental image set; Receive the first pose information set.

7. The method according to claim 3, characterized in that, Obtaining the second pose information set of the vehicle within the first preset time period includes: Obtain the rotation number information of the two rear wheels of the vehicle in each of the second sub-time periods within the first preset time period; By using a preset differential motion model and the rotation number information of each of the two rear wheels in each second sub-time period, the second pose information corresponding to the start and end times of each second sub-time period is determined, thus obtaining the second pose information set of the vehicle in the first preset time period.

8. A camera extrinsic parameter determination device, characterized in that, include: The first acquisition module is used to acquire the first pose information set of the camera set on the vehicle within a first preset time period; The second acquisition module is used to acquire images of the storage location captured by the camera within the first preset time period; The first determining module is used to determine the prior information corresponding to the first pose information set based on the storage location image. The third acquisition module is used to acquire the second pose information set of the vehicle within the first preset time period; The second determining module is used to determine the extrinsic parameters of the camera using the first pose information set, the prior information, and the second pose information set. Wherein, any first pose information in the first pose information set includes three-dimensional position information, as well as first pitch angle information, first yaw angle information, and first roll angle information corresponding to the three-dimensional position information; the prior information includes second pitch angle information, second roll angle information, and height information of the camera relative to the reference plane corresponding to the storage location image.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1-7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images