A method and device for determining the mounting pose of a vehicle-mounted surround-view camera

Abstract

This application discloses a method and apparatus for determining the installation pose of a vehicle-mounted surround-view camera. Specifically, it generates a bird's-eye view to surround view mapping table and a surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle-mounted surround-view camera for the corresponding target vehicle model; it generates a bird's-eye view panoramic image blur metric function based on the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table; and it calculates the optimal installation pose of the vehicle-mounted surround-view camera using a preset algorithm with the goal of minimizing the function value of the bird's-eye view panoramic image blur metric function. Since this solution does not rely on the designer's experience, it avoids the problem of poor accuracy caused by subjective factors.

Description

Technical Field

[0001] This application relates to the field of automotive technology, and more specifically, to a method and apparatus for determining the mounting position of an in-vehicle surround-view camera. Background Technology

[0002] Vehicle surround view cameras have become a standard feature in many mass-produced vehicles. They achieve panoramic surround view functionality through surround view stitching technology, which constructs 2D or 3D panoramic surround view images based on images obtained from vehicle surround view cameras, providing drivers with a bird's-eye view and thus effectively improving parking safety.

[0003] In recent years, with the development of artificial intelligence technology, autonomous parking functions based on vehicle surround-view cameras have become increasingly common. Compared with previous panoramic surround-view functions, autonomous parking functions also place higher demands on the stitching effect of panoramic surround-view images. Not only must the stitched image be clear, but the stitched area must also be as wide as possible. To achieve this function, the installation posture, i.e., the installation position and angle of the vehicle surround-view camera, is crucial. However, currently, the installation posture can only rely on the experience of designers, and in this process, the subjective factors of humans inevitably lead to poor accuracy. Summary of the Invention

[0004] In view of this, this application provides a method and apparatus for determining the installation position of a vehicle-mounted surround view camera, so as to avoid the problem of poor accuracy caused by subjective factors in the process.

[0005] To achieve the above objectives, the following solution is proposed:

[0006] A method for determining the mounting pose of a vehicle-mounted surround-view camera, the method comprising the following steps:

[0007] Based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model, generate a bird's-eye view to surround view mapping table and a surround view to bird's-eye view mapping table.

[0008] Generate a bird's-eye view panoramic image blur metric function for the current pose of the vehicle surround view camera based on the bird's-eye view to ring view mapping table and the ring view to bird's-eye view panoramic image mapping table.

[0009] The optimal installation position of the vehicle-mounted surround view camera is obtained by using a preset algorithm and aiming to minimize the function value of the blur metric function of the bird's-eye view panoramic image.

[0010] Optionally, the step of generating a bird's-eye view to surround view mapping table and a surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model includes the following steps:

[0011] The bird's-eye panoramic image rotation view mapping table is generated based on the internal parameters and the external parameters. The bird's-eye panoramic image rotation view mapping table includes a first mapping matrix in the image width direction and a second mapping matrix in the image length direction.

[0012] The panoramic view to bird's-eye view mapping table is generated based on the internal parameters and the external parameters. The panoramic view to bird's-eye view mapping table includes a third mapping matrix in the image width direction and a fourth mapping matrix in the image length direction.

[0013] Optionally, generating the bird's-eye view panoramic view mapping table based on the internal parameters and the external parameters includes the following steps:

[0014] The direction vector corresponding to any pixel on the surround view is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera.

[0015] The intersection point of the direction vector and the ground is obtained using the external parameters;

[0016] The bird's-eye view pixel position corresponding to each intersection point is obtained by using the pixel scale of the vehicle-mounted surround view camera, resulting in multiple bird's-eye view pixel positions. These multiple bird's-eye view pixel positions constitute the bird's-eye view panoramic rotating view mapping table.

[0017] Optionally, generating the panoramic view-to-bird's-eye view mapping table based on the internal parameters and the external parameters includes the following steps:

[0018] The actual position of any pixel on the bird's-eye view is obtained by using the pixel scale of the vehicle-mounted surround view camera;

[0019] The coordinates of the arbitrary pixel in the camera coordinate system are obtained using the external parameters, and then the direction vector corresponding to the coordinates is obtained.

[0020] The pixel position corresponding to each direction vector is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera. All the pixel positions constitute the surround view to bird's-eye view panoramic image mapping table.

[0021] Optionally, the step of generating a bird's-eye view panoramic image blur metric function based on the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table includes the following steps:

[0022] Calculate the compression rate of the surround view information based on the bird's-eye panoramic image;

[0023] Calculate the blur matrix of each pixel in the bird's-eye view panoramic image based on the information compression rate;

[0024] The ambiguity measurement function of the bird's-eye view panoramic image is constructed based on the ambiguity matrix.

[0025] Optionally, before the steps of generating a bird's-eye view to surround view mapping table and a surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model, the following steps are also included:

[0026] Multiple template images are captured using the vehicle-mounted surround-view camera, and the template images are calibrated using a preset calibration method to obtain the internal parameters.

[0027] A device for determining the mounting pose of a vehicle-mounted surround-view camera, the device comprising:

[0028] The mapping table generation module is configured to generate a bird's-eye view to surround view mapping table and a surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model.

[0029] The function building module is configured to generate a bird's-eye view panoramic image blur metric function for the current pose of the vehicle surround view camera based on the bird's-eye view to ring view mapping table and the ring view to bird's-eye view panoramic image mapping table.

[0030] The function calculation module is configured to use a preset algorithm and aim to minimize the function value of the bird's-eye view panoramic image blur metric function to obtain the optimal installation pose of the vehicle-mounted surround view camera.

[0031] Optionally, the mapping table generation module includes:

[0032] The first generation unit is used to generate the bird's-eye panoramic image rotation view mapping table according to the internal parameters and the external parameters. The bird's-eye panoramic image rotation view mapping table includes a first mapping matrix in the image width direction and a second mapping matrix in the image length direction.

[0033] The second generation unit is used to generate the panoramic view to bird's-eye view mapping table according to the internal parameters and the external parameters. The panoramic view to bird's-eye view mapping table includes a third mapping matrix in the image width direction and a fourth mapping matrix in the image length direction.

[0034] Optionally, the first generation unit is configured to perform the following steps:

[0035] The direction vector corresponding to any pixel on the surround view is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera.

[0036] The intersection point of the direction vector and the ground is obtained using the external parameters;

[0037] The bird's-eye view pixel position corresponding to each intersection point is obtained by using the pixel scale of the vehicle-mounted surround view camera, resulting in multiple bird's-eye view pixel positions. These multiple bird's-eye view pixel positions constitute the bird's-eye view panoramic rotating view mapping table.

[0038] Optionally, the second generation unit is configured to perform the following steps:

[0039] The actual position of any pixel on the bird's-eye view is obtained by using the pixel scale of the vehicle-mounted surround view camera;

[0040] The coordinates of the arbitrary pixel in the camera coordinate system are obtained using the external parameters, and then the direction vector corresponding to the coordinates is obtained.

[0041] The pixel position corresponding to each direction vector is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera. All the pixel positions constitute the surround view to bird's-eye view panoramic image mapping table.

[0042] Optionally, the function building module includes:

[0043] The numerical calculation unit is used to calculate the compression rate of the surround view information based on the bird's-eye view panoramic image;

[0044] The first construction unit is used to construct the blur matrix of each pixel in the bird's-eye view panoramic image based on the information compression rate;

[0045] The second construction unit is used to construct the ambiguity measurement function of the bird's-eye view panoramic image based on the ambiguity matrix.

[0046] Optional, also includes:

[0047] The internal parameter calibration module is used to capture multiple template images using the vehicle surround view camera and calibrate the template images using a preset calibration method before the mapping table generation module generates the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle surround view camera of the corresponding target vehicle model, so as to obtain the internal parameters.

[0048] As can be seen from the above technical solution, this application discloses a method and apparatus for determining the installation pose of an in-vehicle surround view camera. Specifically, it generates a bird's-eye view to surround view mapping table and a surround view to bird's-eye view mapping table based on the internal and external parameters of the in-vehicle surround view camera for the corresponding target vehicle model; it generates a bird's-eye view panoramic image blur metric function for the current pose of the in-vehicle surround view camera based on the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table; and it calculates the optimal installation pose of the in-vehicle surround view camera using a preset algorithm with the goal of minimizing the function value of the bird's-eye view panoramic image blur metric function. Since this solution does not rely on the designer's experience, it can avoid the problem of poor accuracy caused by subjective factors. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a method for determining the installation pose of a vehicle-mounted surround-view camera according to an embodiment of this application.

[0051] Figure 2 This is a flowchart illustrating another method for determining the installation pose of a vehicle-mounted surround-view camera according to an embodiment of this application.

[0052] Figure 3 This is a block diagram of a vehicle-mounted surround view camera installation pose determination device according to an embodiment of this application;

[0053] Figure 4 This is a block diagram of another vehicle-mounted surround view camera mounting pose determination device according to an embodiment of this application;

[0054] Figure 5 This is a block diagram of another vehicle-mounted surround view camera installation pose determination device according to an embodiment of this application;

[0055] Figure 6 This is a block diagram of another vehicle-mounted surround view camera installation pose determination device according to an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0057] The requirements for surround view stitching in autonomous parking are: the stitched image should cover as wide an area as possible, and the image should be as clear as possible. Based on this goal, this application establishes an evaluation standard for the stitched image from the perspective of clarity: each value represents the detail of the pixel in depicting the actual plane; the larger the value, the clearer the pixel depiction.

[0058] Since the stitching clarity is independent of the stitching coverage area, this invention selects a coverage area of ​​20m*20m, corresponding to a 2000*2000 pixel stitched image. Within this image, the fewer the total number of low-resolution pixels, the better the stitching effect is considered. Therefore, using the weighted total number of low-resolution pixels within the 2000*2000 stitched image as the objective function, and the position and orientation of each surround-view camera as optimization variables, a nonlinear optimization equation is established and solved to provide a scientific installation position and orientation. Based on the above discussion, this application provides the following specific embodiments.

[0059] Example 1

[0060] Figure 1 This is a flowchart illustrating a method for determining the installation pose of a vehicle-mounted surround-view camera according to an embodiment of this application.

[0061] like Figure 1 As shown, this mounting pose determination method is used to determine the mounting pose of a vehicle-mounted surround-view camera. The mounting pose includes the mounting position and mounting orientation. The mounting pose determination method includes the following steps:

[0062] S1. Generate a mapping table for converting a bird's-eye view panoramic view to a circular view and a mapping table for converting a circular view to a bird's-eye view panoramic view.

[0063] This operation is based on the premise that the internal and external parameters of the vehicle-mounted surround-view camera to be installed on the target vehicle model have been obtained. The external parameters of the vehicle-mounted surround-view camera can be calculated from its installation position and angle. Therefore, this solution is feasible under specific conditions.

[0064] Based on the aforementioned internal and external parameters, a mapping table for converting a panoramic view to a bird's-eye view and a mapping table for converting a bird's-eye view to a panoramic view are generated. Assuming the pixel size of the panoramic camera is W*H (W is the number of pixels in the width direction of the image, and H is the number of pixels in the height direction), the pixel size of the resulting bird's-eye view is M*N (W is the number of pixels in the width direction of the image, and H is the number of pixels in the height direction), and the pixel scale is s (one pixel represents an actual distance of s).

[0065] In this embodiment, the generation of the panoramic view to bird's-eye view mapping table and the bird's-eye view to panoramic view mapping table is specifically achieved through the following schemes. The panoramic view to bird's-eye view mapping table includes a first mapping matrix and a second mapping matrix, and the bird's-eye view to panoramic view mapping table includes a third mapping matrix and a fourth mapping matrix. The generation method specifically includes:

[0066] The following steps are taken to generate a mapping table for converting the panoramic view to a bird's-eye view:

[0067] First, the actual position (i*s, j*s, 0) of any pixel (i, j) in the bird's-eye view can be obtained using the pixel scale s. Then, the coordinates of this pixel in the camera coordinate system can be obtained using the extrinsic parameters of the surround-view camera, and thus its corresponding direction vector can be obtained. Finally, the pixel position corresponding to this vector can be obtained using the intrinsic parameters of the surround-view camera and the camera projection model, thereby obtaining the mapping table of the surround-view to bird's-eye view panorama. This mapping table is the surround-view pixel position (floating-point number) corresponding to each pixel (integer) in the bird's-eye view, and therefore can be represented as two matrices—the first mapping matrix Mx in the image width direction and the second mapping matrix My in the image length direction, both of which are N rows and M columns.

[0068] Generate the bird's-eye view panoramic view mapping table, specifically as follows:

[0069] First, the direction vector corresponding to any pixel (u,v) in the panoramic view can be obtained using the camera's intrinsic parameters and camera projection model. Then, the intersection point (p,q,0) of this direction vector with the ground can be obtained using the panoramic camera's extrinsic parameters. Finally, the bird's-eye view pixel position (p / i,q / i) corresponding to this corner point is obtained using the pixel scale s, thus obtaining the panoramic-to-panoramic view mapping table. This mapping table represents the bird's-eye view pixel position (floating-point number) corresponding to each pixel (integer) in the panoramic view. Therefore, it can be represented by two matrices—a third mapping matrix Mrx in the image width direction and a fourth mapping matrix Mry in the image length direction, both of which are H rows and W columns.

[0070] S2. Generate a blur measurement function for the bird's-eye view panoramic image from the vehicle-mounted surround-view camera.

[0071] Based on the aforementioned bird's-eye view to panoramic view mapping table and panoramic view to bird's-eye view mapping table, a blur metric function for the current pose of the vehicle's panoramic camera is generated. The final generated bird's-eye view panoramic image can be considered as a lossless, clear bird's-eye view first converted to a panoramic image, and then the panoramic image converted back to a bird's-eye view. The blur calculation includes the following steps:

[0072] First, the information compression rates M1 and M2 (H rows and W columns, representing the information compression rate corresponding to each pixel on the ring view) are calculated based on the mapping matrices Mrx and Mry.

[0073] Then, based on M1 and M2, calculate the blurriness matrix Ma of the bird's-eye view (N rows and M columns, representing the blurriness corresponding to each pixel on the bird's-eye view). The calculation method is as follows:

[0074]

[0075] Finally, construct the blurriness metric function f of the bird's-eye panoramic image. f is set to the number of pixels with blurriness below the threshold t in the bird's-eye view. The formula is expressed as:

[0076]

[0077] Here, 1(M a (i,j)<t) represents that if M a (i,j)<t, then it is 1; otherwise, it is 0.

[0078] S3. Calculate the blurriness metric function of the bird's-eye panoramic image with the goal of minimizing the function value.

[0079] That is, with the goal of minimizing the metric function and taking the installation position and angle of the surround-view camera as the variables to be optimized, through optimization methods such as ant colony algorithm, genetic algorithm, particle filter, grid search, etc., calculate the installation position and installation posture that make f minimum. The installation position and installation posture here are the final optimal installation pose.

[0080] As can be seen from the above technical solution, this embodiment provides a method for determining the installation pose of a vehicle-mounted surround-view camera. The determination method is specifically to generate a bird's-eye panoramic view to panoramic view mapping table and a panoramic view to bird's-eye panoramic view mapping table according to the internal parameters and external parameters of the vehicle-mounted surround-view camera corresponding to the target vehicle model; generate a blurriness metric function of the bird's-eye panoramic image of the current pose of the vehicle-mounted surround-view camera according to the bird's-eye panoramic view to panoramic view mapping table and the panoramic view to bird's-eye panoramic view mapping table; use a preset algorithm and calculate with the goal of minimizing the function value of the blurriness metric function of the bird's-eye panoramic image to obtain the best installation pose of the vehicle-mounted surround-view camera. Since this solution does not rely on the experience of designers, it can avoid the problem of poor accuracy caused by subjective factors.

[0081] In addition, in a specific implementation manner of this embodiment, the following steps are further included, as Figure 2 shown:

[0082] S0. Obtain the internal parameters of the vehicle-mounted surround-view camera.

[0083] Multiple template images are captured using the vehicle-mounted surround-view camera, and these images are calibrated using a preset calibration method to obtain the internal parameters of the vehicle-mounted surround-view camera. Since the internal parameters are a prerequisite for this solution, the above steps, based on obtaining the internal parameters, can achieve the purpose of this application; this step allows users to implement the technical solution of this application even without the internal parameters.

[0084] Therefore, the effect of this operation is to make the solution more complete and independent.

[0085] Example 2

[0086] Figure 3 This is a block diagram of a vehicle-mounted surround view camera installation pose determination device according to an embodiment of this application.

[0087] like Figure 3 As shown, the mounting pose determination device is used to determine the mounting pose of the vehicle-mounted surround view camera. The mounting pose includes the mounting position and the mounting attitude. The mounting pose determination device includes a mapping table generation module 10, a function construction module 20, and a function calculation module 30.

[0088] The mapping table generation module is used to generate mapping tables for converting bird's-eye view panoramic images to ring view images and vice versa.

[0089] This operation is based on the premise that the internal and external parameters of the vehicle-mounted surround-view camera to be installed on the target vehicle model have been obtained. The external parameters of the vehicle-mounted surround-view camera can be calculated from its installation position and angle. Therefore, this solution is feasible under specific conditions.

[0090] Based on the aforementioned internal and external parameters, a mapping table for converting a panoramic view to a bird's-eye view and a mapping table for converting a bird's-eye view to a panoramic view are generated. Assuming the pixel size of the panoramic camera is W*H (W is the number of pixels in the width direction of the image, and H is the number of pixels in the height direction), the pixel size of the resulting bird's-eye view is M*N (W is the number of pixels in the width direction of the image, and H is the number of pixels in the height direction), and the pixel scale is s (one pixel represents an actual distance of s).

[0091] In this embodiment, the generation of the panoramic view to bird's-eye view mapping table and the bird's-eye view to panoramic view mapping table is specifically achieved through the following schemes. The panoramic view to bird's-eye view mapping table includes a first mapping matrix and a second mapping matrix, and the bird's-eye view to panoramic view mapping table includes a third mapping matrix and a fourth mapping matrix. The mapping table generation module specifically includes a first generation unit 11 and a second generation unit 12, as follows: Figure 4 As shown.

[0092] The first generation unit is used to generate the mapping table for converting the panoramic view to a bird's-eye view. The specific execution steps of this unit are as follows:

[0093] First, the actual position (i*s, j*s, 0) of any pixel (i, j) in the bird's-eye view can be obtained using the pixel scale s. Then, the coordinates of this pixel in the camera coordinate system can be obtained using the extrinsic parameters of the surround-view camera, and thus its corresponding direction vector can be obtained. Finally, the pixel position corresponding to this vector can be obtained using the intrinsic parameters of the surround-view camera and the camera projection model, thereby obtaining the mapping table of the surround-view to bird's-eye view panorama. This mapping table is the surround-view pixel position (floating-point number) corresponding to each pixel (integer) in the bird's-eye view, and therefore can be represented as two matrices—the first mapping matrix Mx in the image width direction and the second mapping matrix My in the image length direction, both of which are N rows and M columns.

[0094] The second generation unit is used to generate the bird's-eye view rotation view mapping table. The specific execution steps of this unit are as follows:

[0095] First, the direction vector corresponding to any pixel (u,v) in the panoramic view can be obtained using the camera's intrinsic parameters and camera projection model. Then, the intersection point (p,q,0) of this direction vector with the ground can be obtained using the panoramic camera's extrinsic parameters. Finally, the bird's-eye view pixel position (p / i,q / i) corresponding to this corner point is obtained using the pixel scale s, thus obtaining the panoramic-to-panoramic view mapping table. This mapping table represents the bird's-eye view pixel position (floating-point number) corresponding to each pixel (integer) in the panoramic view. Therefore, it can be represented by two matrices—a third mapping matrix Mrx in the image width direction and a fourth mapping matrix Mry in the image length direction, both of which are H rows and W columns.

[0096] The function building module is used to generate a blur metric function for bird's-eye view panoramic images from vehicle surround-view cameras.

[0097] That is, based on the aforementioned bird's-eye view to panoramic view mapping table and panoramic view to bird's-eye view mapping table, a bird's-eye view panoramic image blur metric function is generated for the current pose of the vehicle-mounted panoramic camera. The final generated bird's-eye view panoramic image can be seen as a lossless, clear bird's-eye view first converted to a panoramic image, and then the panoramic image converted back to a bird's-eye view. This module specifically includes a numerical calculation unit 21, a first construction unit 22, and a second construction unit 23, as follows: Figure 5 As shown:

[0098] The numerical calculation unit is used to calculate the information compression rates M1 and M2 of the ring view (H rows and W columns, representing the information compression rate corresponding to each pixel on the ring view) based on the mapping matrices Mrx and Mry.

[0099] The first construction unit is used to calculate the blurriness matrix Ma (N rows and M columns, representing the blurriness corresponding to each pixel in the bird's-eye view) of the bird's-eye view based on M1 and M2. The calculation method is as follows:

[0100]

[0101] The second construction unit is used to construct the blurriness metric function f of the bird's-eye panoramic image. f is set to the number of pixels in the bird's-eye view with blurriness below the threshold t. The formula is expressed as:

[0102]

[0103] Here, 1(M a (i,j)<t) represents that if M a (i,j)<t, then it is 1; otherwise, it is 0.

[0104] The function calculation module is used to calculate the blurriness metric function of the bird's-eye panoramic image with the goal of minimizing the function value.

[0105] That is, with the goal of minimizing the metric function, taking the installation position and angle of the surround-view camera as the variables to be optimized, through optimization methods such as ant colony algorithm, genetic algorithm, particle filter, grid search, etc., calculate the installation position and installation posture that make f the smallest. The installation position and installation posture here are the final optimal installation pose.

[0106] As can be seen from the above technical solution, this embodiment provides a device for determining the installation pose of a vehicle-mounted surround-view camera. The determining device is specifically configured to generate a bird's-eye panoramic view to panoramic view mapping table and a panoramic view to bird's-eye panoramic view mapping table according to the internal parameters and external parameters of the vehicle-mounted surround-view camera corresponding to the target vehicle model; generate a blurriness metric function of the bird's-eye panoramic image of the current pose of the vehicle-mounted surround-view camera according to the bird's-eye panoramic view to panoramic view mapping table and the panoramic view to bird's-eye panoramic view mapping table; use a preset algorithm and calculate with the goal of minimizing the function value of the blurriness metric function of the bird's-eye panoramic image to obtain the best installation pose of the vehicle-mounted surround-view camera. Since this solution does not rely on the experience of designers, it can avoid the problem of poor accuracy caused by subjective factors.

[0107] In addition, in a specific implementation manner of this embodiment, it further includes an internal parameter calibration module 40, as Figure 6 shown: <s

[0108] The internal parameter calibration module is used to obtain the internal parameters of the vehicle-mounted surround-view camera before the mapping table generation module generates a bird's-eye panoramic view to panoramic view mapping table and a panoramic view to bird's-eye panoramic view mapping table according to the internal parameters and external parameters of the vehicle-mounted surround-view camera corresponding to the target vehicle model.

[0109] Multiple template images are captured using the vehicle-mounted surround-view camera, and these images are calibrated using a preset calibration method to obtain the internal parameters of the vehicle-mounted surround-view camera. Since the internal parameters are a prerequisite for this solution, the above steps, based on obtaining the internal parameters, can achieve the purpose of this application; this step allows users to implement the technical solution of this application even without the internal parameters.

[0110] Therefore, the effect of this operation is to make the solution more complete and independent.

[0111] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0112] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0117] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0118] The technical solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for determining the mounting pose of a vehicle-mounted surround-view camera, characterized in that, The determination method includes the following steps: A bird's-eye view panoramic view mapping table is generated based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model. The bird's-eye view panoramic view mapping table includes a first mapping matrix in the image width direction and a second mapping matrix in the image length direction. A panoramic view to bird's-eye view mapping table is generated based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model. The panoramic view to bird's-eye view mapping table includes a third mapping matrix in the image width direction and a fourth mapping matrix in the image length direction. Generate a bird's-eye view panoramic image blur metric function for the current pose of the vehicle surround view camera based on the bird's-eye view to ring view mapping table and the ring view to bird's-eye view panoramic image mapping table. The optimal installation position of the vehicle-mounted surround view camera is obtained by using a preset algorithm and with the goal of minimizing the function value of the blur metric function of the bird's-eye panoramic image. The step of generating a bird's-eye view panoramic image blur metric function based on the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table includes the following steps: The information compression ratio of the ring view is calculated based on the first mapping matrix and the second mapping matrix, where the information compression ratio represents the information compression ratio corresponding to each pixel on the ring view. The ambiguity matrix of the bird's-eye view panoramic image is calculated based on the information compression rate, the third mapping matrix, and the fourth mapping matrix. The ambiguity matrix represents the ambiguity corresponding to each pixel in the bird's-eye view image. The ambiguity measurement function of the bird's-eye view panoramic image is constructed based on the ambiguity matrix.

2. The installation pose determination method as described in claim 1, characterized in that, The step of generating a bird's-eye view panoramic view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera corresponding to the target vehicle model includes the following steps: The direction vector corresponding to any pixel on the surround view is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera. The intersection point of the direction vector and the ground is obtained using the external parameters; The bird's-eye view pixel position corresponding to each intersection point is obtained by using the pixel scale of the vehicle-mounted surround view camera, resulting in multiple bird's-eye view pixel positions. These multiple bird's-eye view pixel positions constitute the bird's-eye panoramic view to loop view mapping table.

3. The installation pose determination method as described in claim 1, characterized in that, The step of generating a surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle includes the following steps: The actual position of any pixel on the bird's-eye view is obtained by using the pixel scale of the vehicle-mounted surround view camera; The coordinates of the arbitrary pixel in the camera coordinate system are obtained using the external parameters, and then the direction vector corresponding to the coordinates is obtained. The pixel position corresponding to each direction vector is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera. All the pixel positions constitute the surround view to bird's-eye view panoramic image mapping table.

4. The method for determining the installation pose as described in any one of claims 1 to 3, characterized in that, Before generating the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle's surround view camera for the corresponding target vehicle model, the following steps are also included: Multiple template images are captured using the vehicle-mounted surround-view camera, and the template images are calibrated using a preset calibration method to obtain the internal parameters.

5. A device for determining the mounting position of a vehicle-mounted surround-view camera, characterized in that, The determining device includes: The mapping table generation module is configured to generate a bird's-eye view to ring view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model. The bird's-eye view to ring view mapping table includes a first mapping matrix in the image width direction and a second mapping matrix in the image length direction. The module also generates a ring view to bird's-eye view mapping table based on the internal and external parameters of the vehicle-mounted surround view camera of the corresponding target vehicle model. The ring view to bird's-eye view mapping table includes a third mapping matrix in the image width direction and a fourth mapping matrix in the image length direction. The function building module is configured to generate a bird's-eye view panoramic image blur metric function for the current pose of the vehicle surround view camera based on the bird's-eye view to ring view mapping table and the ring view to bird's-eye view panoramic image mapping table. The function calculation module is configured to use a preset algorithm and aim to minimize the function value of the bird's-eye panoramic image blur metric function to obtain the optimal installation pose of the vehicle-mounted surround view camera. The function construction module includes: The numerical calculation unit is used to calculate the information compression rate of the ring view based on the first mapping matrix and the second mapping matrix, wherein the information compression rate represents the information compression rate corresponding to each pixel on the ring view. The first construction unit is used to calculate the blur matrix of the bird's-eye view panoramic image based on the information compression rate, the third mapping matrix and the fourth mapping matrix, wherein the blur matrix represents the blur corresponding to each pixel on the bird's-eye view image. The second construction unit is used to construct the ambiguity measurement function of the bird's-eye view panoramic image based on the ambiguity matrix.

6. The installation pose determination device as described in claim 5, characterized in that, The mapping table generation module includes a first generation unit, configured to perform the following steps: The direction vector corresponding to any pixel on the surround view is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera. The intersection point of the direction vector and the ground is obtained using the external parameters; The bird's-eye view pixel position corresponding to each intersection point is obtained by using the pixel scale of the vehicle-mounted surround view camera, resulting in multiple bird's-eye view pixel positions. These multiple bird's-eye view pixel positions constitute the bird's-eye panoramic view to loop view mapping table.

7. The installation pose determination device as described in claim 5, characterized in that, The mapping table generation module includes a second generation unit, configured to perform the following steps: The actual position of any pixel on the bird's-eye view is obtained by using the pixel scale of the vehicle-mounted surround view camera; The coordinates of the arbitrary pixel in the camera coordinate system are obtained using the external parameters, and then the direction vector corresponding to the coordinates is obtained. The pixel position corresponding to each direction vector is obtained by using the internal parameters and the projection model of the vehicle-mounted surround view camera. All the pixel positions constitute the surround view to bird's-eye view panoramic image mapping table.

8. The mounting posture determination device as described in any one of claims 5 to 7, characterized in that, Also includes: The internal parameter calibration module is used to capture multiple template images using the vehicle surround view camera and calibrate the template images using a preset calibration method before the mapping table generation module generates the bird's-eye view to surround view mapping table and the surround view to bird's-eye view mapping table based on the internal and external parameters of the vehicle surround view camera of the corresponding target vehicle model, so as to obtain the internal parameters.

Images