Method and apparatus for generating bird's-eye view images

Abstract

According to an embodiment of the present disclosure, a method, apparatus, electronic device, computer storage medium, and computer program product for generating a bird's-eye view image are provided. The method described herein includes: determining a unified external parameter for an image acquisition device based on multiple images captured by an image acquisition device, the unified external parameter indicating the position and / or posture of the image acquisition device relative to a reference plane; determining multiple independent external parameters corresponding to the multiple images by adjusting the unified external parameter; and projecting multiple images based on the multiple independent external parameters to generate a bird's-eye view image. Thus, the embodiment of the present disclosure can determine the independent external parameters corresponding to each image, thereby improving the accuracy of bird's-eye view image generation.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, apparatuses, electronic devices, computer storage media, and computer program products for generating bird's-eye view images. Background Art

[0002] In the process of producing map data, the generation of bird's-eye view images (also called bird's-eye view) is an important step. For example, inverse perspective mapping (IPM) can be used to perform IPM projection on a set of images taken by a collection vehicle to complete the ground mapping. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for generating a bird's-eye view image is provided. The method comprises: determining a unified external parameter for the image acquisition device based on a plurality of images captured by the image acquisition device, the unified external parameter indicating a position and / or posture of the image acquisition device relative to a reference plane; determining a plurality of independent external parameters corresponding to the plurality of images by adjusting the unified external parameter; and projecting the plurality of images based on the plurality of independent external parameters to generate the bird's-eye view image.

[0004] In a second aspect of the present disclosure, a device for generating a bird's-eye view image is provided. The device includes: a determination module configured to determine a unified external parameter for an image acquisition device based on multiple images captured by the image acquisition device, the unified external parameter indicating a position and / or posture of the image acquisition device relative to a reference plane; an adjustment module configured to determine multiple independent external parameters corresponding to the multiple images by adjusting the unified external parameter; and a projection module configured to project the multiple images based on the multiple independent external parameters to generate a bird's-eye view image.

[0005] In a third aspect of the present disclosure, an electronic device is provided, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to the first aspect of the present disclosure.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method according to the first aspect of the present disclosure.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer executable instructions, wherein the computer executable instructions implement the method according to the first aspect of the present disclosure when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A block diagram illustrating an example environment in which embodiments of the present disclosure can be implemented;

[0010] Figure 2 A flowchart showing an example process for generating an overhead image according to some embodiments of the present disclosure is shown;

[0011] Figure 3A and Figure 3B A schematic diagram of adjusting external parameters according to some embodiments of the present disclosure is shown;

[0012] Figure 4A and Figure 4B A comparative schematic diagram of a bird's-eye view image according to some embodiments of the present disclosure is shown;

[0013] Figure 5 A block diagram showing an apparatus for generating a bird's-eye view image according to some embodiments of the present disclosure; and

[0014] Figure 6 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] As discussed above, the generation of bird's-eye view images is an important task in the process of map data production. Bird's-eye view images can usually be generated by using the external parameters (also called external parameters) of the image acquisition device (e.g., camera) to perform an inverse perspective transformation to generate the corresponding bird's-eye view image. However, when, for example, a collection vehicle equipped with an image acquisition device encounters a speed bump or an uneven road surface, the collection vehicle will shake, which may cause the projection data results obtained under the unified external parameters to be inconsistent with the real world.

[0018] In view of this, an embodiment of the present disclosure proposes a scheme for generating a bird's-eye view image. In this scheme, a unified external parameter for an image acquisition device can be determined based on multiple images captured by the image acquisition device, and the unified external parameter indicates the position and / or posture of the image acquisition device relative to a reference plane. Further, multiple independent external parameters corresponding to the multiple images can be determined by adjusting the unified external parameter. Accordingly, multiple images can be projected based on the multiple independent external parameters to generate a bird's-eye view image.

[0019] Therefore, the embodiments of the present disclosure can determine the independent external parameters corresponding to each image, which can effectively solve the problem of jitter of the image acquisition device due to the capture of vehicle driving, thereby improving the accuracy of bird's-eye view image generation.

[0020] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0021] Example Environment

[0022] Figure 1 1 shows a block diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, environment 100 may include an image acquisition device 125 for capturing image data.

[0023] by Figure 1 As an example, the image acquisition device 125 may be carried by a collection vehicle 120. Such a collection vehicle 120 may include any appropriate type of transportation, and the present disclosure is not intended to limit the specific type of the collection vehicle 120.

[0024] The collection vehicle 120 may, for example, travel on the road 110 and use the image collection device 125 carried thereon to capture a plurality of images 130. The image collection device 125 may, for example, include a camera deployed on the collection vehicle 120 in an appropriate manner, for example, examples of which may include but are not limited to a driving recorder, etc.

[0025] Furthermore, the electronic device 140 may acquire the plurality of images 130 captured by the image acquisition device 125. As will be described in detail below, the electronic device 140 may generate an overhead image 150 based on the plurality of images 130 captured.

[0026] by Figure 1 As an example, such an overhead image 150 may be a bird's-eye view of the road 110. Furthermore, such an overhead image 150 may be provided for various suitable types of tasks, such as lane line recognition, map production, and the like.

[0027] The electronic device 140 may be any suitable computing device that is communicatively coupled to the vehicle 120, and may be deployed inside the vehicle 120, such as an onboard computing device. Alternatively, the electronic device 140 may also be deployed outside the vehicle 120, such as an edge computing device or a cloud computing device.

[0028] The specific process of generating the bird's-eye view image 150 will be further described below with reference to the accompanying drawings.

[0029] Example Process

[0030] Figure 2 FIG. 2 is a flowchart showing an example process 200 for generating an overhead image according to various embodiments of the present disclosure. For example, the process 200 may be performed by: Figure 1 For convenience of description, the following reference Figure 1 The process 200 is described below.

[0031] like Figure 2 As shown, at box 210, the electronic device 140 determines unified external parameters for the image acquisition device 125 based on the multiple images 130 captured by the image acquisition device 125, wherein the unified external parameters indicate the position and / or posture of the image acquisition device 125 relative to a reference plane.

[0032] In some embodiments, the electronic device 140 may generate a three-dimensional reconstruction model based on the plurality of images 130. Further, the electronic device 140 may determine unified external parameters for the image acquisition device 125 based on the constructed three-dimensional reconstruction model.

[0033] It should be understood that any appropriate three-dimensional reconstruction technology may be used to construct a corresponding three-dimensional model based on the multiple images 130 , and the present disclosure is not intended to limit the specific process of three-dimensional reconstruction.

[0034] Furthermore, the electronic device 140 can calibrate unified external parameters of the image acquisition device 125 based on a set of points corresponding to a reference plane (e.g., a road plane) in the constructed three-dimensional model and the pose information of the image, that is, external parameters applicable to all multiple images 130.

[0035] It should be understood that any appropriate unified external parameter calibration method may be used to determine the unified external parameters of the image acquisition device 125 based on the three-dimensional model.

[0036] Such unified external parameters can be used to describe the position and / or posture of the image acquisition device 125 relative to the reference plane. Taking the vehicle-mounted camera as an example of the image acquisition device 125, such unified external parameters can indicate the conversion relationship between the camera coordinate system of the image acquisition device 125 and the road surface coordinate system of the road surface where the vehicle is located (for example, a rotation matrix and / or a translation matrix, etc.). In some examples, such external parameters can also represent information such as the position and posture of the image acquisition device 125 relative to the reference plane.

[0037] Continue to refer Figure 2 In block 220 , the electronic device 140 determines a plurality of independent external parameters corresponding to the plurality of images 130 by adjusting the unified external parameters.

[0038] Figure 3A FIG. 1 shows a schematic diagram of unified external parameters according to some embodiments of the present disclosure. Figure 3A It can be seen that, taking the vehicle-mounted camera as an example of the image acquisition device 125, when the vehicle is driving on the road (also called the reference plane 320), it may be bumpy due to passing through a deceleration belt or other reasons, which may cause jitter in the acquired image.

[0039] In the case of using uniform external parameters, such as Figure 3A As shown, the reference planes 310 - 1 , 310 - 2 , and 310 - 3 (eg, local road surfaces) estimated based on different images experience large jitters, which results in a large deviation from the actual situation of the reference plane 320 (eg, road surface).

[0040] In order to improve the accuracy of the projection result, the electronic device 140 may determine a corresponding independent external parameter for each image in the plurality of images 130 .

[0041] In some embodiments, taking a single image (also referred to as a target image) among the multiple images 130 as an example, the electronic device 140 may use the determined unified external parameters as the initial external parameters of the target image, and adjust the initial external parameters by means of parameter optimization.

[0042] Specifically, the electronic device 140 may adjust the initial external parameters based on the objective function associated with the target image, thereby determining independent external parameters for the target image.

[0043] In some embodiments, the plurality of images 130 captured by the image acquisition device 125 may be a plurality of captured video frames. In this case, the directions of the reference planes corresponding to the temporally adjacent video frames should be approximately consistent.

[0044] Thus, the objective function can indicate a first loss associated with the target image, and the first loss can indicate an angular difference between a first normal vector and a second normal vector of a reference plane, wherein the first normal vector corresponds to the target image and the second normal vector can correspond to a reference image associated with the target image among the multiple images.

[0045] For example, when the target image is the first video frame among multiple video frames, the reference image may be a second video frame temporally adjacent to the first video frame among the multiple video frames. For example, the second video frame may be a previous frame or a subsequent frame of the first video frame.

[0046] Therefore, the electronic device 140 can optimize the unified external parameters by minimizing the first loss so that the first normal vector is as parallel to the second normal vector as possible.

[0047] In addition, taking the image acquisition device 125 as an example of a vehicle-mounted camera, the image acquisition device 125 moves in a direction that is nearly parallel to the road surface direction as it moves with the acquisition vehicle 120 .

[0048] Thus, in some embodiments, the objective function indicates a second loss associated with the target image, the second loss indicating whether a first normal vector of the reference plane and a trajectory vector of the image acquisition device are perpendicular, the first normal vector corresponding to the target image.

[0049] Accordingly, the electronic device 140 may minimize the second loss so as to make the first normal vector and the trajectory vector as perpendicular as possible.

[0050] Additionally, considering that the calibrated unified extrinsic parameters have a certain reference value, the objective function may also indicate a third loss associated with the target image, and the third loss may indicate the difference between the independent extrinsic parameters and the unified extrinsic parameters.

[0051] Thus, the electronic device 140 can minimize the third loss so that the difference between the determined independent external parameters and the unified external parameters is smaller. Such a third loss can also be called a regularization term constraint to prevent over-optimization.

[0052] As an example, the objective function used by the electronic device 140 may be expressed as:

[0053]

[0054]

[0055] Among them, g i is the ground plane vector corresponding to the i-th image, g′ i is the trajectory vector of the image acquisition device indicated by the i-th image, g″ i is the ground plane vector in the i-th image determined based on the unified external parameters, N represents the number of images, where g′ i and g″ i They can all be calculated from the reconstruction information and the unified external parameters. α, β, and γ are the weights of each loss. is the position and posture of the image acquisition device determined based on the i-th image, is the independent external parameter corresponding to the i-th image, and PickCol(R,n) is the n-th column vector of the matrix R.

[0056] Therefore, according to formula (1), the ground plane vector g corresponding to the i-th image is determined i After that, the local external parameters corresponding to the i-th image can be further determined according to formula (2).

[0057] The above objective function indicates the first loss, the second loss and the third loss introduced above. It should be understood that the above specific objective function is only an example representation, and the electronic device 140 may also consider only part of the above losses, or may also consider other additional losses.

[0058] In some embodiments, as shown in reference formula (1), the first loss, the second loss and the third loss may correspond to different weight coefficients α, β, γ. In some embodiments, in order to improve the accuracy of the independent external parameters, the first weight corresponding to the first loss or the second weight corresponding to the second loss may be greater than the third weight.

[0059] refer to Figure 3B The example shown, with Figure 3A Compared with the unified extrinsic parameters used, the reference planes (eg, local road surfaces) 330 - 1 , 330 - 2 , and 330 - 3 estimated using independent extrinsic parameters will more closely match the true reference plane 320 (eg, road surface).

[0060] At block 230 , the electronic device 140 projects a plurality of images based on a plurality of independent external parameters to generate the bird's-eye view image 150 .

[0061] Specifically, the electronic device 140 can perform an inverse perspective transformation IPM for multiple images based on multiple independent external parameters to project the images from a front view to a BEV (bird's eye view) perspective to generate multiple IPM images for the multiple images. The specific process of the IPM transformation is not described here.

[0062] Furthermore, the electronic device 140 may generate a bird's-eye view image by combining a plurality of IPM images.

[0063] For example, Figure 4A An example bird's-eye view image 400A generated using uniform external parameters is shown. It can be seen that in the bird's-eye view image 400A, projections of some road features are blurred and arrows appear distorted. Figure 4B An example bird's-eye view image 400B generated using independent external parameters corresponding to multiple images is shown. Figure 4B It can be seen that, compared with the bird's-eye view image 400A, the road features become clearer, the distortion of the arrows is reduced, and the projection result of the left curb is more accurate.

[0064] Further, as referenced Figure 1 As discussed, the bird's-eye view image 400B generated using multiple independent external parameters can be further provided for various types of downstream tasks, such as map production, lane line recognition, traffic sign recognition, stop line recognition, and / or recognition of other ground elements.

[0065] Thus, the embodiments of the present disclosure can determine independent external parameters for multiple images by optimizing unified external parameters, and use the independent external parameters to project the corresponding images respectively, thereby improving the generation quality of the bird's-eye view image. In addition, this method can solve the projection error caused by the shaking of the acquisition vehicle.

[0066] Example devices and equipment

[0067] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an apparatus 500 for generating a bird's-eye view image according to some embodiments of the present disclosure is shown.

[0068] like Figure 5 As shown, the device 500 includes a determination module 510, which is configured to determine a unified external parameter for an image acquisition device based on a plurality of images captured by the image acquisition device, wherein the unified external parameter indicates a position and / or posture of the image acquisition device relative to a reference plane; an adjustment module 520, which is configured to determine a plurality of independent external parameters corresponding to the plurality of images by adjusting the unified external parameter; and a projection module 530, which is configured to project the plurality of images based on the plurality of independent external parameters to generate a bird's-eye view image.

[0069] In some embodiments, the adjustment module 520 is further configured to: for a target image among multiple images: set unified external parameters as initial external parameters for the target image; and adjust the initial external parameters based on the objective function associated with the target image to determine independent external parameters for the target image.

[0070] In some embodiments, the objective function indicates a first loss associated with the target image, the first loss indicating an angular difference between a first normal vector and a second normal vector of a reference plane, the first normal vector corresponding to the target image, and the second normal vector corresponding to a reference image associated with the target image in the plurality of images.

[0071] In some embodiments, the multiple images are multiple video frames captured by an image acquisition device, the target image corresponds to a first video frame, and the reference image corresponds to a second video frame in the multiple video frames that is temporally adjacent to the first video frame.

[0072] In some embodiments, the objective function indicates a second loss associated with the target image, the second loss indicating whether a first normal vector of the reference plane and a trajectory vector of the image acquisition device are perpendicular, and the first normal vector corresponds to the target image.

[0073] In some embodiments, the objective function further indicates a third loss associated with the target image, the third loss indicating a difference between the independent extrinsic parameters and the unified extrinsic parameters.

[0074] In some embodiments, in the objective function: a first weight corresponding to the first loss is greater than a third weight corresponding to the third loss; or a second weight corresponding to the second loss is greater than the third weight of the third loss.

[0075] In some embodiments, the generating module 530 is further configured to: perform inverse perspective transformation IPM on multiple images based on multiple independent external parameters to generate multiple IPM maps for the multiple images; and generate a bird's-eye view image by combining the multiple IPM maps.

[0076] In some embodiments, the determination module 510 is further configured to: generate a three-dimensional reconstruction model based on the multiple images; and determine unified external parameters for the image acquisition device based on the three-dimensional reconstruction model.

[0077] In some embodiments, the image acquisition device is carried by a acquisition vehicle, and the reference plane is the road surface on which the vehicle is traveling.

[0078] The units included in the device 500 can be implemented in various ways, including software, hardware, firmware or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 500 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like. In addition, in some embodiments, multiple functions implemented by a unit (or module) can be implemented by sub-units or other independent units included therein.

[0079] Figure 6 1 shows a block diagram of an electronic device / server 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The electronic device / server 600 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

[0080] like Figure 6 As shown, the electronic device / server 600 is in the form of a general electronic device. The components of the electronic device / server 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of the electronic device / server 600.

[0081] The electronic device / server 600 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device / server 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 can be a volatile memory (e.g., register, cache, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 630 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data (e.g., training data for training) and can be accessed within the electronic device / server 600.

[0082] The electronic device / server 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0083] The communication unit 640 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device / server 600 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device / server 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0084] The input device 650 may be one or more input devices, such as a mouse, keyboard, tracking ball, etc. The output device 650 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device / server 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device / server 600, or communicate with any device that allows the electronic device / server 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0085] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above.

[0086] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0087] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0088] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0089] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0090] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the implementations disclosed herein.

Claims

1. A method for generating a bird's-eye view image, comprising: Based on a plurality of images captured by an image acquisition device, determining a unified external parameter for the image acquisition device, the unified external parameter indicating a position and / or posture of the image acquisition device relative to a reference plane, wherein determining the unified external parameter comprises: generating a three-dimensional reconstruction model based on the plurality of images, and calibrating the unified external parameter of the image acquisition device based on a set of points corresponding to the reference plane in the constructed three-dimensional reconstruction model and the pose information of the image; Determining a plurality of independent extrinsic parameters corresponding to the plurality of images by adjusting the unified extrinsic parameter; and The plurality of images are projected based on the plurality of independent extrinsic parameters to generate a bird's-eye view image.

2. The method of claim 1 , wherein determining a plurality of independent extrinsic parameters corresponding to the plurality of images comprises: For a target image among the multiple images: Setting the unified extrinsic parameters as initial extrinsic parameters for the target image; as well as The initial extrinsic parameters are adjusted based on an objective function associated with the target image to determine independent extrinsic parameters for the target image.

3. The method of claim 2, wherein the objective function indicates a first loss associated with the target image, the first loss indicating an angular difference between a first normal vector and a second normal vector of the reference plane, the first normal vector corresponding to the target image, the second normal vector corresponding to a reference image associated with the target image among the multiple images.

4. The method according to claim 3, wherein the multiple images are multiple video frames captured by the image acquisition device, the target image corresponds to a first video frame, and the reference image corresponds to a second video frame in the multiple video frames that is temporally adjacent to the first video frame.

5. The method of claim 3, wherein the objective function indicates a second loss associated with the target image, the second loss indicating whether a first normal vector of the reference plane and a trajectory vector of the image acquisition device are perpendicular, the first normal vector corresponding to the target image.

6. The method of claim 5, wherein the objective function further indicates a third loss associated with the target image, the third loss indicating a difference between the independent extrinsic parameters and the unified extrinsic parameters.

7. The method according to claim 6, wherein: In the objective function: A first weight corresponding to the first loss is greater than a third weight corresponding to the third loss; or A second weight corresponding to the second loss is greater than the third weight of the third loss.

8. The method of claim 1 , wherein projecting the plurality of images based on the plurality of independent external parameters to generate a bird's-eye view image comprises: Based on the plurality of independent external parameters, performing an inverse perspective transformation IPM for the plurality of images to generate a plurality of IPM maps for the plurality of images; as well as The bird's-eye view image is generated by combining the plurality of IPM maps.

9. The method of claim 1, wherein determining unified external parameters for the image acquisition device comprises: Based on the multiple images, generating a three-dimensional reconstruction model; as well as Based on the three-dimensional reconstructed model, the unified external parameters for the image acquisition device are determined.

10. The method according to claim 1, wherein the image acquisition device is carried by a acquisition vehicle, and the reference plane is a road surface on which the vehicle is traveling.

11. A device for generating a bird's-eye view image, comprising: A determination module is configured to determine a unified external parameter for the image acquisition device based on a plurality of images captured by the image acquisition device, wherein the unified external parameter indicates a conversion relationship between a coordinate system corresponding to the image acquisition device and a reference coordinate system, wherein determining the unified external parameter comprises: generating a three-dimensional reconstruction model based on the plurality of images, and calibrating the unified external parameter of the image acquisition device based on a set of points corresponding to a reference plane in the constructed three-dimensional reconstruction model and the pose information of the image; an adjustment module configured to determine a plurality of independent extrinsic parameters corresponding to the plurality of images by adjusting the unified extrinsic parameter; and The projection module is configured to project the plurality of images based on the plurality of independent external parameters to generate a bird's-eye view image.

12. An electronic device comprising: Memory and processor; The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having one or more computer instructions stored thereon, wherein the one or more computer instructions are executed by a processor to implement the method according to any one of claims 1 to 10.

14. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images