Bird-eye view image generation method and device, equipment, storage medium and vehicle

Abstract

The invention relates to a bird's-eye view image generation method and device, equipment, a storage medium and a vehicle, compared with the prior art of reducing the size of an original image, according to the embodiment of the invention, a region of interest is set, internal and external parameters of a virtual camera corresponding to the region of interest are determined, a plurality of entity cameras are used for collecting a plurality of original images, and the image processing efficiency is improved. Since the concerned image corresponding to the region of interest is obtained from the plurality of original images, the concerned image is a refined image for the region of interest, and after the size is reduced, the obstacle information loss of a distance or a small target is relatively small, and the sizes of the plurality of original images and the concerned image are further unified; the method comprises the following steps: obtaining adjusted images corresponding to a plurality of original images and a concerned image, reducing the size of the concerned image, and generating a bird's-eye view image by using each adjusted image and internal and external parameters of a camera corresponding to each adjusted image in combination with a bird's-eye view algorithm, so that the accuracy of the generated bird's-eye view image can be ensured; therefore, the effect of perception and recognition is ensured.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle perception technology, and in particular to a method, apparatus, device, storage medium, and vehicle for generating bird's-eye view images. Background Technology

[0002] In existing autonomous driving functions, identifying targets around the vehicle is crucial. A popular approach is to generate corresponding Bird's Eye View (BEV) images of these targets for identification. However, due to the nature of the BEV algorithm, images and corresponding camera parameters need to be input. This requires unifying multiple original images captured by various cameras on the vehicle to the same image size. Because the BEV algorithm consumes significant processing resources, coupled with the limited processing resources of onboard equipment, the unified image size is relatively small. While each original image covers a wide area and preserves rich details, the reduced image size also shrinks the targets within the original images. This results in significant loss of image information for distant or small targets, leading to inaccurate BEV images and poor performance in subsequent perception and recognition. Therefore, obtaining more accurate BEV images without increasing the unified image size is a pressing issue. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a method, apparatus, device, storage medium, and vehicle for generating bird's-eye view images.

[0004] In a first aspect, embodiments of this disclosure provide a method for generating a bird's-eye view image, including:

[0005] Within the field of view of multiple physical cameras, a region of interest is determined, and a virtual camera is set up for each region of interest.

[0006] Based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, and the position of the region of interest within the field of view of the physical camera, the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest are determined.

[0007] Multiple raw images are acquired using multiple physical cameras, and the image of interest corresponding to the region of interest is obtained from the multiple raw images;

[0008] The dimensions of the plurality of original images and the image of interest are unified to obtain adjusted images corresponding to the plurality of original images and the image of interest, respectively;

[0009] A bird's-eye view image is generated by combining each adjusted image and the corresponding camera intrinsic and extrinsic parameters with a bird's-eye view algorithm. The camera intrinsic and extrinsic parameters corresponding to each adjusted image include the intrinsic and extrinsic parameters of a physical camera or the intrinsic and extrinsic parameters of a virtual camera.

[0010] In some embodiments, determining the region of interest includes:

[0011] Identify the vehicle's current driving scenario, and determine the region of interest based on the current driving scenario; or

[0012] Get the pre-defined region of interest; or

[0013] The region of interest is determined in response to the user's settings for parameters related to the region of interest.

[0014] In some embodiments, identifying the current driving scenario of the vehicle and determining the region of interest based on the current driving scenario includes:

[0015] If the vehicle is traveling straight, the area at a preset distance directly in front of the vehicle is defined as the region of interest; or

[0016] If the vehicle turns, the region corresponding to the turning direction and the direction forward of the turning direction is defined as the region of interest; or

[0017] If there are intersections around the vehicle, the area around the intersections is defined as the region of interest.

[0018] In some embodiments, determining the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest (ROI) based on the intrinsic and extrinsic parameters of the physical camera corresponding to the ROI and the position of the ROI within the field of view of the physical camera includes:

[0019] The parameters of the region of interest are determined, including the identifier of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the positional offset of the region of interest relative to the field of view of the physical camera.

[0020] Obtain the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest;

[0021] Based on the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the positional offset of the region of interest relative to the field of view of the physical camera, the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest are calculated.

[0022] In some embodiments, acquiring multiple raw images using multiple physical cameras and obtaining the interest image corresponding to the region of interest from the multiple raw images includes:

[0023] Acquire the raw images captured by each of the multiple physical cameras;

[0024] The image of interest corresponding to the region of interest is obtained from the original image captured by the physical camera corresponding to the region of interest.

[0025] In some embodiments, generating a bird's-eye view image using each adjusted image and the corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm, includes:

[0026] Based on the camera intrinsic and extrinsic parameters corresponding to each adjusted image, each pixel in each adjusted image is transformed into a three-dimensional space to obtain a three-dimensional point cloud;

[0027] The 3D point cloud is rotated and translated using a bird's-eye view algorithm to make the 3D point cloud parallel to the vehicle ground.

[0028] The three-dimensional point cloud is projected onto a two-dimensional plane to generate a bird's-eye view image.

[0029] In some embodiments, after generating a bird's-eye view image using each adjusted image and the corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm, the method further includes:

[0030] Monitor the time taken during the generation process of the bird's-eye view image;

[0031] If the time taken to generate the bird's-eye view image is greater than a preset time threshold, then the region of interest with the lowest priority is determined as the target region based on the priority of the region of interest, and the virtual camera corresponding to the target region is turned off.

[0032] In a second aspect, embodiments of this disclosure provide a bird's-eye view image generation apparatus, comprising:

[0033] The first determination module is used to determine the region of interest within the field of view of multiple physical cameras, and set up a virtual camera for each region of interest.

[0034] The second determining module is used to determine the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest and the position of the region of interest within the field of view of the physical camera.

[0035] The first obtaining module is used to acquire multiple raw images using multiple physical cameras, and obtain the interest image corresponding to the region of interest from the multiple raw images;

[0036] The second obtaining module is used to unify the size of the plurality of original images and the image of interest to obtain an adjusted image corresponding to the plurality of original images and the image of interest respectively;

[0037] The generation module is used to generate a bird's-eye view image by combining each adjusted image and the camera intrinsic and extrinsic parameters corresponding to each adjusted image with a bird's-eye view algorithm. The camera intrinsic and extrinsic parameters corresponding to each adjusted image include the intrinsic and extrinsic parameters of a physical camera or the intrinsic and extrinsic parameters of a virtual camera.

[0038] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0039] Memory;

[0040] Processor; and

[0041] Computer programs;

[0042] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0043] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method as described in the first aspect.

[0044] Fifthly, embodiments of this disclosure provide a vehicle, including:

[0045] Memory;

[0046] Processor; and

[0047] Computer programs;

[0048] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0049] In a sixth aspect, embodiments of this disclosure also provide a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the bird's-eye view image generation method as described above.

[0050] The bird's-eye view image generation method, apparatus, device, storage medium, and vehicle provided in this disclosure are as follows: A region of interest (ROI) is determined within the field of view of multiple physical cameras. A virtual camera is set up for each ROI. Based on the intrinsic and extrinsic parameters of the physical cameras corresponding to the ROI and the position of the ROI within the field of view of the physical cameras, the intrinsic and extrinsic parameters of the virtual camera corresponding to the ROI are determined. Multiple original images are acquired using the multiple physical cameras. A focus image corresponding to the ROI is obtained from the multiple original images. The sizes of the multiple original images and the focus image are unified to obtain adjusted images corresponding to the multiple original images and the focus image, respectively. A bird's-eye view image is generated using each adjusted image and the corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm. The camera intrinsic and extrinsic parameters corresponding to each adjusted image include the intrinsic and extrinsic parameters of the physical cameras or the virtual cameras. Compared to existing technologies that reduce the size of the original image, this disclosure embodiment sets a region of interest (ROI), determines the intrinsic and extrinsic parameters of the virtual camera corresponding to the ROI, and uses multiple physical cameras to acquire multiple original images. From these multiple original images, a focus image corresponding to the ROI is obtained, resulting in a number of images exceeding the number of physical cameras. Furthermore, the sizes of the multiple original images and the focus image are unified to obtain adjusted images corresponding to the original images and the focus image, respectively. The focus image is then reduced in size. Since the focus image is a refined image of the ROI, the loss of obstacle information for distant or small targets is minimal after size reduction. Then, using each adjusted image and its corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm, a bird's-eye view image is generated, ensuring the accuracy of the generated bird's-eye view image and thus guaranteeing the effectiveness of perception and recognition. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0052] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart of a bird's-eye view image generation method provided in this embodiment of the disclosure;

[0054] Figure 2 A flowchart of a bird's-eye view image generation method provided in another embodiment of this disclosure;

[0055] Figure 3 A flowchart of a bird's-eye view image generation method provided in another embodiment of this disclosure;

[0056] Figure 4 This is a schematic diagram illustrating the process of obtaining a region of interest image corresponding to the region of interest from the original image and an adjusted image of the original image, as provided in an embodiment of this disclosure.

[0057] Figure 5 This is a schematic diagram of the structure of the bird's-eye view image generation apparatus provided in the embodiments of this disclosure;

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0059] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0060] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0061] This disclosure provides a method for generating bird's-eye view images, which will be described below with reference to specific embodiments.

[0062] Figure 1 This is a flowchart illustrating a bird's-eye view image generation method provided in this embodiment. This method can be applied to in-vehicle terminals, which can be portable mobile devices such as smartphones, tablets, laptops, in-vehicle navigation devices, and smart sports equipment; or fixed devices such as personal computers and smart home appliances. This method can be applied to scenarios involving the generation of bird's-eye views. Embodiments of this disclosure can reduce the loss of obstacle information at distant or small targets, ensuring the accuracy of the generated BEV image, thereby guaranteeing the effectiveness of perception and recognition. It is understood that the bird's-eye view image generation method provided in this embodiment can also be applied to other scenarios.

[0063] The following is about Figure 1 The method for generating the bird's-eye view image shown is described below, and the specific steps of this method are as follows:

[0064] S101. Determine the region of interest within the field of view of multiple physical cameras, and set up a virtual camera for each region of interest.

[0065] In this step, multiple physical cameras, such as six cameras, can be deployed on the vehicle, positioned at the front, rear, left front, left rear, right front, and right rear respectively. The in-vehicle terminal then determines the region of interest (ROI) within the field of view of each physical camera. The ROI is the area the user wants to focus on, and a virtual camera can be set up for each ROI. In some embodiments, the in-vehicle terminal can determine the ROI based on the current driving scenario; for example, when turning left, a certain area on the left side and / or in front of the vehicle is the ROI, and these areas can be selected from the field of view of the physical cameras. The field of view of the virtual camera corresponds to the ROI. In some embodiments, the user can pre-set the ROI, and the in-vehicle terminal acquires the pre-set ROI.

[0066] S102. Based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, and the position of the region of interest within the field of view of the physical camera, determine the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest.

[0067] After determining the region of interest (ROI), the vehicle terminal determines the intrinsic and extrinsic parameters (IMP) of the virtual camera corresponding to the ROI based on the intrinsic and extrinsic parameters of the physical camera corresponding to the ROI and the position of the ROI within the field of view of the physical camera. For example, the ROI may be located within the field of view of the physical camera, or it may be a part of the field of view of the physical camera (i.e., the field of view of the virtual camera is also a part of the field of view of the physical camera). This allows us to determine which physical camera specifically corresponds to the ROI through the correspondence between the various field of view regions. Furthermore, since the intrinsic and extrinsic parameters of the physical camera are known or obtainable using existing methods, the IMP of each part within the field of view of the physical camera can also be obtained. Subsequently, based on the IMP of the physical camera corresponding to the ROI and the position of the ROI within the field of view of the physical camera, we can obtain the conversion relationship between the intrinsic and extrinsic parameters of the physical camera and the virtual camera, thereby determining the IMP of the virtual camera corresponding to the ROI. Alternatively, the IMP of the physical camera corresponding to the ROI and the position of the ROI within the field of view of the physical camera can be input into a trained virtual camera IMP calculation model, and the IMP of the virtual camera corresponding to the ROI can be determined through this model. This disclosure may also determine the intrinsic and extrinsic parameters of the virtual camera through other means, without making specific limitations here.

[0068] S103. Multiple raw images are acquired using multiple physical cameras, and the image of interest corresponding to the region of interest is obtained from the multiple raw images.

[0069] Multiple physical cameras, such as six cameras, are deployed on the vehicle, positioned at the front, rear, front left, rear left, front right, and rear right respectively. In this step, the vehicle-mounted terminal uses these cameras to capture multiple raw images, with each camera capturing one raw image. Further, the vehicle-mounted terminal obtains the image of interest corresponding to the region of interest from these raw images. Specifically, the vehicle-mounted terminal determines the physical camera corresponding to the region of interest and obtains the image of interest corresponding to the region of interest from the raw images captured by that camera. In some embodiments, the image of interest corresponding to the region of interest can be cropped according to a preset cropping size and cropping center point.

[0070] S104. Unify the size of the plurality of original images and the image of interest to obtain an adjusted image corresponding to the plurality of original images and the image of interest respectively.

[0071] Furthermore, the vehicle-mounted terminal unifies the dimensions of the multiple original images and the image of interest, obtaining adjusted images corresponding to the multiple original images and the image of interest, respectively. For example... Figure 4 As shown, the original image size is 2160*3840, and the size of the image of interest is 640*1200. The sizes of the multiple original images and the image of interest are unified to a preset size, which can be 540*960. This preset size is user-defined and can also be any other size. It can be seen that during the process of unifying the images to the preset size, the reduction ratio of the image of interest is much smaller than that of the original image. Therefore, the target in the image of interest is reduced by a smaller proportion, and less detail is lost. Subsequently, when generating a bird's-eye view image using the bird's-eye view algorithm, the target information within the region of interest in the bird's-eye view image can be more accurate because the image corresponding to the region of interest retains more detailed information.

[0072] S105. Using each adjusted image and the corresponding camera intrinsic and extrinsic parameters, a bird's-eye view image is generated by combining the bird's-eye view algorithm. The corresponding camera intrinsic and extrinsic parameters of each adjusted image include the intrinsic and extrinsic parameters of a physical camera or the intrinsic and extrinsic parameters of a virtual camera.

[0073] After obtaining the adjusted images corresponding to the plurality of original images and the image of interest, the vehicle terminal uses each adjusted image and the corresponding camera intrinsic and extrinsic parameters to generate a bird's-eye view image using a bird's-eye view algorithm. Specifically, the adjusted image corresponding to each original image corresponds to the intrinsic and extrinsic parameters of a physical camera, and the adjusted image corresponding to the image of interest corresponds to the intrinsic and extrinsic parameters of a virtual camera. In other words, the camera intrinsic and extrinsic parameters corresponding to each adjusted image include either the intrinsic and extrinsic parameters of a physical camera or the intrinsic and extrinsic parameters of a virtual camera.

[0074] This embodiment of the disclosure determines a region of interest (ROI) within the field of view of multiple physical cameras, sets up a virtual camera for each ROI, and determines the ROI of the virtual camera corresponding to the ROI based on the intrinsic and extrinsic parameters of the physical cameras corresponding to the ROI and the position of the ROI within the field of view of the physical cameras. Multiple original images are acquired using the multiple physical cameras, and a focus image corresponding to the ROI is obtained from the multiple original images. The sizes of the multiple original images and the focus image are unified to obtain adjusted images corresponding to the multiple original images and the focus image, respectively. A bird's-eye view image is generated using each adjusted image and its corresponding camera ROI, combined with a bird's-eye view algorithm. The camera ROI of each adjusted image includes the ROI of the physical cameras or the ROI of the virtual cameras. Compared to existing technologies that reduce the size of the original image, this disclosure embodiment sets a region of interest (ROI), determines the intrinsic and extrinsic parameters of the virtual camera corresponding to the ROI, and uses multiple physical cameras to acquire multiple original images. From these multiple original images, a focus image corresponding to the ROI is obtained, resulting in a number of images exceeding the number of physical cameras. Furthermore, the sizes of the multiple original images and the focus image are unified to obtain adjusted images corresponding to the original images and the focus image, respectively. The focus image is then reduced in size. Since the focus image is a refined image of the ROI, the loss of obstacle information for distant or small targets is minimal after size reduction. Then, using each adjusted image and its corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm, a bird's-eye view image is generated, ensuring the accuracy of the generated bird's-eye view image and thus guaranteeing the effectiveness of perception and recognition.

[0075] Figure 2 A flowchart of a bird's-eye view image generation method provided in another embodiment of this disclosure is shown below. Figure 2 As shown, the method includes the following steps:

[0076] S201. Determine the region of interest within the field of view of multiple physical cameras, and set up a virtual camera for each region of interest.

[0077] Specifically, the implementation process and principle of S201 and S101 are the same, and will not be repeated here.

[0078] In some embodiments, S201 may include, but is not limited to, S2011, S2012, or S2013:

[0079] S2011. Identify the current driving scenario of the vehicle, and determine the region of interest based on the current driving scenario of the vehicle.

[0080] In some embodiments, S2011 may include, but is not limited to, step a, step b, or step c:

[0081] Step a: If the vehicle is traveling straight, the area at a preset distance directly in front of the vehicle is determined as the region of interest;

[0082] For example, the preset distance can be 30m, without limitation.

[0083] Step b: If the vehicle turns, the region corresponding to the turning direction and the direction in front of the turning direction is determined as the region of interest.

[0084] For example, if the vehicle turns right, the area corresponding to the right direction and the right front direction of the vehicle is determined as the region of interest.

[0085] Step c: If there are intersections around the vehicle, then the area around the intersections is identified as the region of interest.

[0086] S2012, Obtain the pre-defined region of interest.

[0087] like Figure 4 As shown, users can pre-set areas of interest, and the vehicle terminal can obtain these pre-set areas of interest.

[0088] S2013. In response to the user's setting of parameters for the region of interest, the region of interest is determined.

[0089] For example, users can set parameters for the region of interest, and the vehicle terminal can determine the region of interest in response to the user's parameter settings.

[0090] S202. Determine the parameters of the region of interest, including the identifier of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the positional offset relationship of the region of interest relative to the field of view of the physical camera.

[0091] After determining the region of interest (ROI), the vehicle-mounted terminal acquires parameters of the ROI. In some embodiments, the parameters of the ROI may include the identifier of the physical camera corresponding to the ROI, the scaling ratio of the ROI to the field of view of the physical camera, the positional offset of the ROI relative to the field of view of the physical camera, and other parameters such as the priority of the ROI, which are not limited here. Figure 4 As shown, the scaling ratio is [0.5, 0.5], and the scaling ratios for both the horizontal and vertical axes are 0.5.

[0092] S203. Obtain the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest.

[0093] For example, the calibrated intrinsic and extrinsic parameters of each physical camera can be stored in the vehicle-mounted terminal or in a server; this embodiment of the disclosure does not impose any limitation. In this step, the vehicle-mounted terminal acquires the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest.

[0094] S204. Based on the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the positional offset relationship of the region of interest relative to the field of view of the physical camera, calculate the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest.

[0095] Specifically, the calibrated intrinsic and extrinsic parameters of the physical camera: R (rotation matrix), T 3×1 (Translation matrix) is the extrinsic parameter matrix of the physical camera, K is the intrinsic parameter matrix of the physical camera, and (u, v) are the pixel coordinates in the image coordinate system of the physical camera. The transformation relationship from the image coordinate system of the physical camera to the world coordinate system is as follows:

[0096]

[0097] in,

[0098] The translation matrix T of the virtual camera corresponding to the region of interest 3×3 As follows: r x and r y It's the scaling factor, d x and d y It is the positional offset relationship between the region of interest and the field of view of the physical camera, d x d represents the offset of the center point (or top-left corner) of the region of interest relative to the center point (or top-left corner) of the field of view of the physical camera along the x-axis. y This represents the offset of the center point (or top-left corner) of the region of interest relative to the center point (or top-left corner) of the field of view of the physical camera along the y-axis.

[0099]

[0100] The derivation process of the virtual camera's intrinsic parameters is as follows: (u', v') are the pixel coordinates in the image coordinate system of the virtual camera.

[0101]

[0102] The intrinsic parameter matrix K of the physical camera and the translation matrix T of the virtual camera. 3×3The intrinsic parameter matrix of the virtual camera can be calculated as follows:

[0103]

[0104] S205. Acquire the raw images captured by each of the multiple physical cameras.

[0105] The vehicle is equipped with multiple physical cameras, such as six cameras, positioned at the front, rear, front left, rear left, front right, and rear right of the vehicle. In this step, the onboard terminal acquires multiple raw images from these physical cameras, with each physical camera capturing one raw image.

[0106] S206. Obtain the image of interest corresponding to the region of interest from the original image captured by the physical camera corresponding to the region of interest.

[0107] Furthermore, the vehicle-mounted terminal identifies the physical camera corresponding to the region of interest and obtains the image of interest corresponding to the region of interest from the original image captured by the physical camera. In some embodiments, the image of interest corresponding to the region of interest can be cropped according to a preset cropping size and cropping center point. Figure 4 As shown, the cropping center point of the image of interest corresponding to virtual camera A, i.e., the image of interest corresponding to region of interest A, is [950, 500], and the cropping size is 640*1200.

[0108] S207. Unify the size of the plurality of original images and the image of interest to obtain an adjusted image corresponding to the plurality of original images and the image of interest, respectively.

[0109] Specifically, the implementation process and principle of S207 and S104 are the same, and will not be repeated here.

[0110] S208. Using each adjusted image and the corresponding camera intrinsic and extrinsic parameters of each adjusted image, a bird's-eye view image is generated by combining the bird's-eye view algorithm. The corresponding camera intrinsic and extrinsic parameters of each adjusted image include the intrinsic and extrinsic parameters of a physical camera or the intrinsic and extrinsic parameters of a virtual camera.

[0111] Specifically, the implementation process and principle of S208 and S105 are the same, and will not be repeated here.

[0112] Compared to existing technologies, this disclosure embodiment determines a region of interest (ROI) and its parameters. These parameters include the identifier of the physical camera corresponding to the ROI, the scaling ratio between the ROI and the field of view of the physical camera, and the positional offset of the ROI relative to the field of view of the physical camera. Then, the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the ROI are obtained. Based on these parameters, the scaling ratio, and the positional offset, the intrinsic and extrinsic parameters of the virtual camera corresponding to the ROI are calculated. Further, original images from each of the multiple physical cameras are acquired. A focus image corresponding to the ROI is obtained from these original images. The dimensions of the original images and the focus image are unified to obtain adjusted images corresponding to both. Finally, a bird's-eye view image is generated using each adjusted image and its corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm. The camera intrinsic and extrinsic parameters of each adjusted image include those of the physical camera or the virtual camera. Compared to existing technologies that reduce the size of the original image, this embodiment unifies the size of the multiple original images and the image of interest to obtain adjusted images corresponding to the multiple original images and the image of interest, respectively. The image of interest is reduced in size. Since the image of interest is a refined image of the region of interest, the loss of obstacle information of distant or small targets is small after the size is reduced. Then, by using each adjusted image and the camera intrinsic and extrinsic parameters corresponding to each adjusted image, combined with the bird's-eye view algorithm, a bird's-eye view image can be generated, which can ensure the accuracy of the generated bird's-eye view image, thereby ensuring the effect of perception and recognition.

[0113] Figure 3 A flowchart of a bird's-eye view image generation method provided in another embodiment of this disclosure is shown below. Figure 3 As shown, the method includes the following steps:

[0114] S301. Determine the region of interest within the field of view of multiple physical cameras, and set up a virtual camera for each region of interest.

[0115] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0116] S302. Based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, and the position of the region of interest within the field of view of the physical camera, determine the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest.

[0117] Specifically, the implementation process and principle of S302 and S102 are the same, and will not be repeated here.

[0118] S303. Acquire multiple raw images using multiple physical cameras, and obtain the image of interest corresponding to the region of interest from the multiple raw images.

[0119] Specifically, the implementation process and principle of S303 and S103 are the same, and will not be repeated here.

[0120] S304. Unify the size of the plurality of original images and the image of interest to obtain an adjusted image corresponding to the plurality of original images and the image of interest respectively.

[0121] Specifically, the implementation process and principle of S304 and S104 are the same, and will not be repeated here.

[0122] S305. Based on the camera intrinsic and extrinsic parameters corresponding to each adjusted image, each pixel in each adjusted image is converted into a three-dimensional space to obtain a three-dimensional point cloud.

[0123] After obtaining the adjusted images corresponding to the multiple original images and the image of interest, the vehicle terminal transforms each pixel in each adjusted image into a three-dimensional space based on the camera intrinsic and extrinsic parameters corresponding to each adjusted image, thus obtaining a three-dimensional point cloud. Specifically, based on the camera intrinsic parameters corresponding to each adjusted image, each adjusted image is transformed from the image coordinate system to the camera coordinate system. Furthermore, based on the camera extrinsic parameters corresponding to each adjusted image, each adjusted image is transformed from the camera coordinate system to the three-dimensional space of the vehicle coordinate system, thus obtaining a three-dimensional point cloud.

[0124] S306. Using a bird's-eye view algorithm, rotate and translate the three-dimensional point cloud so that the three-dimensional point cloud is parallel to the vehicle ground.

[0125] The vehicle-mounted terminal rotates and translates the three-dimensional point cloud so that the three-dimensional point cloud is parallel to the vehicle ground.

[0126] S307. Project the three-dimensional point cloud onto a two-dimensional plane to generate a bird's-eye view image.

[0127] Furthermore, the vehicle-mounted terminal projects the three-dimensional point cloud onto a two-dimensional plane to generate a bird's-eye view image.

[0128] S308. Monitor the time spent in the process of generating the bird's-eye view image.

[0129] In this step, the vehicle-mounted terminal monitors the time consumed in the generation process of the bird's-eye view image. Essentially, this step determines whether the vehicle-mounted terminal's computing power is insufficient to meet the perception requirements of the bird's-eye view image. If the time consumed in generating the bird's-eye view image exceeds a preset time threshold, it is determined that the vehicle-mounted terminal's computing power is insufficient to meet the perception requirements of the bird's-eye view image. Therefore, the number of input images needs to be reduced, specifically the number of images corresponding to the region of interest, to better meet the requirements for perception using the bird's-eye view image.

[0130] S309. If the time taken to generate the bird's-eye view image is greater than a preset time threshold, then the region of interest with the lowest priority is determined as the target region based on the priority of the region of interest, and the virtual camera corresponding to the target region is turned off.

[0131] If the time taken to generate the bird's-eye view image exceeds a preset time threshold, the region of interest with the lowest priority is determined as the target region based on the priority of the region of interest, and the virtual camera corresponding to the target region is turned off, thereby reducing the number of input attention images corresponding to the region of interest, so as to better meet the requirements of using bird's-eye view images for perception.

[0132] In some embodiments, if after turning off the virtual camera corresponding to the target area, it is detected that the time consumed in the generation process of the bird's-eye view image is still greater than a preset time threshold, then the region of interest with the lowest priority among the remaining regions of interest is determined as the target area, the virtual camera corresponding to the target area is turned off, and the number of input attention images corresponding to the region of interest is further reduced, so as to better meet the requirements of using bird's-eye view images for perception.

[0133] In some embodiments, if the time taken to generate the bird's-eye view image is less than or equal to a preset time threshold, then the bird's-eye view image generated using the original image and the image of interest corresponding to the region of interest is used for perception.

[0134] This embodiment of the disclosure determines regions of interest (ROIs) within the fields of view of multiple physical cameras. A virtual camera is set up for each ROI. Based on the intrinsic and extrinsic parameters of the physical cameras corresponding to the ROIs and the position of the ROIs within the fields of view of the physical cameras, the intrinsic and extrinsic parameters of the virtual camera corresponding to the ROIs are determined. Multiple raw images are acquired using the multiple physical cameras, and a focus image corresponding to the ROI is obtained from these raw images. Further, the dimensions of the multiple raw images and the focus image are unified to obtain adjusted images corresponding to the original images and the focus image, respectively. Based on the camera intrinsic and extrinsic parameters of each adjusted image, each pixel in each adjusted image is transformed into a three-dimensional space to obtain a three-dimensional point cloud. Then, a bird's-eye view algorithm is used to rotate and translate the three-dimensional point cloud so that it is parallel to the vehicle ground. The three-dimensional point cloud is projected onto a two-dimensional plane to generate a bird's-eye view image. The time consumed in generating the bird's-eye view image is monitored. If the time consumed in generating the bird's-eye view image exceeds a preset time threshold, the lowest priority ROI is determined as the target region, and the virtual camera corresponding to the target region is turned off. Compared to existing technologies, since the time spent monitoring the generation process of the bird's-eye view image is greater than a preset time threshold, the lowest priority region of interest is determined as the target region based on the priority of the region of interest, and the virtual camera corresponding to the target region is turned off, thereby reducing the number of input attention images corresponding to the region of interest, so as to better meet the requirements of using bird's-eye view images for perception.

[0135] Figure 5 This is a schematic diagram of the structure of the bird's-eye view image generation device provided in the embodiments of this disclosure. The bird's-eye view image generation device may be an in-vehicle terminal as described in the above embodiments, or it may be a component or assembly within the in-vehicle terminal. The bird's-eye view image generation device provided in the embodiments of this disclosure can execute the processing flow provided in the bird's-eye view image generation method embodiments, such as... Figure 5As shown, the bird's-eye view image generation device 40 includes: a first determining module 41, a second determining module 42, a first obtaining module 43, a second obtaining module 44, and a generation module 45; wherein, the first determining module 41 is used to determine the region of interest within the field of view of multiple physical cameras, and set a virtual camera for each region of interest; the second determining module 42 is used to determine the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, and the position of the region of interest within the field of view of the physical camera; the first obtaining module 43 is used to acquire multiple original images using multiple physical cameras, and obtain the image of interest corresponding to the region of interest from the multiple original images; the second obtaining module 44 is used to unify the size of the multiple original images and the image of interest, and obtain adjusted images corresponding to the multiple original images and the image of interest respectively; the generation module 45 is used to generate a bird's-eye view image using each adjusted image and the camera intrinsic and extrinsic parameters corresponding to each adjusted image, combined with a bird's-eye view algorithm, wherein the camera intrinsic and extrinsic parameters corresponding to each adjusted image include the intrinsic and extrinsic parameters of the physical camera or the intrinsic and extrinsic parameters of the virtual camera.

[0136] Optionally, when the first determining module 41 determines the region of interest, it is specifically used to: identify the current driving scenario of the vehicle and determine the region of interest based on the current driving scenario of the vehicle; or obtain a pre-set region of interest; or determine the region of interest in response to the user's setting of parameters for the region of interest.

[0137] Optionally, the first determining module 41 identifies the current driving scenario of the vehicle, and when determining the region of interest based on the current driving scenario of the vehicle, it is specifically used as follows: if the vehicle is going straight, the area at a preset distance directly in front of the vehicle is determined as the region of interest; or if the vehicle is turning, the area corresponding to the turning direction and the direction in front of the turning direction is determined as the region of interest; or if there is an intersection around the vehicle, the area of ​​the intersection around the vehicle is determined as the region of interest.

[0138] Optionally, when the second determining module 42 determines the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest and the position of the region of interest within the field of view of the physical camera, it is specifically used to: determine the parameters of the region of interest, the parameters of the region of interest including the identifier of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the positional offset relationship of the region of interest relative to the field of view of the physical camera; obtain the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest; and calculate the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest based on the calibrated intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the positional offset relationship of the region of interest relative to the field of view of the physical camera.

[0139] Optionally, when the first obtaining module 43 acquires multiple original images using multiple physical cameras and obtains the attention image corresponding to the region of interest from the multiple original images, it is specifically used to: acquire the original image acquired by each of the multiple physical cameras; and obtain the attention image corresponding to the region of interest from the original image acquired by the physical camera corresponding to the region of interest.

[0140] Optionally, when the generation module 45 generates a bird's-eye view image using each adjusted image and the corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm, it specifically performs the following steps: based on the corresponding camera intrinsic and extrinsic parameters, converting each pixel in each adjusted image into a three-dimensional space to obtain a three-dimensional point cloud; rotating and translating the three-dimensional point cloud using a bird's-eye view algorithm to make the three-dimensional point cloud parallel to the vehicle ground; and projecting the three-dimensional point cloud onto a two-dimensional plane to generate a bird's-eye view image.

[0141] Optionally, after generating a bird's-eye view image using each adjusted image and the corresponding camera intrinsic and extrinsic parameters, combined with a bird's-eye view algorithm, the device 40 further includes: a monitoring module 46 and a shut-down module 47; wherein, the monitoring module 46 is used to monitor the time consumed in the generation process of the bird's-eye view image; the shut-down module 47 is used to determine the lowest priority region of interest as the target region based on the priority of the region of interest if the time consumed in the generation process of the bird's-eye view image is greater than a preset time threshold, and shut down the virtual camera corresponding to the target region.

[0142] Figure 5 The bird's-eye view image generation device shown in the embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0143] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 6 It shows a schematic diagram of a structure suitable for implementing the electronic device 600 in the embodiments of this disclosure. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0144] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603 to implement the bird's-eye view image generation method as described in the embodiments of this disclosure. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0145] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0146] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the bird's-eye view image generation method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0147] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0148] Additionally, this disclosure also provides a vehicle, including: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the bird's-eye view image generation method as described above.

[0149] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0150] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0151] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0152] Within the field of view of multiple physical cameras, a region of interest is determined, and a virtual camera is set up for each region of interest.

[0153] Based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest, and the position of the region of interest within the field of view of the physical camera, the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest are determined.

[0154] Multiple raw images are acquired using multiple physical cameras, and the image of interest corresponding to the region of interest is obtained from the multiple raw images;

[0155] The dimensions of the plurality of original images and the image of interest are unified to obtain adjusted images corresponding to the plurality of original images and the image of interest, respectively;

[0156] A bird's-eye view image is generated by combining each adjusted image and the corresponding camera intrinsic and extrinsic parameters with a bird's-eye view algorithm. The camera intrinsic and extrinsic parameters corresponding to each adjusted image include the intrinsic and extrinsic parameters of a physical camera or the intrinsic and extrinsic parameters of a virtual camera.

[0157] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0158] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0160] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0161] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0162] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0163] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0164] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0165] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating a bird's-eye view image, characterized in that: The method comprises: Determine the region of interest within the field of view corresponding to the multiple physical cameras, and set a virtual camera corresponding to each region of interest; Determine the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest and the position of the region of interest within the field of view of the physical camera; Using multiple physical cameras to collect multiple original images, and obtaining a focus image corresponding to the region of interest from the multiple original images; Unifying the sizes of the multiple original images and the focus image to obtain adjusted images corresponding to the multiple original images and the focus image respectively; A bird's-eye view image is generated by using each adjusted image and the camera internal and external parameters corresponding to each adjusted image in combination with a bird's-eye view algorithm. The camera internal and external parameters corresponding to each adjusted image include the internal and external parameters of a physical camera or the internal and external parameters of a virtual camera.

2. The method according to claim 1, characterized in that The determining of the region of interest comprises: Identify a current driving scene of the vehicle, and determine an area of ​​interest based on the current driving scene of the vehicle; or Acquire a pre-defined region of interest; or In response to the user setting of the parameters of the region of interest, the region of interest is determined.

3. The method according to claim 2, characterized in that The identifying the current driving scene of the vehicle and determining the region of interest according to the current driving scene of the vehicle includes: If the vehicle is traveling straight, an area at a preset distance directly in front of the vehicle is determined as the area of ​​interest; or If the vehicle turns, then the area corresponding to the vehicle turning direction and the direction forward of the turning direction is determined as the area of ​​interest; or If there is an intersection around the vehicle, the area of ​​the intersection around the vehicle is determined as the area of ​​interest.

4. The method according to claim 1, characterized in that: The determining, based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest and the position of the region of interest within the field of view of the physical camera, the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest comprises: Determine parameters of the region of interest, wherein the parameters of the region of interest include an identifier of a physical camera corresponding to the region of interest, a zoom ratio of the region of interest to a field of view of the physical camera, and a position offset relationship of the region of interest relative to the field of view of the physical camera; Obtaining calibrated internal and external parameters of the physical camera corresponding to the region of interest; Based on the calibrated internal and external parameters of the physical camera corresponding to the region of interest, the scaling ratio of the region of interest to the field of view of the physical camera, and the position offset relationship between the region of interest and the field of view of the physical camera, the internal and external parameters of the virtual camera corresponding to the region of interest are calculated.

5. The method according to claim 1, characterized in that The method of collecting a plurality of original images by using a plurality of physical cameras and obtaining the focus image corresponding to the region of interest from the plurality of original images includes: Acquire an original image captured by each of the multiple physical cameras; The focus image corresponding to the region of interest is obtained from the original image captured by the physical camera corresponding to the region of interest.

6. The method according to claim 1, characterized in that The step of using each adjusted image and the camera internal and external parameters corresponding to each adjusted image in combination with a bird's-eye view algorithm to generate a bird's-eye view image includes: Based on the camera internal and external parameters corresponding to each adjusted image, each pixel in each adjusted image is converted into a three-dimensional space to obtain a three-dimensional point cloud; Rotating and translating the three-dimensional point cloud in combination with a bird's-eye view algorithm so that the three-dimensional point cloud is parallel to the vehicle ground; The three-dimensional point cloud is projected onto a two-dimensional plane to generate a bird's-eye view image.

7. The method according to claim 1, characterized in that After generating a bird's-eye view image by using each adjusted image and the camera internal and external parameters corresponding to each adjusted image in combination with a bird's-eye view algorithm, the method further includes: Monitor the time taken to generate the bird's eye view image; If the time consumed in the generation process of the bird's-eye view image is greater than a preset time threshold, the region of interest corresponding to the lowest priority is determined as the target region according to the priorities of the regions of interest, and the corresponding virtual camera of the target region is closed.

8. A bird's-eye view image generating device, characterized in that: include: A first determination module is used to determine the region of interest within the field of view corresponding to the multiple physical cameras, and to set a virtual camera corresponding to each region of interest; A second determination module is used to determine the intrinsic and extrinsic parameters of the virtual camera corresponding to the region of interest based on the intrinsic and extrinsic parameters of the physical camera corresponding to the region of interest and the position of the region of interest within the field of view of the physical camera; A first obtaining module is used to collect a plurality of original images using a plurality of physical cameras, and obtain a focus image corresponding to the region of interest from the plurality of original images; A second obtaining module is used to unify the sizes of the multiple original images and the focus image to obtain adjusted images corresponding to the multiple original images and the focus image respectively; A generation module is used to generate a bird's-eye view image by using each adjusted image and the camera internal and external parameters corresponding to each adjusted image in combination with a bird's-eye view algorithm. The camera internal and external parameters corresponding to each adjusted image include the internal and external parameters of a physical camera or the internal and external parameters of a virtual camera.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

Images