3D view generation method and device, related equipment and computer program product

Abstract

The invention discloses a 3D view generation method and device, related equipment and a computer program product, and the method comprises the steps: obtaining the distance information of an obstacle around a vehicle, adaptively adjusting the size of the bowl bottom radius of a 3D bowl-shaped model according to the distance, and if the obstacle distance is smaller than a set distance threshold value, generating a 3D view. The size of the bowl bottom radius is adjusted to be smaller than the default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the obstacle distance is not smaller than the set distance threshold value. When it is detected that the distance of the obstacle is too close, the bowl bottom radius size of the 3D bowl-shaped model is reduced, so that the stretching amount when the obstacle image is mapped to the 3D bowl-shaped model is relieved, the size of the obstacle after mapping better conforms to the actual situation, and the quality of the generated 3D view is improved.

Description

Technical Field

[0001] The present application relates to the technical field of 3D view generation, and more specifically, to a 3D view generation method, apparatus, related equipment and computer program product. Background Art

[0002] In the field of active vehicle safety, multiple wide-angle fisheye cameras in the Around View Monitor (AVM) can capture images around the vehicle and stitch them into a 3D view for users to browse real-time images around the vehicle in the car.

[0003] Currently, most vehicles are not equipped with devices such as laser radar that have the ability to collect 3D spatial information. Fisheye cameras can only capture two-dimensional image information. In order to model the real world, one solution is to use a 3D bowl-shaped model to map the image captured by the camera onto the 3D bowl-shaped model to obtain a 3D view, allowing users to feel like they are observing the world around them from a 3D free perspective. The 3D bowl-shaped model assumes that the world is a 3D bowl, and the images captured by the camera are all from the mapped images on this 3D bowl. The world coordinates of the bottom of the 3D bowl are 0, and the world coordinates of the edge of the 3D bowl are greater than 0.

[0004] Although the above solution can realize the creation of 3D views, there are some problems in actual use. For example, when the vehicle is very close to surrounding obstacles (such as pedestrians, other vehicles, walls, pillars, etc.), the traditional modeling of world coordinates based on a fixed 3D bowl-shaped model will cause serious stretching when mapping the image taken by the camera to the 3D bowl-shaped model, and the size ratio of the object will be seriously out of balance. Figure 1 As shown, when the traditional 3D bowl-shaped model is used to generate the 3D view, since there is a light truck very close to the front of the current vehicle, the light truck will be stretched very high after being mapped to the 3D bowl-shaped model, resulting in the vehicle size ratio being obviously inconsistent with reality, affecting the user's viewing experience. Summary of the invention

[0005] In view of the above problems, this application is proposed to provide a 3D view generation method, apparatus, related equipment and computer program product to solve the problem that when a 3D bowl-shaped model is used to generate a 3D view in the traditional way, close objects will be stretched, resulting in the object size ratio not being in line with reality, affecting the user's viewing. The specific solution is as follows:

[0006] In a first aspect, a 3D view generation method is provided, comprising:

[0007] Obtain distance information of obstacles around the vehicle;

[0008] adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information, wherein when the distance information is less than a set distance threshold, the bowl bottom radius size of the 3D bowl-shaped model is less than a default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the distance information is not less than the set distance threshold;

[0009] The world coordinates are modeled according to the 3D bowl-shaped model, and according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera, the 3D bowl-shaped model is mapped using the image taken by the camera to obtain a 3D view, which is then output for display.

[0010] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes:

[0011] For each camera orientation:

[0012] Obtaining distance information of obstacles in the direction;

[0013] According to the distance information of the obstacle in the direction, the bottom radius of the 3D bowl-shaped model used to model the world coordinates in the direction is adjusted to obtain an adjusted 3D bowl-shaped model corresponding to the camera in the direction;

[0014] The process of modeling the world coordinates according to the 3D bowl-shaped model, and mapping the 3D bowl-shaped model using the image captured by the camera according to the conversion relationship between the world coordinate system and the image coordinate system captured by the camera to obtain the 3D view includes:

[0015] For each camera orientation:

[0016] Modeling the world coordinates of the direction using the 3D bowl-shaped model corresponding to the camera in the direction, and mapping the 3D bowl-shaped model using the image taken by the camera in the direction according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera in the direction;

[0017] After the 3D bowl-shaped model is mapped using images taken by cameras in all directions, a 3D stitched view is obtained.

[0018] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes:

[0019] When the distance information is not less than the set distance threshold, maintaining the size of the bowl bottom radius of the 3D bowl-shaped model as the default radius size;

[0020] When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a first size, where the first size is a fixed value smaller than the default radius size.

[0021] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes:

[0022] When the distance information is not less than the set distance threshold, maintaining the size of the bowl bottom radius of the 3D bowl-shaped model as the default radius size;

[0023] When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a second size, where the second size is smaller than the default radius size and becomes smaller as the distance information becomes smaller.

[0024] In a possible design, in another implementation of the first aspect of the embodiment of the present application, before outputting and displaying the 3D view, the method further includes:

[0025] According to the current position of the vehicle, the UI image of the vehicle is rendered into the 3D view, and the processed 3D view is obtained for output display.

[0026] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the formula of the 3D bowl-shaped model is expressed as follows:

[0027]

[0028] Among them, Z w Indicates the world coordinates of the 3D bowl edge, X w Indicates the X-axis coordinate value of a point in the world coordinate system, Y w It represents the Y-axis coordinate value of a position point in the world coordinate system, BowlRadius represents the radius of the bowl bottom, and BowlHeightScale represents the stretching coefficient of the height of the 3D bowl edge.

[0029] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the set distance threshold has a value range of 0.3m-1.5m.

[0030] In a third aspect, a 3D view generating device is provided, comprising:

[0031] A distance detection unit, used to obtain distance information of obstacles around the vehicle;

[0032] a bowl bottom radius adjustment unit, configured to adjust the bowl bottom radius of the 3D bowl-shaped model according to the distance information, wherein when the distance information is less than a set distance threshold, the bowl bottom radius size of the 3D bowl-shaped model is less than a default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the distance information is not less than the set distance threshold;

[0033] The mapping unit is used to model the world coordinates according to the 3D bowl-shaped model, and according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera, use the image taken by the camera to map the 3D bowl-shaped model, obtain a 3D view, and output it for display.

[0034] In a fourth aspect, an electronic device is provided, comprising: a memory and a processor;

[0035] The memory is used to store programs;

[0036] The processor is used to execute the program to implement each step of the 3D view generation method as described in any one of the first aspects of the present application.

[0037] In a fifth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the 3D view generation method as described in any one of the first aspects of the present application are implemented.

[0038] In a sixth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the various steps of the 3D view generation method as described in any one of the first aspects of the present application.

[0039] It can be known from the above technical solution that the present application obtains the distance information of obstacles around the vehicle and adaptively adjusts the size of the bowl bottom radius of the 3D bowl-shaped model according to the distance size. For example, when the obstacle distance is less than the set distance threshold, the size of the bowl bottom radius is adjusted to be smaller than the default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the obstacle distance is not less than the set distance threshold. When the present application detects that the obstacle distance is too close, by reducing the bowl bottom radius size of the 3D bowl-shaped model, the stretching amount when the obstacle image is mapped to the 3D bowl-shaped model is alleviated, so that the size of the obstacle after mapping is more in line with the actual situation, thereby improving the quality of the generated 3D view. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0041] Figure 1 An example of a vehicle 3D view effect generated using an existing 3D bowl-shaped model is shown;

[0042] Figure 2 A schematic diagram of a 3D view generation method provided in an embodiment of the present application;

[0043] Figure 3 An example of a vehicle 3D view effect generated by the method of the present application is shown;

[0044] Figure 4 A schematic diagram of the modeling effect of a 3D bowl-shaped model is shown as an example;

[0045] Figure 5 A schematic diagram of the structure of a 3D view generating device provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] In the field of active vehicle safety, multiple wide-angle fisheye cameras in the Around View Monitor (AVM) can capture images of the vehicle's surroundings and stitch them into a 3D view, allowing users to feel like they are observing the world around them from a 3D free perspective.

[0049] There is a fixed conversion relationship between the world coordinate system where the vehicle is located and the image coordinate system captured by the camera on the vehicle. The conversion relationship between the world coordinate system and the image coordinate system captured by the camera can be calculated based on the camera's extrinsic rotation matrix R, extrinsic translation vector T and intrinsic matrix K.

[0050] In one possible implementation, the world coordinate system P is defined as world (X w ,Y w ,Z w ), where the x-axis is the front of the vehicle, the y-axis is the left side of the vehicle, and the z-axis is the top of the vehicle. The origin of the world coordinates is O world At ground level.

[0051] Define the camera coordinate system P camera (Xc ,Y c ,Z c ), where the x-axis is to the right of the camera, the y-axis is below the camera, and the z-axis is in front of the camera.

[0052] Therefore, the world coordinate system P world (X w ,Y w ,Z w ) and the camera coordinate system P camera (X c ,Y c ,Z c ) has a rotation R and translation T relationship. Through the camera's external parameter rotation matrix R, external parameter translation vector T and internal parameter matrix K, we can find any point P in the world coordinate system. world (X w ,Y w ,Z w ) in the pixel coordinates P in the image captured by the camera image (u,v). The formula is as follows:

[0053]

[0054] That is, ZcP image =K(RP word +t)=KTP word =KP camera .

[0055] Of course, the above is only an example of an optional definition method of the world coordinate system and the camera coordinate system. In addition, the world coordinate system and the camera coordinate system can be defined separately in other ways, and the conversion relationship between the two coordinate systems can be calculated accordingly.

[0056] The existing solution models the 3D bowl-shaped model in world coordinates: that is, the world coordinate Z of the ground plane w Fixed to 0. In order to allow users to see objects above the ground plane in the 3D view, it is necessary to imagine a 3D bowl in the world. The images taken by the 4 wide-angle fisheye cameras in the AVM system are all from the texture image on this 3D bowl. The bottom of the 3D bowl is a circle, and the world coordinates of the bottom of the bowl are Z. w 0, the world coordinate Z of the edge of the 3D bowl w Greater than 0.

[0057] Through the conversion relationship between the above world coordinate system and the image coordinate system captured by the camera, any world coordinate P on the 3D bowl can be obtained. world (X w ,Y w ,Z w ) in the pixel coordinates P in the image captured by the cameraimage (u,v), the image mapping of the 3D bowl is completed.

[0058] Although the existing solutions can realize the creation of 3D views, there are some problems in actual use. For example, when the vehicle is very close to surrounding obstacles (such as pedestrians, other vehicles, walls, pillars, etc.), the traditional method of modeling the world coordinates based on a fixed 3D bowl-shaped model will cause serious stretching when mapping the image captured by the camera to the 3D bowl-shaped model, and the object size ratio will be seriously unbalanced, not three-dimensional enough, affecting the user's viewing experience.

[0059] To this end, the present application provides a 3D view generation method, which can be applied to a vehicle-mounted terminal, such as a vehicle computer, to display the generated 3D view on the vehicle computer for users to browse the scene outside the vehicle.

[0060] Next, combine Figure 2 The 3D view generation method of the present application may include the following steps:

[0061] Step S100: Obtain distance information of obstacles around the vehicle.

[0062] Specifically, the distance information of the surrounding obstacles can be measured by a distance measurement module installed on the vehicle. For example, the distance measurement module can be an ultrasonic radar or other equipment.

[0063] In this step, obstacle distance information in any one or more directions of the front, rear, left and right sides of the vehicle can be obtained. Obstacles generally refer to other tangible objects other than the vehicle, such as other vehicles, walls, pillars, trees, pedestrians, etc.

[0064] Step S110: adjusting the bottom radius of the 3D bowl-shaped model according to the distance information.

[0065] In this embodiment, considering that the distance between the obstacle and the vehicle is different, the stretching of the obstacle image after being mapped to the 3D bowl-shaped model will be different. In particular, when the distance between the obstacle and the vehicle is too close, the obstacle image will be severely stretched after being mapped to the 3D bowl-shaped model, resulting in an imbalance in the size ratio of the object. In order to alleviate this problem, in this step, a fixed 3D bowl-shaped model is not used, but the bowl bottom radius of the 3D bowl-shaped model is dynamically adjusted according to the different distance information of the obstacle. The change of the bowl bottom radius will affect the stretching of the object image after being mapped to the 3D bowl-shaped model.

[0066] Specifically, when the distance information is not less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model can be set to a default radius size; when the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model can be set to be smaller than the default radius size.

[0067] The distance threshold may be set in the range of 0.3m-1.5m.

[0068] By dynamically adjusting the bowl bottom radius size, when the obstacle is too close, by reducing the bowl bottom radius size, the stretching of the obstacle image after being mapped to the 3D bowl-shaped model can be effectively alleviated, so that the obstacle image mapping effect is closer to the actual size and conforms to the visual perception size of the object.

[0069] Step S120, modeling the world coordinates according to the 3D bowl-shaped model, and mapping the 3D bowl-shaped model using the image captured by the camera according to the conversion relationship between the world coordinate system and the image coordinate system captured by the camera, to obtain a 3D view, and output it for display.

[0070] The 3D bowl-shaped model used in this step is the 3D bowl-shaped model processed in the previous step. The image taken by the camera is mapped to the processed 3D bowl-shaped model according to the conversion relationship between the coordinate systems to obtain a 3D view for output display.

[0071] In a possible implementation, after obtaining the 3D view, the UI image of the vehicle may be rendered into the 3D view according to the current position of the vehicle, and the processed 3D view may be obtained for output display.

[0072] Reference Figure 3 As shown, the car in the middle of the image is the UI image of the car. Figure 3 This is the effect of using the dynamic 3D bowl-shaped model of this application. When it is detected that the light truck in front of the vehicle is too close, the bowl bottom radius is adjusted from the default 7 meters to 5.6 meters. Figure 4 and Figure 1 It can be seen that after adopting the dynamic 3D bowl-shaped model of the present application, the size of the light truck in front of the vehicle is more in line with the actual size and has a stronger three-dimensional sense.

[0073] The 3D view generation method provided by the embodiment of the present application obtains the distance information of the obstacles around the vehicle and adaptively adjusts the size of the bowl bottom radius of the 3D bowl-shaped model according to the distance size. For example, when the obstacle distance is less than the set distance threshold, the size of the bowl bottom radius is adjusted to be smaller than the default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the obstacle distance is not less than the set distance threshold. When the present application detects that the obstacle distance is too close, the bowl bottom radius size of the 3D bowl-shaped model is reduced, thereby alleviating the stretching amount when the obstacle image is mapped to the 3D bowl-shaped model, so that the size of the obstacle after mapping is more in line with the actual situation, thereby improving the quality of the generated 3D view.

[0074] In some embodiments of the present application, an optional 3D bowl-shaped model is provided. The formula thereof can be expressed as:

[0075]

[0076] Among them, Z w Indicates the world coordinates of the 3D bowl edge, X w Indicates the X-axis coordinate value of a point in the world coordinate system, Y w It represents the Y-axis coordinate value of a position point in the world coordinate system, BowlRadius represents the radius of the bowl bottom, and BowlHeightScale represents the stretching coefficient of the height of the 3D bowl edge.

[0077] For example, when taking the world coordinate X w The front and rear distance of the vehicle is 10000mm, Y w When the vehicle is 10000mm around, BowlRadius is 7000mm, and BowlHeightScale is 1000, the world coordinate P world (X w ,Y w ,Z w ) The effect after 3D bowl-shaped modeling is as follows Figure 4 shown.

[0078] When adjusting the bowl bottom radius BowlRadius of the 3D bowl-shaped model and the stretch coefficient BowlHeightScale of the height of the 3D bowl edge, the flatness of the ground plane of the bowl bottom map of the 3D bowl-shaped model and the three-dimensional sense of the height of the bowl edge map can be adjusted.

[0079] In some embodiments of the present application, for the aforementioned step S110, the process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information, several possible implementation methods are introduced.

[0080] An optional adjustment method is as follows:

[0081] When the distance information is not less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is maintained as a default radius size.

[0082] When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a first size, where the first size is a fixed value smaller than a default radius size.

[0083] For example, taking the distance threshold as 1.5m, when the distance information of the obstacle is not less than 1.5m, the size of the bowl bottom radius of the 3D bowl-shaped model is maintained at 7m. When the distance information of the obstacle is less than 1.5m, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a fixed size, such as 5.6m.

[0084] Another optional adjustment is as follows:

[0085] When the distance information is not less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is maintained as a default radius size.

[0086] When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a second size, which is smaller than the default radius size and becomes smaller as the distance information becomes smaller.

[0087] Different from the previous implementation, when the obstacle distance is less than the set distance threshold, the second size of the bowl bottom radius is not a fixed value, but changes with the change of distance information, specifically, becomes smaller as the distance information becomes smaller.

[0088] The relationship between the second size and the obstacle distance information may be a piecewise function relationship, or a linear or nonlinear relationship.

[0089] The following is an example of a piecewise functional relationship between the second size R2 and the obstacle distance x:

[0090]

[0091] The distance threshold is set as x1', and when the obstacle distance x is greater than x1', the second size R2 takes a fixed value of L1. When the obstacle distance x is less than x1', it can be further divided into several segments, and when x belongs to different ranges, the second size R2 can also take different values.

[0092] The linear or nonlinear functional relationship between the second size R2 and the distance x is exemplified as follows:

[0093]

[0094] When the obstacle distance x is smaller than x1′, the second size R2 and the obstacle distance x satisfy the relationship of the function f(x), and the function f(x) may be a linear or nonlinear function.

[0095] This embodiment provides some possible corresponding relationships between the bowl bottom radius of the 3D bowl-shaped model and the obstacle distance. In addition, the optimal bowl bottom radius at different obstacle distances can be obtained through experiments, and then the relationship between the obstacle distance and the bowl bottom radius can be fitted to obtain the functional relationship between the obstacle distance and the bowl bottom radius.

[0096] In some embodiments of the present application, taking the case where more than one camera is configured on a vehicle, different cameras are used to capture images in different directions. For example, four wide-angle fisheye cameras are respectively arranged in the front, back, left and right of the vehicle to capture 360-degree images around the vehicle.

[0097] Furthermore, the distance measurement module can measure and obtain obstacle distance information in each direction.

[0098] On this basis, the 3D view generation method of the present application can be implemented according to the method of the following embodiment:

[0099] For each camera orientation:

[0100] Obtain the distance information of obstacles in the direction.

[0101] According to the distance information of the obstacle in the direction, the bottom radius of the 3D bowl-shaped model used to model the world coordinates in the direction is adjusted to obtain an adjusted 3D bowl-shaped model corresponding to the camera in the direction.

[0102] Taking the example of a vehicle equipped with cameras in four directions: front, back, left, and right, the bottom radius of the 3D bowl-shaped model in each direction can be adjusted based on the obstacle information in the four directions to obtain the adjusted 3D bowl-shaped model corresponding to the camera in each direction.

[0103] Furthermore, for each direction of the camera:

[0104] The world coordinates of the direction are modeled using the 3D bowl-shaped model corresponding to the camera in the direction, and the 3D bowl-shaped model is mapped using the image taken by the camera in the direction according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera in the direction.

[0105] Taking the front camera as an example, the world coordinates in front of the vehicle can be modeled using the adjusted 3D bowl-shaped model corresponding to the front camera, and according to the conversion relationship between the world coordinate system and the image coordinate system taken by the front camera, the adjusted 3D bowl-shaped model corresponding to the front camera can be mapped using the image taken by the front camera.

[0106] The same processing is performed on the cameras in other directions. After the images taken by the cameras in various directions are used to map the 3D bowl-shaped model, a 3D stitching view is obtained.

[0107] It is understandable that the distance information of obstacles in different directions may be different, and thus the sizes of the bowl bottom radius of the adjusted 3D bowl-shaped model corresponding to cameras in different directions may be the same or different.

[0108] According to the method of this embodiment, the bowl bottom radius of the 3D bowl-shaped model in the direction of obstacles in different directions of the vehicle can be dynamically adjusted according to the distance of the obstacle, so as to ensure that the effect of obstacle image mapping in this direction is more consistent with the actual size of the obstacle, thereby ensuring the overall quality of the final 3D stitched view.

[0109] The following describes a 3D view generating device provided in an embodiment of the present application. The 3D view generating device described below and the 3D view generating method described above may refer to each other.

[0110] See also Figure 5 , Figure 5 A schematic diagram of the structure of a 3D view generating device disclosed in an embodiment of the present application.

[0111] like Figure 5 As shown, the device may include:

[0112] The distance detection unit 11 is used to obtain the distance information of obstacles around the vehicle;

[0113] A bowl bottom radius adjustment unit 12, used to adjust the bowl bottom radius of the 3D bowl-shaped model according to the distance information, wherein when the distance information is less than a set distance threshold, the bowl bottom radius size of the 3D bowl-shaped model is less than a default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the distance information is not less than the set distance threshold;

[0114] The mapping unit 13 is used to model the world coordinates according to the 3D bowl-shaped model, and according to the conversion relationship between the world coordinate system and the image coordinate system captured by the camera, use the image captured by the camera to map the 3D bowl-shaped model, obtain a 3D view, and output it for display.

[0115] In a possible implementation, the set distance threshold value ranges from 0.3m to 1.5m.

[0116] In a possible implementation, the process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information by the bowl bottom radius adjustment unit includes:

[0117] For each camera orientation:

[0118] Obtaining distance information of obstacles in the direction;

[0119] According to the distance information of the obstacle in the direction, the bottom radius of the 3D bowl-shaped model used to model the world coordinates in the direction is adjusted to obtain an adjusted 3D bowl-shaped model corresponding to the camera in the direction;

[0120] The process of modeling the world coordinates according to the 3D bowl-shaped model, and mapping the 3D bowl-shaped model using the image captured by the camera according to the conversion relationship between the world coordinate system and the image coordinate system captured by the camera to obtain the 3D view includes:

[0121] For each camera orientation:

[0122] Modeling the world coordinates of the direction using the 3D bowl-shaped model corresponding to the camera in the direction, and mapping the 3D bowl-shaped model using the image taken by the camera in the direction according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera in the direction;

[0123] After the 3D bowl-shaped model is mapped using images taken by cameras in all directions, a 3D stitched view is obtained.

[0124] In a possible implementation, the process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information by the bowl bottom radius adjustment unit includes:

[0125] When the distance information is not less than the set distance threshold, maintaining the size of the bowl bottom radius of the 3D bowl-shaped model as the default radius size;

[0126] When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a first size, where the first size is a fixed value smaller than the default radius size.

[0127] In another possible implementation, the process of the bowl bottom radius adjustment unit adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes:

[0128] When the distance information is not less than the set distance threshold, maintaining the size of the bowl bottom radius of the 3D bowl-shaped model as the default radius size;

[0129] When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a second size, where the second size is smaller than the default radius size and becomes smaller as the distance information becomes smaller.

[0130] In a possible implementation, before outputting and displaying the 3D view, the mapping unit is also used to render the UI image of the vehicle into the 3D view according to the current position of the vehicle, and obtain the processed 3D view for output display.

[0131] In a possible implementation, the formula of the 3D bowl-shaped model is expressed as follows:

[0132]

[0133] Among them, Z w Indicates the world coordinates of the 3D bowl edge, X w Indicates the X-axis coordinate value of a point in the world coordinate system, Y w It represents the Y-axis coordinate value of a position point in the world coordinate system, BowlRadius represents the radius of the bowl bottom, and BowlHeightScale represents the stretching coefficient of the height of the 3D bowl edge.

[0134] The present application also provides an electronic device in an embodiment. Figure 6 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, tablet computers, vehicle-mounted terminals, servers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0135] like Figure 6 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the 3D view generation method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0136] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0137] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any 3D view generation method provided in the embodiment of the present application.

[0138] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any 3D view generation method provided in the embodiment of the present application.

[0139] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0140] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0141] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0142] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0143] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A 3D view generation method, characterized in that: include: Obtain distance information of obstacles around the vehicle; adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information, wherein when the distance information is less than a set distance threshold, the bowl bottom radius size of the 3D bowl-shaped model is less than a default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the distance information is not less than the set distance threshold; The world coordinates are modeled according to the 3D bowl-shaped model, and according to the conversion relationship between the world coordinate system and the image coordinate system captured by the camera, the 3D bowl-shaped model is mapped using the image captured by the camera to obtain a 3D view, which is then output for display.

2. The method according to claim 1, characterized in that The process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes: For each camera orientation: Obtaining distance information of obstacles in the direction; According to the distance information of the obstacle in the direction, the bottom radius of the 3D bowl-shaped model used to model the world coordinates in the direction is adjusted to obtain an adjusted 3D bowl-shaped model corresponding to the camera in the direction; The process of modeling the world coordinates according to the 3D bowl-shaped model, and mapping the 3D bowl-shaped model using the image captured by the camera according to the conversion relationship between the world coordinate system and the image coordinate system captured by the camera to obtain the 3D view includes: For each camera orientation: Modeling the world coordinates of the direction using the 3D bowl-shaped model corresponding to the camera in the direction, and mapping the 3D bowl-shaped model using the image taken by the camera in the direction according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera in the direction; After the 3D bowl-shaped model is mapped using images taken by cameras in all directions, a 3D stitched view is obtained.

3. The method according to claim 1, characterized in that The process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes: When the distance information is not less than the set distance threshold, maintaining the size of the bowl bottom radius of the 3D bowl-shaped model as the default radius size; When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a first size, where the first size is a fixed value smaller than the default radius size.

4. The method according to claim 1, characterized in that: The process of adjusting the bowl bottom radius of the 3D bowl-shaped model according to the distance information includes: When the distance information is not less than the set distance threshold, maintaining the size of the bowl bottom radius of the 3D bowl-shaped model as the default radius size; When the distance information is less than the set distance threshold, the size of the bowl bottom radius of the 3D bowl-shaped model is adjusted to a second size, where the second size is smaller than the default radius size and becomes smaller as the distance information becomes smaller.

5. The method according to claim 1, characterized in that Before outputting the 3D view, also include: According to the current position of the vehicle, the UI image of the vehicle is rendered into the 3D view, and the processed 3D view is obtained for output display.

6. The method according to claim 1, characterized in that The formula of the 3D bowl-shaped model is as follows: Among them, Z w Indicates the world coordinates of the 3D bowl edge, X w Indicates the X-axis coordinate value of a point in the world coordinate system, Y w It represents the Y-axis coordinate value of a position point in the world coordinate system, BowlRadius represents the radius of the bowl bottom, and BowlHeightScale represents the stretching coefficient of the height of the 3D bowl edge.

7. The method according to any one of claims 1 to 6, characterized in that: The distance threshold is set in the range of 0.3m-1.5m.

8. A 3D view generation device, characterized in that: include: A distance detection unit, used to obtain distance information of obstacles around the vehicle; a bowl bottom radius adjustment unit, configured to adjust the bowl bottom radius of the 3D bowl-shaped model according to the distance information, wherein when the distance information is less than a set distance threshold, the bowl bottom radius size of the 3D bowl-shaped model is less than a default radius size, and the default radius size is the size of the bowl bottom radius of the 3D bowl-shaped model when the distance information is not less than the set distance threshold; A mapping unit is used to model the world coordinates according to the 3D bowl-shaped model, and according to the conversion relationship between the world coordinate system and the image coordinate system taken by the camera, use the image taken by the camera to map the 3D bowl-shaped model, obtain a 3D view, and output it for display.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the 3D view generation method according to any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the 3D view generation method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the 3D view generation method according to any one of claims 1 to 7 is implemented.

Images