Vehicle all-round looking three-dimensional effect picture generation method, device, equipment and medium

Abstract

The embodiment of the invention provides a vehicle all-round looking three-dimensional effect picture generation method and device, equipment and a medium. The method comprises the steps of obtaining position difference information between adjacent cameras arranged on the periphery of a vehicle, determining a mapping relation between coordinate points on a stereoscopic projection model and image pixels of camera images according to the position difference information, generating a vehicle whole-body looking-around stereoscopic effect picture according to the mapping relation, the stereoscopic projection model and the camera images, and displaying the vehicle whole-body looking-around stereoscopic effect picture according to the vehicle whole-body looking-around stereoscopic effect picture. According to the position difference information, the mapping relation between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is determined, movement of the camera image projected to the stereoscopic projection model in three directions is achieved, finally, dislocation of the vehicle all-round looking stereoscopic effect picture in the horizontal direction and the height direction is optimized, the problem of splicing dislocation is solved, and the image quality is improved. The user experience is improved, and then the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for generating a vehicle surrounding stereoscopic effect image, a vehicle surrounding stereoscopic effect image generating device, an electronic device and a computer-readable storage medium. Background Art

[0002] The Around View Monitor (AVM) system uses four external fisheye cameras mounted on the front bumper, trunk, and left and right rearview mirrors of the vehicle as Figure 1 The data source shown is processed through steps such as image dedistortion, four-way image stitching, and 3D (three-dimensional) rendering to ultimately present a product with 360-degree environmental information around the vehicle.

[0003] Since the four fisheye cameras used in the panoramic view system output 2D (two-dimensional plane) images, the depth information (i.e. the distance between the object and the vehicle) is lost. When presenting a 3D view, the panoramic view system uses a preset depth (i.e. a bowl model, a bowl-shaped stereo projection model). When the actual depth of the object is inconsistent with the preset depth, it will cause stitching misalignment. Figure 2 As shown, the 3D object misalignment includes horizontal misalignment and vertical misalignment. From a top-down perspective, the AVM camera on the front bumper of the vehicle and the AVM camera on the right rearview mirror of the vehicle shoot an actual object, and then project the captured images onto the bowl mold (preset depth) respectively. The projection of the object captured by the AVM camera on the front bumper on the bowl mold is generally biased inward, and the projection of the object captured by the AVM camera on the right rearview mirror on the bowl mold is generally biased outward. From a side-view perspective, the AVM camera on the front bumper of the vehicle is generally low in height, and the AVM camera on the left or right rearview mirror of the vehicle is generally high in height. The AVM camera on the front bumper and the AVM camera on the left or right rearview mirror shoot an actual object, and then project the captured images onto the bowl mold (preset depth) respectively. The projection of the object captured by the AVM camera on the front bumper on the bowl mold is generally low, and the projection of the object captured by the AVM camera on the left or right rearview mirror on the bowl mold is generally high. Therefore, the 3D view of the panoramic surround view system has a splicing misalignment problem, which causes the 3D view of the panoramic surround view system to be seriously misaligned, bringing a very poor user experience and even affecting driving safety. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method for generating a vehicle surrounding view stereoscopic effect map that overcomes the above problems or at least partially solves the above problems, a corresponding vehicle surrounding view stereoscopic effect map generating device, an electronic device and a computer-readable storage medium.

[0005] In order to solve the above problems, an embodiment of the present invention discloses a method for generating a vehicle surrounding stereoscopic effect map, the method comprising:

[0006] Acquiring position difference information between adjacent cameras arranged around the vehicle;

[0007] Determining a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image according to the position difference information;

[0008] A stereoscopic rendering of the vehicle's surroundings is generated according to the mapping relationship, the stereoscopic projection model and the camera image.

[0009] Optionally, the position difference information includes height difference, position difference in the left-right direction of the vehicle, and position difference in the front-rear direction of the vehicle.

[0010] Optionally, before acquiring position difference information between adjacent cameras arranged around the vehicle, the method further comprises:

[0011] Determine position difference information between adjacent cameras based on external parameter information of multiple cameras;

[0012] Alternatively, the position difference information between adjacent cameras is determined based on the gyroscope information of multiple cameras.

[0013] Optionally, determining a mapping relationship between a coordinate point on a stereoscopic projection model and an image pixel of a camera image according to the position difference information includes:

[0014] According to the position difference information, the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model is corrected; wherein the target coordinate point is a coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras;

[0015] According to the internal parameter information and external parameter information of multiple cameras and the coordinate points on the stereoscopic projection model, the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is determined so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0016] Optionally, the correcting the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model according to the position difference information includes:

[0017] For a target coordinate point whose height is lower than a preset height, taking the product of the quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjusting the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point;

[0018] For a target coordinate point whose height is not lower than the preset height, the position difference information is used as a second adjustment amount, and the coordinate value of the target coordinate point is adjusted according to the second adjustment amount to obtain an adjusted target coordinate point.

[0019] Optionally, determining a mapping relationship between a coordinate point on a stereoscopic projection model and an image pixel of a camera image according to the position difference information includes:

[0020] Determine a mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image according to the intrinsic parameter information and the extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model;

[0021] According to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship is corrected to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0022] Optionally, the step of correcting the pixel position of the target image pixel corresponding to the target coordinate point whose height on the stereoscopic projection model is not zero in the mapping relationship according to the position difference information to obtain a corrected mapping relationship includes:

[0023] For a target coordinate point whose height is lower than a preset height, determining a third adjustment amount of the image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting the pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel;

[0024] For a target coordinate point whose height is not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and the pixel position of the target image pixel is adjusted according to the fourth adjustment amount to obtain an adjusted target image pixel.

[0025] The embodiment of the present invention further discloses a device for generating a vehicle surrounding stereoscopic effect image, the device comprising:

[0026] An information acquisition module, used to acquire position difference information between adjacent cameras arranged around the vehicle;

[0027] A relationship determination module, used to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image according to the position difference information;

[0028] The rendering generation module is used to generate a vehicle surrounding stereo rendering according to the mapping relationship, the stereo projection model and the camera image.

[0029] Optionally, the position difference information includes height difference, position difference in the left-right direction of the vehicle, and position difference in the front-rear direction of the vehicle.

[0030] Optionally, the device further comprises:

[0031] A first information determination module, configured to determine the position difference information between adjacent cameras according to external parameter information of a plurality of cameras before acquiring the position difference information between adjacent cameras arranged around the vehicle;

[0032] Alternatively, the second information determination module is used to determine the position difference information between adjacent cameras according to the gyroscope information of multiple cameras.

[0033] Optionally, the relationship determination module includes:

[0034] A coordinate correction submodule, used to correct the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model according to the position difference information; wherein the target coordinate point is a coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras;

[0035] The first mapping determination submodule is used to determine the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image based on the internal parameter information and external parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0036] Optionally, the coordinate correction submodule includes:

[0037] a first adjustment unit, configured to, for a target coordinate point whose height is lower than a preset height, use a product of a quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjust the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point;

[0038] The second adjustment unit is used to use the position difference information as a second adjustment amount for a target coordinate point whose height is not lower than the preset height, and adjust the coordinate value of the target coordinate point according to the second adjustment amount to obtain an adjusted target coordinate point.

[0039] Optionally, the relationship determination module includes:

[0040] A second mapping determination submodule, used to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on the intrinsic parameter information and extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model;

[0041] A pixel position correction submodule is used to correct the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship according to the position difference information, so as to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras.

[0042] Optionally, the pixel position correction submodule includes:

[0043] a third adjustment unit, for determining, for a target coordinate point whose height is lower than a preset height, a third adjustment amount of an image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting a pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel;

[0044] The fourth adjustment unit is used to determine a fourth adjustment amount of the image pixel according to the position difference information for a target coordinate point whose height is not lower than the preset height, and adjust the pixel position of the target image pixel according to the fourth adjustment amount to obtain an adjusted target image pixel.

[0045] The embodiment of the present invention further discloses an electronic device, including:

[0046] one or more processors; and

[0047] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the apparatus to perform any of the methods described above.

[0048] The embodiment of the present invention further discloses a computer-readable storage medium, wherein a computer program stored in the storage medium enables a processor to execute any of the methods described above.

[0049] According to an embodiment of the present invention, by acquiring position difference information between adjacent cameras arranged around a vehicle, a mapping relationship between coordinate points on a stereoscopic projection model and image pixels of a camera image is determined according to the position difference information, and a vehicle surrounding stereoscopic rendering is generated according to the mapping relationship, the stereoscopic projection model and the camera image, so that a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image is determined according to the position difference information, and movement of the camera image projected onto the stereoscopic projection model in three directions is achieved, and finally the horizontal and height misalignment of the vehicle surrounding stereoscopic rendering is optimized, thereby overcoming the problem of splicing misalignment, improving the user experience, and further improving the driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic diagram of the images from the four cameras on the vehicle;

[0051] Figure 2 It is a schematic diagram of the principle of splicing dislocation;

[0052] Figure 3 It is a flowchart of the steps of an embodiment of a method for generating a vehicle surrounding stereoscopic effect image of the present invention;

[0053] Figure 4 This is a schematic diagram of a rounded rectangular open bowl mold;

[0054] Figure 5 It is a schematic diagram of a circular or oval open bowl mold;

[0055] Figure 6 It is a schematic diagram of a circular or oval closed bowl mold;

[0056] Figure 7 It is a schematic diagram of the effect before and after the misalignment optimization;

[0057] Figure 8 It is a flowchart of the steps of an embodiment of a method for generating a vehicle surrounding stereoscopic effect image of the present invention;

[0058] Fig. 9 It is a structural block diagram of an embodiment of a vehicle surrounding stereoscopic effect image generating device of the present invention;

[0059] Fig.10 The present invention is a structural block diagram of an electronic device for generating a surround-view stereoscopic effect image of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] Reference Figure 1 , shows a flowchart of a method for generating a vehicle surrounding stereoscopic effect map according to an embodiment of the present invention, the method may specifically include the following steps:

[0062] Step 101, obtaining position difference information between adjacent cameras arranged around a vehicle.

[0063] In the embodiment of the present invention, the panoramic view system requires the use of multiple cameras installed around the vehicle. Due to different installation positions or camera displacement caused by vehicle collision, there are vertical and horizontal deviations between the multiple cameras.

[0064] In the embodiment of the present invention, information describing the position difference between adjacent cameras in three dimensions is recorded as position difference information. For example, there are four cameras in the front, back, left, and right directions of a vehicle, and position difference information between the front camera and the left camera, position difference information between the front camera and the right camera, position difference information between the rear camera and the left camera, and position difference information between the rear camera and the right camera are recorded.

[0065] In an optional embodiment of the present invention, the position difference information includes a height difference, a position difference in a left-right direction of the vehicle, and a position difference in a front-rear direction of the vehicle.

[0066] The height difference refers to the difference in height between the two cameras in the up and down directions of the vehicle. For example, usually the heights of the left camera and the right camera are the same. The height difference between the front camera and the left and right cameras is diffZ_F, and the height difference between the rear camera and the left and right cameras is diffZ_B.

[0067] Among them, the position difference in the left and right directions of the vehicle refers to the difference in the positions of the two cameras in the left and right directions of the vehicle, for example, the position difference diffX_RF between the front camera and the right camera in the left and right directions of the vehicle, the position difference diffX_LF between the front camera and the left camera in the left and right directions of the vehicle, the position difference diffX_RB between the rear camera and the right camera in the left and right directions of the vehicle, and the position difference diffX_LB between the rear camera and the left camera in the left and right directions of the vehicle.

[0068] Among them, the position difference in the front-to-back direction of the vehicle refers to the difference in the positions of the two cameras in the front-to-back direction of the vehicle, for example, the position difference diffY_RF between the front camera and the right camera in the front-to-back direction of the vehicle, the position difference diffY_LF between the front camera and the left camera in the front-to-back direction of the vehicle, the position difference diffY_RB between the rear camera and the right camera in the front-to-back direction of the vehicle, and the position difference diffY_LB between the rear camera and the left camera in the front-to-back direction of the vehicle.

[0069] In an optional embodiment of the present invention, before obtaining the position difference information between adjacent cameras arranged around the vehicle, it may also include: determining the position difference information between adjacent cameras based on external parameter information of multiple cameras; or determining the position difference information between adjacent cameras based on gyroscope information of multiple cameras.

[0070] The camera's external parameter information refers to the location information of the camera installation, including three position parameters (left and right, front and back, high and low) and three angle parameters (pitch angle - up and down deflection, yaw angle - left and right deflection, roll angle - self-rotation).

[0071] The intrinsic parameter information of a camera refers to the information that describes the degree of camera distortion.

[0072] The combination of external reference information and internal reference information can be used to realize the mapping of each pixel in the image to the real world. The mapping is divided into two parts: forward (real world to pixel) mapping and reverse (pixel to real world) mapping. In forward mapping: a point (X, Y, Z) in the real world can be accurately mapped to a pixel in the image based on the internal reference information and the external reference information. In reverse mapping: the direction relationship between an object and a vehicle in the image can be calculated based on the internal reference information and the external reference information, but the distance relationship cannot be calculated. For example, based on the internal reference information and the external reference information, it can be calculated whether a car in the image is on the left front or the right front side, but it cannot be calculated how far away the car is.

[0073] In one implementation, the position difference information between adjacent cameras can be calculated based on the external parameter information of multiple cameras. For example, the above diffZ_F, diffX_RF, and diffY_RF can be calculated based on the external parameter information of the front camera and the right camera; the above diffZ_F, diffX_LF, and diffY_LF can be calculated based on the external parameter information of the front camera and the left camera; the above diffZ_B, diffX_RB, and diffY_RB can be calculated based on the external parameter information of the rear camera and the right camera; and the above diffZ_B, diffX_LB, and diffY_LB can be calculated based on the external parameter information of the rear camera and the left camera.

[0074] Each camera can be equipped with a gyroscope, which can be a horizontal, vertical, pitch, heading and angular velocity sensor. The information collected by the gyroscopes of multiple cameras is obtained, that is, the gyroscope information.

[0075] In another implementation, the position and attitude of the camera may be determined according to the gyroscope information, and the position difference information between adjacent cameras may be calculated according to the positions and attitudes of adjacent cameras.

[0076] Specifically, any applicable method may be used to determine the position difference information between adjacent cameras, and the embodiment of the present invention does not limit this.

[0077] In the embodiment of the present invention, the position difference information only needs to be calculated once, or it can be calculated periodically. When it is needed, the previously determined position difference information can be directly obtained.

[0078] Step 102: Determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image according to the position difference information.

[0079] In the embodiments of the present invention, a stereoscopic projection model refers to a stereoscopic projection model for generating a three-dimensional stereoscopic image. For example, a bowl model is a model with a body center as the center, a rectangular bottom surface, and a certain curve around it, which is similar to a bowl shape. It is used to generate 3D views in a panoramic surround system. A common bowl model is a rounded rectangular open-type bowl model as follows: Figure 4 As shown, the round or oval open bowl mold is as follows Figure 5 As shown, the circular or oval closed bowl mold is as follows Figure 6 The bowl mold acts as a stereoscopic projection canvas, onto which the four images used by the panoramic surround view system are projected to generate a 3D image of the vehicle's surroundings.

[0080] In the embodiment of the present invention, a virtual stereoscopic projection model is defined by setting the coordinates of each point. For example, a virtual three-dimensional bowl model surrounding the vehicle is defined by setting the coordinates of each point.

[0081] In an embodiment of the present invention, the vehicle surrounding stereoscopic effect map finally generated by the panoramic surround view system is generated by projecting a stereoscopic projection model. The internal parameter information and external parameter information of multiple cameras of the panoramic surround view system are loaded and analyzed, and the coordinates of each point of the stereoscopic projection model are loaded. Then, by forward mapping, the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is determined. For example, the image pixels corresponding to each grid point (i.e., coordinate point) on the bowl mold on the four cameras are calculated. It is worth noting that if the camera cannot see the bowl mold grid points, such as the front camera cannot see the bowl mold grid points behind the vehicle, the bowl mold grid points corresponding to the image pixels are set to empty or infinite.

[0082] In an embodiment of the present invention, the stitching misalignment problem occurs when the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is determined as described above. There is an error in the mapping relationship between the image pixels of the camera images of adjacent cameras and the coordinate points on the stereoscopic projection model, and this error is caused by the position difference between adjacent cameras.

[0083] In an embodiment of the present invention, a mapping relationship between coordinate points on a stereoscopic projection model and image pixels of a camera image is determined based on the position difference information so that the camera images of adjacent cameras are not misaligned. There are multiple specific implementations for determining the mapping relationship based on the position difference information, which are not limited in the embodiment of the present invention.

[0084] In a specific implementation method, when calculating the mapping relationship between the coordinate points on the stereoscopic projection model and the coordinates of the image pixels, the coordinate points on the stereoscopic projection model are moved in three directions according to the position difference information, and then the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels is calculated to realize the movement of the camera image on the stereoscopic projection model in three directions of up, down, left, right, front and back, and finally optimize the horizontal and height misalignment of the vehicle's surrounding stereoscopic effect map.

[0085] In another specific implementation method, when calculating the mapping relationship between the coordinate points on the stereoscopic projection model and the coordinates of the image pixels, the image pixels of the camera image are moved in three directions according to the position difference information, and then the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels is calculated to realize the movement of the camera image in three directions of up, down, left, right, front and back on the stereoscopic projection model, and finally optimize the horizontal and height misalignment of the vehicle's surrounding stereoscopic effect map.

[0086] In the embodiment of the present invention, the internal reference information and external reference information, as well as the stereo projection model, are all necessary data for the panoramic surround view system, and no new data dependency is introduced. Only two additional processes are added to the vehicle surround view stereo effect map generation process of the existing panoramic surround view system, in which the position difference information only needs to be calculated once, and the movement of the coordinate points on the stereo projection model or the image pixels of the camera image only adds one step of addition operation, and is also only calculated once, which basically does not increase performance consumption.

[0087] Step 103 , generating a vehicle surrounding stereoscopic rendering according to the mapping relationship, the stereoscopic projection model and the camera image.

[0088] In the embodiment of the present invention, the camera images of multiple cameras are rendered onto the stereoscopic projection model according to the mapping relationship obtained by the above calculation. Then, the three-dimensional image rendered on the stereoscopic projection model is photographed according to the preset virtual camera angle, and finally a stereoscopic rendering of the vehicle's surroundings is generated.

[0089] In a specific implementation, the above-mentioned method of generating a vehicle's surrounding stereoscopic rendering can be accomplished using an open graphics library (OpenGL). OpenGL is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. It is a commonly used third-party dependency library for GPU image rendering and is used to render track lines, top views, 3D views, etc. in a panoramic surround view system.

[0090] For example, Figure 7 The schematic diagrams of the effects before and after the misalignment optimization are shown. The left figure is a stereoscopic effect diagram of the vehicle body when the solution of the present invention is not adopted. The curb and the vehicle in the frame can be seen to be obviously misaligned. The right figure is a stereoscopic effect diagram of the vehicle body after the solution of the present invention is adopted. The curb and the vehicle in the frame are well spliced.

[0091] According to an embodiment of the present invention, by acquiring position difference information between adjacent cameras arranged around a vehicle, a mapping relationship between coordinate points on a stereoscopic projection model and image pixels of a camera image is determined according to the position difference information, and a vehicle surrounding stereoscopic rendering is generated according to the mapping relationship, the stereoscopic projection model and the camera image, so that a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image is determined according to the position difference information, and movement of the camera image projected onto the stereoscopic projection model in three directions is achieved, and finally the horizontal and height misalignment of the vehicle surrounding stereoscopic rendering is optimized, thereby overcoming the problem of splicing misalignment, improving the user experience, and further improving the driving safety.

[0092] Reference Figure 8 , shows a flowchart of a method for generating a vehicle surrounding stereoscopic effect map according to an embodiment of the present invention, the method may specifically include the following steps:

[0093] Step 201, obtaining position difference information between adjacent cameras arranged around the vehicle.

[0094] In the embodiment of the present invention, the specific implementation method of this step can refer to the description in the above embodiment and will not be repeated here.

[0095] Step 202, based on the position difference information, correct the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model; wherein the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras.

[0096] In the embodiment of the present invention, when not corrected, the mapping relationship between the coordinate point on the stereoscopic projection model and the image pixel of the camera image is that the coordinate point (X, Y, Z) is mapped to the image pixel (x, y).

[0097] In the embodiment of the present invention, different processing is performed on coordinate points at different heights on the stereoscopic projection model.

[0098] In an embodiment of the present invention, no correction is made to the coordinate points with a height of zero, that is, the coordinate points on the ground. At this time, based on the intrinsic parameter information and extrinsic parameter information of multiple cameras, the determined mapping relationship is still the mapping of the coordinate point (X, Y, Z) to the image pixel (x, y).

[0099] In an embodiment of the present invention, the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model needs to be corrected, wherein the target coordinate point is the coordinate point corresponding to the camera images of the front camera and the rear camera, or the coordinate point corresponding to the camera images of the left camera and the right camera, or the coordinate point corresponding to the camera images of the front camera and the rear camera, and the coordinate point corresponding to the camera images of the left camera and the right camera.

[0100] When the target coordinate point is the coordinate point corresponding to the camera images of the front camera and the rear camera, the coordinate value of the target coordinate point is corrected according to the position difference information so that the camera images of the front camera and the rear camera are moved and the camera images of the left camera and the right camera are aligned so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0101] When the target coordinate point is the coordinate point corresponding to the camera images of the left camera and the right camera, the coordinate value of the target coordinate point is corrected according to the position difference information so that the camera images of the left camera and the right camera are moved and the camera images of the front camera and the rear camera are aligned so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0102] When the target coordinate point is the coordinate point corresponding to the camera images of the front camera and the rear camera, and the coordinate point corresponding to the camera images of the left camera and the right camera, the coordinate value of the target coordinate point is corrected according to the position difference information, so that the camera images of the front camera and the rear camera, and the camera images of the left camera and the right camera are moved, and the camera images of the front camera and the rear camera and the camera images of the left camera and the right camera are aligned with each other, so that the camera images of the front and rear cameras and the camera images of the left camera and the right camera are not misaligned.

[0103] In the embodiment of the present invention, X, Y, and Z in the target coordinate point (X, Y, and Z) are corrected according to the position difference information, thereby moving the target coordinate point.

[0104] In an optional embodiment of the present invention, an implementation method of correcting the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model according to the position difference information may include: for a target coordinate point whose height is lower than a preset height, taking the product of the quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjusting the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point; for a target coordinate point whose height is not lower than the preset height, taking the position difference information as a second adjustment amount, and adjusting the coordinate value of the target coordinate point according to the second adjustment amount to obtain an adjusted target coordinate point.

[0105] The preset height is set according to actual needs and can be determined through multiple experiments. The embodiment of the present invention does not limit this.

[0106] For a target coordinate point whose height is not zero and is lower than a preset height, first calculate the quotient scale of the height of the target coordinate point and the preset height as a coefficient. Then calculate the product of the quotient scale and the position difference information as the first adjustment amount. For example, if the position difference information is the height difference a, the position difference b in the left-right direction of the vehicle, and the position difference c in the front-back direction of the vehicle, then the corresponding first adjustment amounts are scale×a, scale×b, and scale×c.

[0107] Then, according to the first adjustment amount, the coordinate value of the target coordinate point whose height is not zero and is lower than the preset height is adjusted to obtain an adjusted target coordinate point.

[0108] For target coordinate points whose height is not zero and not lower than the preset height, the position difference information is used as the second adjustment amount. For example, if the position difference information is the height difference a, the position difference b in the left-right direction of the vehicle, and the position difference c in the front-back direction of the vehicle, the corresponding second adjustment amounts are a, b, and c.

[0109] Then, according to the second adjustment amount, the coordinate value of the target coordinate point whose height is not zero and is not lower than the preset height is adjusted to obtain an adjusted target coordinate point.

[0110] For example, if the height of the front camera is 40cm, and the heights of the left and right cameras are 120cm, then the height difference between the front camera and the left and right cameras is 80cm, and the coordinate value of the bowl model grid point (i.e., the target coordinate point) is (X, Y, Z). For grid points whose height is lower than the transition value (preset height 150cm), add scale×80cm to the height of the coordinate value of the bowl model grid point to obtain (X, Y, Z+scale×80cm), and then map the image pixel coordinates (x1, y1) based on the internal and external reference information. At this time, the mapping relationship is that the target coordinate point (X, Y, Z+scale×80cm) is mapped to the image pixel (x1, y1). For grid points whose height is not lower than the transition value, add 80cm to the height of the coordinate value of the bowl model grid point, and then map the image pixel coordinates (x2, y2) based on the internal and external reference information. At this time, the mapping relationship is that the target coordinate point (X, Y, Z+80cm) is mapped to the image pixel (x2, y2). For horizontal misalignment optimization, according to the horizontal position difference, the coordinate values ​​X and Y of the bowl mold grid points are added or subtracted with deviation values. The other processes are the same as the above processes and will not be described in detail.

[0111] The reason for setting the preset height and scale is that the coordinate points with a height of zero are not processed. If the same adjustment amount is added to all the target coordinate points with a height that is not zero, it will cause a sudden change at the intersection of the bottom and the top of the stereoscopic projection model, so it is necessary to transition within a certain range. The preset height is the transition area range. When the height of the target coordinate point of the stereoscopic projection model gradually increases from 0 to the preset height, the scale gradually increases from 0 to 1. After this treatment, the target coordinate point with a height of 1 only adds the first adjustment amount of 1 / preset height × height difference, which is basically unchanged. The higher the height, the greater the change value. After the height is higher than the preset height, only the height difference is changed uniformly, and the final effect is better and the transition is softer. Here, the height difference is taken as an example, and the correction in the horizontal direction is the same.

[0112] Step 203, based on the internal parameter information and external parameter information of multiple cameras and the coordinate points on the stereoscopic projection model, determine the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0113] In an embodiment of the present invention, a specific method of determining the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is performed based on the intrinsic parameter information and extrinsic parameter information of multiple cameras and the coordinate points on the stereoscopic projection model. The above-mentioned forward projection method is adopted, and no other additional processing is required except for adjusting the target coordinate points among the coordinate points on the stereoscopic projection model.

[0114] Step 204 , generating a vehicle surrounding stereoscopic rendering according to the mapping relationship, the stereoscopic projection model and the camera image.

[0115] In the embodiment of the present invention, the specific implementation method of this step can refer to the description in the above embodiment and will not be repeated here.

[0116] According to an embodiment of the present invention, by acquiring position difference information between adjacent cameras arranged around a vehicle, the coordinate value of a target coordinate point whose height is not zero on the stereoscopic projection model is corrected according to the position difference information; wherein the target coordinate point is a coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras, and according to the internal parameter information and external parameter information of multiple cameras, and the coordinate point on the stereoscopic projection model, a mapping relationship between the coordinate point on the stereoscopic projection model and the image pixels of the camera image is determined, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned, and a vehicle body surround stereoscopic effect map is generated according to the mapping relationship, the stereoscopic projection model and the camera image, so that according to the position difference information, the mapping relationship between the coordinate point on the stereoscopic projection model and the image pixels of the camera image is determined, and the movement of the camera image projected onto the stereoscopic projection model in three directions is realized, and finally the horizontal and height misalignment of the vehicle body surround stereoscopic effect map is optimized, thereby overcoming the splicing misalignment problem, improving the user experience, and then improving the driving safety.

[0117] In an optional embodiment of the present invention, a specific implementation method of determining the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image based on the position difference information may include: determining the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image based on the intrinsic parameter information and extrinsic parameter information of multiple cameras and the coordinate points on the stereoscopic projection model; correcting the pixel positions of the target image pixels corresponding to the target coordinate points on the stereoscopic projection model with a non-zero height in the mapping relationship based on the position difference information to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate points are the coordinate points corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0118] According to the intrinsic parameter information and extrinsic parameter information of multiple cameras and the coordinate points on the stereoscopic projection model, a specific method of determining the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is adopted, using the above-mentioned forward projection method.

[0119] Then, according to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point in the mapping relationship is corrected to obtain a corrected mapping relationship. Similar to the above-mentioned correction of the coordinate value of the target coordinate point, according to the intrinsic parameter information and extrinsic parameter information of the camera, the position difference information can be converted into pixel position difference information of the pixel position of the image pixel. According to the pixel position difference information, the pixel position of the target image pixel is corrected to obtain a corrected mapping relationship.

[0120] In an optional embodiment of the present invention, according to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point whose height on the stereoscopic projection model is not zero in the mapping relationship is corrected to obtain a corrected mapping relationship. A specific implementation method may include: for a target coordinate point whose height is lower than a preset height, determining a third adjustment amount of the image pixel according to the product of the quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting the pixel position of the target image pixel according to the third adjustment amount to obtain the adjusted target image pixel; for a target coordinate point whose height is not lower than the preset height, determining a fourth adjustment amount of the image pixel according to the position difference information, and adjusting the pixel position of the target image pixel according to the fourth adjustment amount to obtain the adjusted target image pixel.

[0121] For the target coordinate point whose height is not zero and is lower than the preset height, first calculate the quotient scale of the height of the target coordinate point and the preset height as a coefficient. Then calculate the product of the quotient scale and the position difference information. According to the product, determine the third adjustment amount of the image pixel. For example, the position difference information is the height difference a, the position difference b in the left and right direction of the vehicle, and the position difference c in the front and rear direction of the vehicle, then the corresponding products are scale×a, scale×b, scale×c, and then according to the projection relationship between the stereoscopic projection model and the camera image, the product is converted into the third adjustment amount.

[0122] Then, according to the third adjustment amount, the pixel position of the target image pixel corresponding to the target coordinate point whose height is not zero and is lower than the preset height is adjusted to obtain the adjusted target image pixel.

[0123] For a target coordinate point whose height is not zero and is not lower than a preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information. For example, the position difference information is a height difference a, a position difference b in the left-right direction of the vehicle, and a position difference c in the front-back direction of the vehicle. According to the projection relationship between the stereoscopic projection model and the camera image, the position difference information is converted into the fourth adjustment amount.

[0124] Then, according to the fourth adjustment amount, the pixel position of the target image pixel corresponding to the target coordinate point whose height is not zero and not less than the preset height is adjusted to obtain the adjusted target image pixel.

[0125] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0126] Reference Fig. 9 The structure block diagram of an embodiment of a vehicle surrounding stereoscopic effect image generation device of the present invention is shown, which may specifically include the following modules:

[0127] An information acquisition module 301 is used to acquire position difference information between adjacent cameras arranged around the vehicle;

[0128] A relationship determination module 302, used to determine a mapping relationship between a coordinate point on a stereoscopic projection model and an image pixel of a camera image according to the position difference information;

[0129] The rendering generation module 303 is used to generate a vehicle surrounding stereo rendering according to the mapping relationship, the stereo projection model and the camera image.

[0130] Optionally, the position difference information includes height difference, position difference in the left-right direction of the vehicle, and position difference in the front-rear direction of the vehicle.

[0131] Optionally, the device further comprises:

[0132] A first information determination module, configured to determine the position difference information between adjacent cameras according to external parameter information of a plurality of cameras before acquiring the position difference information between adjacent cameras arranged around the vehicle;

[0133] Alternatively, the second information determination module is used to determine the position difference information between adjacent cameras according to the gyroscope information of multiple cameras.

[0134] Optionally, the relationship determination module includes:

[0135] A coordinate correction submodule, used to correct the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model according to the position difference information; wherein the target coordinate point is a coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras;

[0136] The first mapping determination submodule is used to determine the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image based on the internal parameter information and external parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0137] Optionally, the coordinate correction submodule includes:

[0138] a first adjustment unit, configured to, for a target coordinate point whose height is lower than a preset height, use a product of a quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjust the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point;

[0139] The second adjustment unit is used to use the position difference information as a second adjustment amount for a target coordinate point whose height is not lower than the preset height, and adjust the coordinate value of the target coordinate point according to the second adjustment amount to obtain an adjusted target coordinate point.

[0140] Optionally, the relationship determination module includes:

[0141] A second mapping determination submodule, used to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on the intrinsic parameter information and extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model;

[0142] A pixel position correction submodule is used to correct the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship according to the position difference information, so as to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras.

[0143] Optionally, the pixel position correction submodule includes:

[0144] a third adjustment unit, for determining, for a target coordinate point whose height is lower than a preset height, a third adjustment amount of an image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting a pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel;

[0145] The fourth adjustment unit is used to determine a fourth adjustment amount of the image pixel according to the position difference information for a target coordinate point whose height is not lower than the preset height, and adjust the pixel position of the target image pixel according to the fourth adjustment amount to obtain an adjusted target image pixel.

[0146] According to an embodiment of the present invention, by acquiring position difference information between adjacent cameras arranged around a vehicle, a mapping relationship between coordinate points on a stereoscopic projection model and image pixels of a camera image is determined according to the position difference information, and a vehicle surrounding stereoscopic rendering is generated according to the mapping relationship, the stereoscopic projection model and the camera image, so that a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image is determined according to the position difference information, and movement of the camera image projected onto the stereoscopic projection model in three directions is achieved, and finally the horizontal and height misalignment of the vehicle surrounding stereoscopic rendering is optimized, thereby overcoming the problem of splicing misalignment, improving the user experience, and further improving the driving safety.

[0147] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0148] Fig. 9 1 is a block diagram of an electronic device 600 for generating a vehicle surround stereoscopic effect map according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0149] Reference Fig. 9 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0150] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of generating the above-mentioned vehicle body surround stereoscopic effect map. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0151] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0152] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0153] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0154] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0155] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0156] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of contact between the user and the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0157] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned shutdown control method.

[0159] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to complete the above-mentioned shutdown control method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables a server to execute a method for generating a vehicle surrounding stereoscopic effect map, the method comprising:

[0161] Acquiring position difference information between adjacent cameras arranged around the vehicle;

[0162] Determining a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image according to the position difference information;

[0163] A stereoscopic rendering of the vehicle's surroundings is generated according to the mapping relationship, the stereoscopic projection model and the camera image.

[0164] Optionally, the position difference information includes height difference, position difference in the left-right direction of the vehicle, and position difference in the front-rear direction of the vehicle.

[0165] Optionally, before acquiring position difference information between adjacent cameras arranged around the vehicle, the method further comprises:

[0166] Determine position difference information between adjacent cameras based on external parameter information of multiple cameras;

[0167] Alternatively, the position difference information between adjacent cameras is determined based on the gyroscope information of multiple cameras.

[0168] Optionally, determining a mapping relationship between a coordinate point on a stereoscopic projection model and an image pixel of a camera image according to the position difference information includes:

[0169] According to the position difference information, the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model is corrected; wherein the target coordinate point is a coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras;

[0170] According to the internal parameter information and external parameter information of multiple cameras and the coordinate points on the stereoscopic projection model, the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is determined so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0171] Optionally, the correcting the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model according to the position difference information includes:

[0172] For a target coordinate point whose height is lower than a preset height, taking the product of the quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjusting the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point;

[0173] For a target coordinate point whose height is not lower than the preset height, the position difference information is used as a second adjustment amount, and the coordinate value of the target coordinate point is adjusted according to the second adjustment amount to obtain an adjusted target coordinate point.

[0174] Optionally, determining a mapping relationship between a coordinate point on a stereoscopic projection model and an image pixel of a camera image according to the position difference information includes:

[0175] Determine a mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image according to the intrinsic parameter information and the extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model;

[0176] According to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship is corrected to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0177] Optionally, the step of correcting the pixel position of the target image pixel corresponding to the target coordinate point whose height on the stereoscopic projection model is not zero in the mapping relationship according to the position difference information to obtain a corrected mapping relationship includes:

[0178] For a target coordinate point whose height is lower than a preset height, determining a third adjustment amount of the image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting the pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel;

[0179] For a target coordinate point whose height is not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and the pixel position of the target image pixel is adjusted according to the fourth adjustment amount to obtain an adjusted target image pixel.

[0180] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0181] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0182] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0183] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0185] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0186] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0187] The above is a detailed introduction to the vehicle body surround view stereoscopic effect diagram generation method and vehicle body surround view stereoscopic effect diagram generation device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for generating a vehicle surrounding stereoscopic effect image, characterized in that: The method comprises: Acquiring position difference information between adjacent cameras arranged around the vehicle; Determining a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image according to the position difference information; According to the mapping relationship 、 The stereo projection model and camera images generate a stereoscopic rendering of the vehicle's surroundings.

2. The method according to claim 1, characterized in that The position difference information includes a height difference, a position difference in a left-right direction of the vehicle, and a position difference in a front-rear direction of the vehicle.

3. The method according to claim 1, characterized in that Before acquiring the position difference information between adjacent cameras arranged around the vehicle, the method further includes: Determine position difference information between adjacent cameras based on external parameter information of multiple cameras; Alternatively, the position difference information between adjacent cameras is determined based on the gyroscope information of multiple cameras.

4. The method according to claim 1, characterized in that: Determining the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image according to the position difference information includes: According to the position difference information, the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model is corrected; wherein the target coordinate point is a coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras; Based on the internal parameter information and external parameter information of multiple cameras and the coordinate points on the stereoscopic projection model, a mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image is determined so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

5. The method according to claim 4, characterized in that The step of correcting the coordinate value of the target coordinate point whose height is not zero on the stereoscopic projection model according to the position difference information includes: For a target coordinate point whose height is lower than a preset height, taking the product of the quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjusting the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point; For a target coordinate point whose height is not lower than the preset height, the position difference information is used as a second adjustment amount, and the coordinate value of the target coordinate point is adjusted according to the second adjustment amount to obtain an adjusted target coordinate point.

6. The method according to claim 1, characterized in that Determining the mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image according to the position difference information includes: Determine a mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera image according to the intrinsic parameter information and the extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model; According to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship is corrected to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

7. The method according to claim 6, characterized in that The step of correcting the pixel position of the target image pixel corresponding to the target coordinate point whose height on the stereoscopic projection model is not zero in the mapping relationship according to the position difference information to obtain a corrected mapping relationship includes: For a target coordinate point whose height is lower than a preset height, determining a third adjustment amount of the image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting the pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel; For a target coordinate point whose height is not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and the pixel position of the target image pixel is adjusted according to the fourth adjustment amount to obtain an adjusted target image pixel.

8. A vehicle surrounding stereoscopic effect image generation device, characterized in that: The method comprises: An information acquisition module, used to acquire position difference information between adjacent cameras arranged around the vehicle; A relationship determination module, used to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image according to the position difference information; The rendering generation module is used to generate a vehicle surrounding stereo rendering according to the mapping relationship, the stereo projection model and the camera image.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the apparatus to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the method according to any one of claims 1 to 7.

Images