Vehicle all-around look-around stereoscopic effect map generation method, device, equipment and medium

Abstract

Embodiments of the present application provide a vehicle all-around look-around stereoscopic effect map generation method, device, equipment and medium. The method comprises: acquiring position difference information between adjacent cameras arranged around the vehicle; determining a mapping relationship between a coordinate point on a stereoscopic projection model and an image pixel of a camera image according to the position difference information; and generating a vehicle all-around look-around stereoscopic effect map according to the mapping relationship, the stereoscopic projection model and the camera image. According to the position difference information, the mapping relationship between the coordinate point on the stereoscopic projection model and the image pixel of the camera image is determined, movement in three directions of the camera image projected onto the stereoscopic projection model is realized, and finally the misalignment of the vehicle all-around look-around stereoscopic effect map in the horizontal and height directions is optimized, the splicing misalignment problem is overcome, the user experience is improved, and the driving safety is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a vehicle all-around view stereoscopic effect map generation method, a vehicle all-around view stereoscopic effect map generation device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] The panoramic view system (AVM, Around View Monitor) is a product that uses four external fisheye cameras mounted on the front bumper, trunk, and left and right rearview mirrors of a vehicle as data sources, performs image distortion removal, image splicing, 3D (three-dimensional) rendering, and other steps on the four images, and finally presents 360-degree environmental information around the vehicle. Figure 1

[0003] Since the four fisheye cameras used by the panoramic view system output 2D (two-dimensional) images, the depth information (i.e., the distance between the object and the vehicle) is lost, and the panoramic view system adopts a preset depth (i.e., a bowl model, a bowl-shaped stereographic projection model) when presenting a 3D view. When the actual depth of an object does not match the preset depth, it will cause misalignment in splicing. As shown in Figure 2 As shown in the figure, 3D object misalignment includes horizontal misalignment and vertical misalignment. From the overhead perspective, the AVM camera on the front bumper of the vehicle and the AVM camera on the right side of the vehicle's rearview mirror capture an object that actually exists, and then project the captured images onto the bowl model (preset depth). The projection of the object captured by the AVM camera on the front bumper of the vehicle on the bowl model is generally inward, and the projection of the object captured by the AVM camera on the right side of the vehicle's rearview mirror on the bowl model is generally outward. From the side view perspective, the AVM camera on the front bumper of the vehicle is generally low in height, and the AVM camera on the left or right side of the vehicle's rearview mirror is generally high in height. The AVM camera on the front bumper and the AVM camera on the left or right side of the vehicle's rearview mirror capture an object that actually exists, and then project the captured images onto the bowl model (preset depth). The projection of the object captured by the AVM camera on the front bumper on the bowl model is generally low, and the projection of the object captured by the AVM camera on the left or right side of the vehicle's rearview mirror on the bowl model is generally high. Therefore, the 3D view of the panoramic view system has a misalignment problem in splicing, which causes the 3D view of the panoramic view system to be severely misaligned, resulting in a very poor user experience and even affecting driving safety. SUMMARY

[0004] In view of the above problems, the embodiments of the present application are proposed to provide a vehicle all-around view stereoscopic effect map generation method, a corresponding vehicle all-around view stereoscopic effect map generation device, an electronic device and a computer readable storage medium that overcome the above problems or at least partially solve the above problems. ​

[0005] To solve the above problems, the embodiment of the present application discloses a vehicle surround view stereoscopic effect map generation method, the method comprises:

[0006] Obtaining position difference information between adjacent cameras arranged around the vehicle;

[0007] According to the position difference information, determining the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera images;

[0008] According to the mapping relationship, the stereographic model and the camera images, generating a vehicle surround view stereoscopic effect map.

[0009] Optionally, the position difference information includes height difference, position difference in the left-right direction of the vehicle, and position difference in the front-rear direction of the vehicle.

[0010] Optionally, before the position difference information between adjacent cameras arranged around the vehicle is obtained, the method further comprises:

[0011] According to the extrinsic information of the plurality of cameras, determining the position difference information between adjacent cameras;

[0012] Or, according to the gyroscope information of the plurality of cameras, determining the position difference information between adjacent cameras.

[0013] Optionally, the determining the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera images according to the position difference information comprises:

[0014] According to the position difference information, correcting the coordinate values of target coordinate points on the stereographic model with a height not equal to zero; wherein the target coordinate points are coordinate points corresponding to the camera images of the front-rear cameras and / or the camera images of the left-right cameras;

[0015] According to the intrinsic information and extrinsic information of the plurality of cameras, and the coordinate points on the stereographic model, determining the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera images, so that the camera images of the front-rear cameras and the camera images of the left-right cameras are not misaligned.

[0016] Optionally, the correcting the coordinate values of target coordinate points on the stereographic model with a height not equal to zero according to the position difference information comprises:

[0017] For target coordinate points with a height lower than a preset height, multiplying the quotient of the height of the target coordinate point and the preset height and the position difference information to obtain a first adjustment amount, and adjusting the coordinate values of the target coordinate points according to the first adjustment amount to obtain adjusted target coordinate points;

[0018] For the target coordinate point with a height not lower than the preset height, the position difference information is taken as a second adjustment amount, and a coordinate value of the target coordinate point is adjusted according to the second adjustment amount, so that an adjusted target coordinate point is obtained.

[0019] Optionally, the determining of the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image according to the position difference information comprises:

[0020] determining the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image according to the internal parameter information and the external parameter information of the plurality of cameras and the coordinate point on the stereographic projection model;

[0021] According to the position difference information, the pixel position of a target image pixel corresponding to a target coordinate point with a height of zero on the stereographic projection model in the mapping relationship is corrected, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is a coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0022] Optionally, the correcting of the pixel position of the target image pixel corresponding to the target coordinate point with the height of zero on the stereographic projection model in the mapping relationship according to the position difference information comprises:

[0023] For the target coordinate point with a height lower than the preset height, a third adjustment amount of the image pixel is determined according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and the pixel position of the target image pixel is adjusted according to the third adjustment amount, so that an adjusted target image pixel is obtained.

[0024] For the target coordinate point with a height not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and the pixel position of the target image pixel is adjusted according to the fourth adjustment amount, so that an adjusted target image pixel is obtained.

[0025] The embodiment of the application further discloses a vehicle all-around surround stereoscopic effect diagram generation device, the device comprises:

[0026] an information acquisition module configured to acquire position difference information between adjacent cameras arranged around a vehicle;

[0027] a relationship determination module configured to determine a mapping relationship between a coordinate point on a stereographic projection model and an image pixel of a camera image according to the position difference information;

[0028] An effect map generation module is configured to generate a surround view stereoscopic effect map of the vehicle according to the mapping relationship, the stereographic model and the camera images.

[0029] Optionally, the position difference information includes a height difference, a position difference in a left-right direction of the vehicle, and a position difference in a front-rear direction of the vehicle.

[0030] Optionally, the apparatus further includes:

[0031] A first information determination module is configured to determine position difference information between adjacent cameras arranged around the vehicle according to extrinsic information of the plurality of cameras before the position difference information is acquired.

[0032] Alternatively, a second information determination module is configured to determine the position difference information between the adjacent cameras according to gyroscope information of the plurality of cameras.

[0033] Optionally, the relationship determination module includes:

[0034] A coordinate correction submodule is configured to correct coordinate values of target coordinate points with a non-zero height on the stereographic model according to the position difference information, wherein the target coordinate points are coordinate points corresponding to camera images of front-rear cameras and / or camera images of left-right cameras.

[0035] A first mapping determination submodule is configured to determine a mapping relationship between coordinate points on the stereographic model and image pixels of the camera images according to intrinsic information and extrinsic information of the plurality of cameras and the coordinate points on the stereographic model, so that camera images of the front-rear cameras and camera images of the left-right cameras are not misaligned.

[0036] Optionally, the coordinate correction submodule includes:

[0037] A first adjustment unit is configured to, for a target coordinate point with a height lower than a preset height, take a product of a quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjust coordinate values of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point.

[0038] A second adjustment unit is configured to, for a target coordinate point with a height not lower than the preset height, take the position difference information as a second adjustment amount, and adjust coordinate values of the target coordinate point according to the second adjustment amount to obtain an adjusted target coordinate point.

[0039] Optionally, the relationship determination module includes:

[0040] a second mapping determination submodule, configured to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on intrinsic parameter information and extrinsic parameter information of the plurality of cameras and the coordinate points on the stereoscopic projection model;

[0041] A pixel position correction submodule is used to correct the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship based on the position difference information, to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras.

[0042] Optionally, the pixel position correction submodule includes:

[0043] a third adjustment unit, configured to determine, for a target coordinate point whose height is lower than a preset height, a third adjustment amount of an image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjust a pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel;

[0044] The fourth adjustment unit is used to determine a fourth adjustment amount of the image pixel according to the position difference information for a target coordinate point whose height is not lower than the preset height, and adjust the pixel position of the target image pixel according to the fourth adjustment amount to obtain an adjusted target image pixel.

[0045] An embodiment of the present invention further discloses an electronic device, including:

[0046] one or more processors; and

[0047] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the apparatus to perform any of the methods described above.

[0048] An embodiment of the present invention further discloses a computer-readable storage medium, wherein a computer program stored in the computer-readable storage medium enables a processor to execute any one of the methods described above.

[0049] According to the embodiment of the present application, by acquiring position difference information between adjacent cameras arranged around the vehicle, determining the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera image according to the position difference information, and generating the vehicle all-around look-around stereographic effect drawing according to the mapping relationship, the stereographic model and the camera image, the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera image is determined according to the position difference information, the movement of the camera image projected on the stereographic model in three directions is realized, and finally the misalignment of the vehicle all-around look-around stereographic effect drawing in the horizontal and height directions is optimized, the splicing misalignment problem is overcome, the user experience is improved, and the driving safety is improved. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a schematic diagram of images of four cameras on a vehicle;

[0051] Figure 2 is a schematic diagram of the principle of splicing misalignment;

[0052] Figure 3 is a step flowchart of a vehicle all-around look-around stereographic effect drawing generation method embodiment of the present application;

[0053] Figure 4 is a schematic diagram of a bowl mold of a rounded rectangle open type;

[0054] Figure 5 is a schematic diagram of a bowl mold of a rounded or elliptical open type;

[0055] Figure 6 is a schematic diagram of a bowl mold of a rounded or elliptical closed type;

[0056] Figure 7 is a schematic diagram of the effect before and after misalignment optimization;

[0057] Figure 8 is a step flowchart of a vehicle all-around look-around stereographic effect drawing generation method embodiment of the present application;

[0058] Figure 9 is a structural block diagram of a vehicle all-around look-around stereographic effect drawing generation device embodiment of the present application;

[0059] Figure 10 is a structural block diagram of an electronic device for vehicle all-around look-around stereographic effect drawing generation according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0061] Refer toFigure 1 FIG. 1 shows a flow chart of a method for generating a stereoscopic image of a vehicle surround view according to an embodiment of the present application, which can include the following steps:

[0062] In step 101, position difference information between adjacent cameras arranged around the vehicle is obtained.

[0063] In the embodiment of the present application, the panoramic surround view system needs to use multiple cameras arranged around the vehicle. Due to different installation positions, or camera displacement caused by vehicle collision, there is a deviation in the vertical and horizontal directions between the multiple cameras.

[0064] In the embodiment of the present application, the information describing the position difference between adjacent cameras in three dimensions is referred to as position difference information. For example, there are four cameras in the front, back, left and right directions of the vehicle. The position difference information between the front camera and the left camera, the position difference information between the front camera and the right camera, the position difference information between the back camera and the left camera, and the position difference information between the back camera and the right camera.

[0065] In an optional embodiment of the present application, the position difference information includes a height difference, a position difference in the left-right direction of the vehicle, and a position difference in the front-back direction of the vehicle.

[0066] The height difference refers to the difference in height of the two cameras in the up-down direction of the vehicle. For example, the height of the left camera and the right camera is usually consistent, the height difference diffZ_F between the front camera and the left and right cameras, and the height difference diffZ_B between the back camera and the left and right cameras.

[0067] The position difference in the left-right direction of the vehicle refers to the difference in position of the two cameras in the left-right direction of the vehicle. For example, the position difference diffX_RF of the front camera and the right camera in the left-right direction of the vehicle, the position difference diffX_LF of the front camera and the left camera in the left-right direction of the vehicle, the position difference diffX_RB of the back camera and the right camera in the left-right direction of the vehicle, and the position difference diffX_LB of the back camera and the left camera in the left-right direction of the vehicle.

[0068] The position difference in the front-back direction of the vehicle refers to the difference in position of the two cameras in the front-back direction of the vehicle. For example, the position difference diffY_RF of the front camera and the right camera in the front-back direction of the vehicle, the position difference diffY_LF of the front camera and the left camera in the front-back direction of the vehicle, the position difference diffY_RB of the back camera and the right camera in the front-back direction of the vehicle, and the position difference diffY_LB of the back camera and the left camera in the front-back direction of the vehicle.

[0069] In an optional embodiment of the present application, before the position difference information between the adjacent cameras arranged around the vehicle is acquired, the method further comprises: determining the position difference information between the adjacent cameras according to the extrinsic information of the plurality of cameras; or determining the position difference information between the adjacent cameras according to the gyroscope information of the plurality of cameras.

[0070] The extrinsic information of the camera refers to the position information of the camera installation, including three position parameters (left and right, front and back, high and low) and three angle parameters (pitch angle - up and down deflection, yaw angle - left and right deflection, roll angle - self rotation).

[0071] The intrinsic information of the camera refers to the information describing the degree of distortion of the camera.

[0072] The extrinsic information and the intrinsic information can be combined to realize the mapping of each pixel in the image to the real world, which is divided into two parts: forward (real world to pixel) mapping and reverse (pixel to real world) mapping. In forward mapping: according to the intrinsic information and the extrinsic information, a point (X, Y, Z) in the real world can be accurately mapped to a certain pixel in the image. In reverse mapping: according to the intrinsic information and the extrinsic information, the directional relationship of an object in the image and the vehicle can be calculated, but the distance relationship cannot be calculated. For example, according to the intrinsic information and the extrinsic information, a car in the image can be calculated to be on the left front or the right front side, but the distance of the car cannot be calculated.

[0073] In an implementation, according to the extrinsic information of the plurality of cameras, the position difference information between the adjacent cameras can be calculated. For example, according to the extrinsic information of the front camera and the right camera, the diffZ_F, diffX_RF, and diffY_RF described above can be calculated; according to the extrinsic information of the front camera and the left camera, the diffZ_F, diffX_LF, and diffY_LF described above can be calculated; according to the extrinsic information of the rear camera and the right camera, the diffZ_B, diffX_RB, and diffY_RB described above can be calculated; and according to the extrinsic information of the rear camera and the left camera, the diffZ_B, diffX_LB, and diffY_LB described above can be calculated.

[0074] A gyroscope can be installed on each camera, and the gyroscope can have horizontal, vertical, pitch, heading, and angular velocity sensors. The information collected by the gyroscope of the plurality of cameras, i.e., the gyroscope information, is acquired.

[0075] In another implementation, the position and attitude of the camera can be determined according to the gyroscope information, and the position difference information between the adjacent cameras can be calculated according to the position and attitude of the adjacent cameras.

[0076] The position difference information between adjacent cameras can be determined in any applicable manner, and embodiments of the present application do not limit this.

[0077] In embodiments of the present application, the position difference information is calculated only once or periodically. When needed, the previously determined position difference information can be directly obtained.

[0078] In step 102, a mapping relationship between coordinate points on the stereographic model and image pixels of camera images is determined according to the position difference information.

[0079] In embodiments of the present application, the stereographic model refers to a stereographic model for generating a three-dimensional stereoscopic image. For example, a bowl model is a model with a center of the vehicle body as the center, a rectangular bottom surface, and a certain curved upper periphery similar to a bowl shape, which is used for generating a 3D view in a surround view system. A common bowl model is an open bowl model with a rectangular shape and rounded corners as shown in the following figure. Figure 4 An open bowl model with a circular or elliptical shape is shown in the following figure. Figure 5 An open bowl model with a circular or elliptical shape is shown in the following figure. Figure 6 The bowl model functions as a stereographic cloth, and four images used by the surround view system are projected onto the bowl model to generate a 3D image around the vehicle body.

[0080] In embodiments of the present application, a virtual stereographic model is defined by setting the coordinates of each point. For example, a virtual three-dimensional bowl model surrounding the vehicle is defined by setting the coordinates of each point.

[0081] In embodiments of the present application, a vehicle surround view stereoscopic effect image generated by the surround view system is generated by projection using a stereographic model. The intrinsic and extrinsic parameters of the multiple cameras of the surround view system are loaded and analyzed, and the coordinates of each point of the stereographic model are loaded. Then, a forward mapping is performed to determine the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera images, for example, to calculate the corresponding image pixels of each grid point (i.e., coordinate point) on the bowl model in the four cameras. It should be noted that if the camera cannot see the bowl model grid point, such as the front camera that cannot see the bowl model grid point behind the vehicle, the image pixel corresponding to the bowl model grid point is set to empty or infinity.

[0082] In embodiments of the present application, the misalignment problem occurs when determining the mapping relationship between the coordinate points on the stereographic model and the image pixels of the camera images. The mapping relationship between the image pixels of the camera images of adjacent cameras and the coordinate points on the stereographic model is incorrect, and this error is caused by the position difference between adjacent cameras.

[0083] In the embodiment of the present application, according to the position difference information, a mapping relationship between a coordinate point on the stereographic projection model and an image pixel of the camera image is determined, so that the camera images of adjacent cameras are not misaligned. The specific implementation of determining the mapping relationship according to the position difference information can include various manners, and the embodiment of the present application does not limit this.

[0084] In a specific implementation, when calculating the mapping relationship between the coordinate point on the stereographic projection model and the coordinate of the image pixel, according to the position difference information, the coordinate point on the stereographic projection model is moved in three directions, and then the mapping relationship between the coordinate point on the stereographic projection model and the image pixel is calculated, so as to realize the movement of the camera image on the stereographic projection model in three directions of up, down, left, right, front and back, and finally optimize the misalignment in the horizontal and height directions of the vehicle all-around view stereographic effect diagram.

[0085] In another specific implementation, when calculating the mapping relationship between the coordinate point on the stereographic projection model and the coordinate of the image pixel, according to the position difference information, the image pixel of the camera image is moved in three directions, and then the mapping relationship between the coordinate point on the stereographic projection model and the image pixel is calculated, so as to realize the movement of the camera image on the stereographic projection model in three directions of up, down, left, right, front and back, and finally optimize the misalignment in the horizontal and height directions of the vehicle all-around view stereographic effect diagram.

[0086] In the embodiment of the present application, the intrinsic information and the extrinsic information used, and the stereographic projection model are all data necessary for the panoramic view system, and no new data dependence is introduced. Only two additional processes are added in the vehicle all-around view stereographic effect diagram generation process of the existing panoramic view system, the position difference information is calculated only once, the movement of the coordinate point on the stereographic projection model or the image pixel of the camera image only adds one addition operation, and is also calculated only once, and the performance consumption is basically not increased.

[0087] In step 103, a vehicle all-around view stereographic effect diagram is generated according to the mapping relationship, the stereographic projection model and the camera image.

[0088] In the embodiment of the present application, the camera images of the plurality of cameras are rendered onto the stereographic projection model according to the mapping relationship calculated above. Then, a three-dimensional stereographic image rendered on the stereographic projection model is photographed according to a preset virtual camera angle, and finally a vehicle all-around view stereographic effect diagram is generated.

[0089] In one specific implementation, the above-mentioned method of generating a surround-view 3D rendering of the vehicle can be accomplished using the Open Graphics Library (OpenGL). OpenGL is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. It is a commonly used third-party library for GPU image rendering and is used in surround-view systems to render track lines, top-down views, 3D views, and more.

[0090] For example, Figure 7 The schematic diagrams of the effects before and after the misalignment optimization are shown. The left figure is a three-dimensional effect diagram of the vehicle body when the solution of the present invention is not adopted. The curb and the vehicle in the frame can be seen to be obviously misaligned. The right figure is a three-dimensional effect diagram of the vehicle body after the solution of the present invention is adopted. The curb and the vehicle in the frame are well spliced.

[0091] According to an embodiment of the present invention, by obtaining position difference information between adjacent cameras arranged around a vehicle, a mapping relationship between coordinate points on a stereoscopic projection model and image pixels of a camera image is determined based on the position difference information, and a surround-view stereoscopic effect diagram of the vehicle is generated based on the mapping relationship, the stereoscopic projection model, and the camera image. In this way, a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image is determined based on the position difference information, and movement of the camera image projected onto the stereoscopic projection model in three directions is achieved. Ultimately, the horizontal and height misalignment of the surround-view stereoscopic effect diagram of the vehicle is optimized, thereby overcoming the splicing misalignment problem, improving the user experience, and thereby improving driving safety.

[0092] Reference Figure 8 , shows a flowchart of an embodiment of a method for generating a vehicle surrounding stereoscopic effect image according to the present invention, which may specifically include the following steps:

[0093] Step 201: Acquire position difference information between adjacent cameras arranged around the vehicle.

[0094] In the embodiment of the present invention, the specific implementation method of this step can be found in the description of the above embodiment and will not be repeated here.

[0095] Step 202 : Correcting the coordinate values ​​of target coordinate points whose heights are not zero on the stereoscopic projection model based on the position difference information; wherein the target coordinate points are coordinate points corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0096] In the embodiment of the present application, the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image is that the coordinate point (X, Y, Z) is mapped to the image pixel (x, y) without correction.

[0097] In the embodiment of the present application, different coordinate points with different heights on the stereographic projection model are processed differently.

[0098] In the embodiment of the present application, the coordinate point with zero height, i.e. the coordinate point on the ground, is not corrected, and the mapping relationship determined according to the internal parameter information and the external parameter information of the plurality of cameras is still that the coordinate point (X, Y, Z) is mapped to the image pixel (x, y).

[0099] In the embodiment of the present application, the coordinate value of the target coordinate point with non-zero height on the stereographic projection model needs to be corrected, wherein the target coordinate point is the coordinate point corresponding to the camera images of the front camera and the rear camera, or the coordinate point corresponding to the camera images of the left camera and the right camera, or the coordinate point corresponding to the camera images of the front camera and the rear camera, and the coordinate point corresponding to the camera images of the left camera and the right camera.

[0100] When the target coordinate point is the coordinate point corresponding to the camera images of the front camera and the rear camera, the coordinate value of the target coordinate point is corrected according to the position difference information, so that the camera images of the front camera and the rear camera are moved to align with the camera images of the left camera and the right camera, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0101] When the target coordinate point is the coordinate point corresponding to the camera images of the left camera and the right camera, the coordinate value of the target coordinate point is corrected according to the position difference information, so that the camera images of the left camera and the right camera are moved to align with the camera images of the front camera and the rear camera, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0102] When the target coordinate point is the coordinate point corresponding to the camera images of the front camera and the rear camera, and the coordinate point corresponding to the camera images of the left camera and the right camera, the coordinate value of the target coordinate point is corrected according to the position difference information, so that the camera images of the front camera and the rear camera, and the camera images of the left camera and the right camera are all moved, and the camera images of the front camera and the rear camera and the camera images of the left camera and the right camera are aligned with each other, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0103] In the embodiment of the present application, X, Y and Z in the target coordinate point (X, Y, Z) are corrected according to the position difference information, so as to move the target coordinate point.

[0104] In an alternative embodiment of the present application, the implementation of correcting the coordinate values of the target coordinate points with non-zero height on the stereoscopic projection model according to the position difference information can include: for the target coordinate points with height lower than the preset height, multiplying the quotient of the height of the target coordinate point and the preset height and the position difference information to obtain a first adjustment amount, and adjusting the coordinate values of the target coordinate points according to the first adjustment amount to obtain the adjusted target coordinate points; for the target coordinate points with height not lower than the preset height, taking the position difference information as a second adjustment amount, and adjusting the coordinate values of the target coordinate points according to the second adjustment amount to obtain the adjusted target coordinate points.

[0105] The preset height is set according to actual needs, and can be determined through multiple experiments, which is not limited in the embodiments of the present application.

[0106] For the target coordinate points with non-zero height and lower than the preset height, the quotient of the height of the target coordinate point and the preset height is calculated as a coefficient, and the product of the quotient and the position difference information is calculated as a first adjustment amount. For example, the position difference information includes height difference a, position difference b in the left-right direction of the vehicle, and position difference c in the front-back direction of the vehicle, and the corresponding first adjustment amount is scale×a, scale×b, and scale×c.

[0107] Then, the coordinate values of the target coordinate points with non-zero height and lower than the preset height are adjusted according to the first adjustment amount to obtain the adjusted target coordinate points.

[0108] For the target coordinate points with non-zero height and not lower than the preset height, the position difference information is taken as a second adjustment amount. For example, the position difference information includes height difference a, position difference b in the left-right direction of the vehicle, and position difference c in the front-back direction of the vehicle, and the corresponding second adjustment amount is a, b, and c.

[0109] Then, the coordinate values of the target coordinate points with non-zero height and not lower than the preset height are adjusted according to the second adjustment amount to obtain the adjusted target coordinate points.

[0110] For example, the height of the front camera is 40 cm, the height of the left and right cameras is 120 cm, the height difference between the front camera and the left and right cameras is 80 cm, and the coordinate value of the bowl grid point (i.e. the target coordinate point) is (X, Y, Z). For the grid point with a height lower than the transition value (preset height 150 cm), the height of the coordinate value of the bowl grid point is added by scale x 80 cm to obtain (X, Y, Z + scale x 80 cm), and then the coordinate (x1, y1) of the image pixel is mapped according to the intrinsic information and the extrinsic information, at this time the mapping relationship is that the target coordinate point (X, Y, Z + scale x 80 cm) is mapped to the image pixel (x1, y1). For the grid point with a height not lower than the transition value, the height of the coordinate value of the bowl grid point is added by 80 cm, and then the coordinate (x2, y2) of the image pixel is mapped according to the intrinsic information and the extrinsic information, at this time the mapping relationship is that the target coordinate point (X, Y, Z + 80 cm) is mapped to the image pixel (x2, y2). The horizontal direction offset optimization adds or subtracts the deviation value to the coordinate values X and Y of the bowl grid point according to the position difference in the horizontal direction, and other processes are consistent with the above process and will not be described in detail.

[0111] The reason for setting the preset height and the scale is that the coordinate point with a height of zero is not processed, and if the target coordinate point with a height not equal to zero is added by the same adjustment amount, the junction between the bottom and the upper part of the stereoscopic projection model will be abrupt, so a transition within a certain range is needed. The preset height is the range of the transition area, and when the height of the target coordinate point of the stereoscopic projection model gradually changes from 0 to the preset height, the scale gradually changes from 0 to 1. After such processing, the target coordinate point with a height of 1 is only added by the first adjustment amount of 1 / preset height x height difference, and basically no change, the higher the height, the greater the change value, and after the height is higher than the preset height, only the height difference is changed, the final effect is better, and the transition is softer. Here, the height difference is taken as an example, and the correction in the horizontal direction is the same.

[0112] In step 203, the mapping relationship between the coordinate point on the stereoscopic projection model and the image pixel of the camera image is determined according to the intrinsic information and the extrinsic information of the plurality of cameras and the coordinate point on the stereoscopic projection model, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0113] In the embodiment of the present application, the specific way of determining the mapping relationship between the coordinate point on the stereoscopic projection model and the image pixel of the camera image according to the intrinsic information and the extrinsic information of the plurality of cameras and the coordinate point on the stereoscopic projection model adopts the above-mentioned forward projection manner, and no other additional processing is needed except for adjusting the target coordinate point in the coordinate point on the stereoscopic projection model.

[0114] In step 204, the vehicle all-around look stereoscopic effect diagram is generated according to the mapping relationship, the stereoscopic projection model and the camera image.

[0115] In the embodiment of the present application, the specific implementation of this step can refer to the description in the foregoing embodiments, which will not be described here again.

[0116] According to the embodiment of the present application, by acquiring the position difference information between the adjacent cameras arranged around the vehicle, the coordinate value of the target coordinate point with a non-zero height on the stereographic model is corrected according to the position difference information; wherein the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras, the mapping relationship between the coordinate point on the stereographic model and the image pixel of the camera image is determined according to the internal and external parameter information of the multiple cameras and the coordinate point on the stereographic model, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned, the vehicle surround view stereographic effect drawing is generated according to the mapping relationship, the stereographic model and the camera image, so that according to the position difference information, the mapping relationship between the coordinate point on the stereographic model and the image pixel of the camera image is determined, the movement in three directions of the camera image projected onto the stereographic model is realized, and finally the misalignment of the vehicle surround view stereographic effect drawing in the horizontal and height directions is optimized, the splicing misalignment problem is overcome, the user experience is improved, and the driving safety is improved.

[0117] In an optional embodiment of the present application, in a specific implementation of determining the mapping relationship between the coordinate point on the stereographic model and the image pixel of the camera image according to the position difference information, it can include: determining the mapping relationship between the coordinate point on the stereographic model and the image pixel of the camera image according to the internal and external parameter information of the multiple cameras and the coordinate point on the stereographic model; correcting the pixel position of the target image pixel corresponding to the target coordinate point on the stereographic model in the mapping relationship according to the position difference information, to obtain the corrected mapping relationship, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras.

[0118] The specific way of determining the mapping relationship between the coordinate point on the stereographic model and the image pixel of the camera image according to the internal and external parameter information of the multiple cameras and the coordinate point on the stereographic model adopts the above-mentioned forward projection manner.

[0119] Then, according to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point in the mapping relationship is corrected, so as to obtain the corrected mapping relationship. Similarly to the correction of the coordinate value of the target coordinate point, according to the internal parameter information and the external parameter information of the camera, the position difference information can be converted into pixel position difference information of the pixel position of the image pixel. According to the pixel position difference information, the pixel position of the target image pixel is corrected, and the corrected mapping relationship is obtained.

[0120] In an optional embodiment of the present application, according to the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point with a height not being zero on the stereographic projection model in the mapping relationship is corrected, and a specific implementation manner of the corrected mapping relationship can include: for the target coordinate point with a height lower than a preset height, a third adjustment amount of the image pixel is determined according to the product of the quotient of the height of the target coordinate point and the preset height and the position difference information, and the pixel position of the target image pixel is adjusted according to the third adjustment amount, so as to obtain the adjusted target image pixel; for the target coordinate point with a height not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and the pixel position of the target image pixel is adjusted according to the fourth adjustment amount, so as to obtain the adjusted target image pixel.

[0121] For the target coordinate point with a height not being zero and lower than a preset height, the quotient of the height of the target coordinate point and the preset height is calculated as a coefficient. Then, the product of the quotient and the position difference information is calculated. According to the product, the third adjustment amount of the image pixel is determined. For example, the position difference information is a height difference a, a position difference b in the left-right direction of the vehicle, and a position difference c in the front-back direction of the vehicle, and the corresponding product is scale×a, scale×b, and scale×c. Then, according to the projection relationship between the stereographic projection model and the camera image, the product is converted into the third adjustment amount.

[0122] Then, according to the third adjustment amount, the pixel position of the target image pixel corresponding to the target coordinate point with a height not being zero and lower than a preset height is adjusted, so as to obtain the adjusted target image pixel.

[0123] For the target coordinate point with a height not being zero and not lower than a preset height, the fourth adjustment amount of the image pixel is determined according to the position difference information. For example, the position difference information is a height difference a, a position difference b in the left-right direction of the vehicle, and a position difference c in the front-back direction of the vehicle, and according to the projection relationship between the stereographic projection model and the camera image, the position difference information is converted into the fourth adjustment amount.

[0124] Then, according to the fourth adjustment amount, the pixel position of the target image pixel corresponding to the target coordinate point with a height not being zero and not lower than the preset height is adjusted to obtain an adjusted target image pixel.

[0125] It should be noted that, for the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the action sequence described, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0126] With reference to Figure 9 A structural block diagram of an embodiment of a vehicle surround view stereoscopic effect map generation device of the present application is shown, which can specifically include the following modules:

[0127] An information acquisition module 301 is configured to acquire position difference information between adjacent cameras arranged around a vehicle;

[0128] A relationship determination module 302 is configured to determine a mapping relationship between a coordinate point on a stereographic projection model and an image pixel of a camera image according to the position difference information;

[0129] An effect map generation module 303 is configured to generate a vehicle surround view stereoscopic effect map according to the mapping relationship, the stereographic projection model and the camera image.

[0130] Optionally, the position difference information includes a height difference, a position difference in a left-right direction of the vehicle, and a position difference in a front-rear direction of the vehicle.

[0131] Optionally, the device further includes:

[0132] A first information determination module is configured to determine the position difference information between adjacent cameras according to extrinsic information of multiple cameras before the position difference information between adjacent cameras arranged around the vehicle is acquired;

[0133] Or, a second information determination module is configured to determine the position difference information between adjacent cameras according to gyroscope information of multiple cameras.

[0134] Optionally, the relationship determination module includes:

[0135] A coordinate correction sub-module is configured to correct a coordinate value of a target coordinate point with a height not being zero on the stereographic projection model according to the position difference information; wherein the target coordinate point is a coordinate point corresponding to a camera image of a front-rear camera and / or a camera image of a left-right camera;

[0136] The first mapping determination submodule is configured to determine a mapping relationship between the coordinate points on the stereographic projection model and the image pixels of the camera images according to the internal parameter information and the external parameter information of the plurality of cameras and the coordinate points on the stereographic projection model, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

[0137] Optionally, the coordinate correction submodule comprises:

[0138] The first adjustment unit is configured to, for a target coordinate point with a height lower than a preset height, take a product of a quotient of the height of the target coordinate point and the preset height and the position difference information as a first adjustment amount, and adjust a coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point.

[0139] The second adjustment unit is configured to, for a target coordinate point with a height not lower than the preset height, take the position difference information as a second adjustment amount, and adjust a coordinate value of the target coordinate point according to the second adjustment amount to obtain an adjusted target coordinate point.

[0140] Optionally, the relationship determination module comprises:

[0141] The second mapping determination submodule is configured to determine a mapping relationship between the coordinate points on the stereographic projection model and the image pixels of the camera images according to the internal parameter information and the external parameter information of the plurality of cameras and the coordinate points on the stereographic projection model.

[0142] The pixel position correction submodule is configured to correct a pixel position of a target image pixel corresponding to a target coordinate point with a height not equal to zero on the stereographic projection model in the mapping relationship according to the position difference information to obtain a corrected mapping relationship, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is a coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0143] Optionally, the pixel position correction submodule comprises:

[0144] The third adjustment unit is configured to, for a target coordinate point with a height lower than a preset height, determine a third adjustment amount of an image pixel according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjust a pixel position of the target image pixel according to the third adjustment amount to obtain an adjusted target image pixel.

[0145] The fourth adjusting unit is configured to determine a fourth adjusting amount of the image pixel according to the position difference information for a target coordinate point with a height not lower than the preset height, and adjust the pixel position of the target image pixel according to the fourth adjusting amount to obtain an adjusted target image pixel.

[0146] According to the embodiment of the present application, the position difference information between adjacent cameras arranged around the vehicle is acquired, the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image is determined according to the position difference information, and the vehicle all-around look-around stereographic effect drawing is generated according to the mapping relationship, the stereographic projection model and the camera image, so that the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image is determined according to the position difference information, the movement of the camera image projected on the stereographic projection model in three directions is realized, and the misalignment of the vehicle all-around look-around stereographic effect drawing in the horizontal and vertical directions is finally optimized, the splicing misalignment problem is overcome, the user experience is improved, and the driving safety is improved.

[0147] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts are referred to the part of the method embodiment.

[0148] Figure 9 FIG. 6 is a structural block diagram of an electronic device 600 for vehicle all-around look-around stereographic effect drawing generation according to an exemplary embodiment. The electronic device 600 can be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0149] Referring to Figure 9 The electronic device 600 can include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0150] The processing component 602 usually controls overall operations of the electronic device 600, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 602 can include one or more processors 620 to execute instructions to complete all or part of the steps of the vehicle all-around look-around stereographic effect drawing generation described above. Further, the processing component 602 can include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 can include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0151] The memory 604 is configured to store various types of data to support the operation of the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disc, or optical disc.

[0152] The power supply component 606 supplies power for various components of the electronic device 600. The power supply component 606 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 600.

[0153] The multimedia component 608 includes a screen providing an output interface between the electronic device 600 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. The front and rear cameras can receive external multimedia data when the electronic device 600 is in an operation mode, such as a photographing mode or a video mode. Each of the front and rear cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0154] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) configured to receive external audio signals when the electronic device 600 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0155] The I / O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which can be a keyboard, a click wheel, a button, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0156] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of components, such as the display and keypad of the electronic device 600. The sensor assembly 614 can also detect changes in the position of the electronic device 600 or a component of the electronic device 600, the presence or absence of user contact with the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and temperature changes of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0157] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned shutdown control method.

[0159] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 604 including instructions. The instructions can be executed by the processor 620 of the electronic device 600 to implement the above-described shutdown control method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables a server to perform a method for generating a surround view stereoscopic effect map of a vehicle, the method comprising:

[0161] obtaining position difference information between adjacent cameras arranged around the vehicle;

[0162] determining, according to the position difference information, a mapping relationship between coordinate points on a stereographic projection model and image pixels of camera images;

[0163] generating a surround view stereoscopic effect map of the vehicle according to the mapping relationship, the stereographic projection model, and the camera images.

[0164] Optionally, the position difference information comprises a height difference, a position difference in a left-right direction of the vehicle, and a position difference in a front-rear direction of the vehicle.

[0165] Optionally, before the obtaining of the position difference information between adjacent cameras arranged around the vehicle, the method further comprises:

[0166] determining, according to extrinsic information of the plurality of cameras, the position difference information between the adjacent cameras;

[0167] or, determining, according to gyroscope information of the plurality of cameras, the position difference information between the adjacent cameras.

[0168] Optionally, the determining, according to the position difference information, of the mapping relationship between coordinate points on a stereographic projection model and image pixels of camera images comprises:

[0169] correcting, according to the position difference information, coordinate values of target coordinate points on the stereographic projection model having a non-zero height; wherein the target coordinate points are coordinate points corresponding to camera images of front-rear cameras and / or camera images of left-right cameras;

[0170] determining, according to intrinsic information and extrinsic information of the plurality of cameras and the coordinate points on the stereographic projection model, the mapping relationship between the coordinate points on the stereographic projection model and the image pixels of the camera images, so that the camera images of the front-rear cameras and the camera images of the left-right cameras are not misaligned.

[0171] Optionally, the correcting, according to the position difference information, of the coordinate values of the target coordinate points on the stereographic projection model having a non-zero height comprises:

[0172] for a target coordinate point having a height lower than a preset height, multiplying a quotient of the height of the target coordinate point and the preset height by the position difference information to obtain a first adjustment amount, and adjusting the coordinate values of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point.

[0173] For the target coordinate point with a height not lower than the preset height, the position difference information is taken as a second adjustment amount, and a coordinate value of the target coordinate point is adjusted according to the second adjustment amount, to obtain an adjusted target coordinate point.

[0174] Optionally, the determining of the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image according to the position difference information comprises:

[0175] determining the mapping relationship between the coordinate point on the stereographic projection model and the image pixel of the camera image according to the internal parameter information and the external parameter information of the plurality of cameras and the coordinate point on the stereographic projection model;

[0176] correcting a pixel position of a target image pixel corresponding to a target coordinate point with a height not being zero on the stereographic projection model in the mapping relationship according to the position difference information, to obtain a corrected mapping relationship, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is a coordinate point corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras.

[0177] Optionally, the correcting of the pixel position of the target image pixel corresponding to the target coordinate point with the height not being zero on the stereographic projection model in the mapping relationship according to the position difference information, to obtain the corrected mapping relationship, comprises:

[0178] For the target coordinate point with a height lower than the preset height, a third adjustment amount of the image pixel is determined according to a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and a pixel position of the target image pixel is adjusted according to the third adjustment amount, to obtain an adjusted target image pixel.

[0179] For the target coordinate point with a height not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and a pixel position of the target image pixel is adjusted according to the fourth adjustment amount, to obtain an adjusted target image pixel.

[0180] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0181] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer program instructions.

[0182] Embodiments of the present application are described herein with reference to the Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing the function specified by the block or blocks.

[0183] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing the function specified by the block or blocks.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable devices provide steps for implementing the functions specified in the flow or multiple flows and / or blocks. Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing the function specified by the block or blocks.

[0185] Although preferred embodiments of the present application have been described, those skilled in the art will recognize that additional modifications and variations can be made thereto without departing from the scope of the present application. It is therefore intended that the appended claims cover all such modifications and variations as fall within the scope of the present application.

[0186] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0187] The vehicle all-around look-around stereoscopic effect map generation method and the vehicle all-around look-around stereoscopic effect map generation device provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for generating a vehicle surrounding stereoscopic effect image, characterized in that: The method comprises: Acquiring position difference information between adjacent cameras arranged around the vehicle; determining a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on the position difference information; Generate a surround-view stereoscopic rendering of the vehicle according to the mapping relationship, the stereo projection model and the camera image; Determining a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on the position difference information includes: Correcting the coordinate values ​​of target coordinate points on the stereoscopic projection model whose heights are not zero based on the position difference information; wherein the target coordinate points are coordinate points corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras; Based on the intrinsic parameter information and extrinsic parameter information of multiple cameras and the coordinate points on the stereoscopic projection model, a mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera images is determined so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

2. The method according to claim 1, characterized in that The position difference information includes height difference, position difference in the left-right direction of the vehicle, and position difference in the front-rear direction of the vehicle.

3. The method according to claim 1, characterized in that Before acquiring position difference information between adjacent cameras disposed around the vehicle, the method further includes: Determine position difference information between adjacent cameras based on external parameter information of multiple cameras; Alternatively, position difference information between adjacent cameras is determined based on gyroscope information of multiple cameras.

4. The method according to claim 1, wherein The step of correcting the coordinate value of the target coordinate point having a non-zero height on the stereoscopic projection model according to the position difference information includes: For a target coordinate point whose height is lower than a preset height, multiplying the quotient of the height of the target coordinate point and the preset height by the position difference information as a first adjustment amount, and adjusting the coordinate value of the target coordinate point according to the first adjustment amount to obtain an adjusted target coordinate point; For a target coordinate point whose height is not lower than the preset height, the position difference information is used as a second adjustment amount, and the coordinate value of the target coordinate point is adjusted according to the second adjustment amount to obtain an adjusted target coordinate point.

5. The method according to claim 1, wherein Determining a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on the position difference information includes: Determining a mapping relationship between the coordinate points on the stereoscopic projection model and the image pixels of the camera images based on the intrinsic parameter information and the extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model; Based on the position difference information, the pixel position of the target image pixel corresponding to the target coordinate point with a non-zero height on the stereoscopic projection model in the mapping relationship is corrected to obtain a corrected mapping relationship so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned; the target coordinate point is the coordinate point corresponding to the camera image of the front and rear cameras and / or the camera image of the left and right cameras.

6. The method according to claim 5, characterized in that The step of correcting the pixel positions of the target image pixels corresponding to the target coordinate points having a non-zero height on the stereoscopic projection model in the mapping relationship according to the position difference information to obtain a corrected mapping relationship includes: For a target coordinate point whose height is lower than a preset height, determining a third adjustment amount for the image pixel based on a product of a quotient of the height of the target coordinate point and the preset height and the position difference information, and adjusting the pixel position of the target image pixel based on the third adjustment amount to obtain an adjusted target image pixel; For a target coordinate point whose height is not lower than the preset height, a fourth adjustment amount of the image pixel is determined according to the position difference information, and the pixel position of the target image pixel is adjusted according to the fourth adjustment amount to obtain an adjusted target image pixel.

7. A device for generating a vehicle's surrounding stereoscopic effect image, characterized in that: The device comprises: An information acquisition module, configured to acquire position difference information between adjacent cameras disposed around the vehicle; a relationship determination module, configured to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera image based on the position difference information; An effect diagram generation module, configured to generate a surround-view stereo effect diagram of the vehicle based on the mapping relationship, the stereo projection model, and the camera image; The relationship determination module includes: a coordinate correction submodule, configured to correct the coordinate values ​​of target coordinate points on the stereoscopic projection model whose height is not zero based on the position difference information; wherein the target coordinate points are coordinate points corresponding to the camera images of the front and rear cameras and / or the camera images of the left and right cameras; The first mapping determination submodule is used to determine a mapping relationship between coordinate points on the stereoscopic projection model and image pixels of the camera images based on intrinsic parameter information and extrinsic parameter information of the multiple cameras and the coordinate points on the stereoscopic projection model, so that the camera images of the front and rear cameras and the camera images of the left and right cameras are not misaligned.

8. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the machine-readable media to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer program stored therein enables a processor to execute the method according to any one of claims 1 to 6.

Images