Method, device and storage medium for generating 3D image

Abstract

The application provides a 3D image generation method and related equipment, which can obtain a clearer 3D image and improve the viewing experience of a user. The method comprises the following steps: reading an initial color image and an initial depth image from a back-end cache, wherein the initial color image and the initial depth image have a correlation relationship; performing remapping on the initial depth image to obtain a target depth image; determining a relative displacement corresponding to the initial color image; performing rendering on the initial color image based on the relative displacement and the target depth image to obtain a target color image; and generating a 3D image according to the target color image and the initial color image.

Description

Technical field

[0001] The present invention belongs to the field of image processing, and in particular relates to a method, device and storage medium for generating a 3D image. [Background Technology]

[0002] With the development of science and technology, products with 3D display technology have been widely used in people's daily lives. The 3D videos displayed by 3D display technology have a stronger visual impact and can give consumers an immersive feeling.

[0003] Currently, products with 3D display technology generally convert 2D videos into 3D videos through a view conversion method for stereoscopic display. However, the 3D video obtained by converting 2D videos into 3D videos through the view conversion method currently has an imperfect 3D display effect and is prone to blurring, which reduces the user's viewing experience. [Summary of the invention]

[0004] The present invention provides a method, device and storage medium for generating a 3D image, which can obtain a clearer 3D image and improve the viewing experience of users.

[0005] A first aspect of the present invention provides a method for generating a 3D image, comprising:

[0006] Reading an initial color image and an initial depth image from a backend cache, wherein the initial color image and the initial depth image are associated with each other;

[0007] Remapping the initial depth image to obtain a target depth image;

[0008] determining a relative displacement corresponding to the initial color image;

[0009] Rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image;

[0010] A 3D image is generated according to the target color image and the initial color image.

[0011] A second aspect of the present invention provides a 3D image generation device, comprising:

[0012] A reading unit, configured to read an initial color image and an initial depth image from a backend cache, wherein the initial color image and the initial depth image are associated with each other;

[0013] A mapping unit, configured to remap the initial depth image to obtain a target depth image;

[0014] a determining unit, configured to determine a relative displacement corresponding to the initial color image;

[0015] a rendering unit, configured to render the initial color image based on the relative displacement and the target depth image to obtain a target color image;

[0016] A generating unit is configured to generate a 3D image according to the target color image and the initial color image.

[0017] A third aspect of an embodiment of the present invention provides a computer device comprising at least one connected processor, memory, and transceiver, wherein the memory is used to store program code, and the processor is used to call the program code in the memory to execute the steps of the 3D image generation method described in the first aspect above.

[0018] A fourth aspect of the embodiments of the present invention provides a computer storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the steps of the method for generating a 3D image as described in any one of the above aspects.

[0019] Compared with the related art, in the embodiment provided by the present invention, the initial depth image of the to-be-generated 3D image is remapped to obtain a target depth image, and the relative displacement corresponding to the initial color image of the to-be-generated 3D image is determined. The initial color image is then rendered using the relative displacement and the target depth image to obtain a target color image, and the target color image is interwoven with the initial color image to generate a 3D image. Thus, a better and clearer 3D image can be obtained, thereby improving the user's viewing experience.

Brief Description of the Drawings

[0020] Figure 1 A schematic diagram of a flow chart of a method for generating a 3D image provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the angle between the human eye and the screen provided in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of relative camera displacement during rendering of an initial color image provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of a preset path provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of a virtual structure of a 3D image generation device provided by an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of the hardware structure of a server provided in an embodiment of the present invention. [Specific implementation method]

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0027] The following describes a method for generating a 3D image from the perspective of a 3D image generating device. The 3D image generating device may be a server or a service unit in the server, and the specific details are not limited thereto.

[0028] Please refer to Figure 1 , Figure 1 A schematic flow chart of a method for generating a 3D image provided in an embodiment of the present invention includes:

[0029] 101. Read the initial color image and the initial depth image from the backend cache.

[0030] In this embodiment, modern 3D rendering frameworks (OpenGL, Metal, Vulkan, DirextX) use dual / multi-buffer technology, meaning that the currently rendered image is not displayed on the screen in real time. The rendered content is stored in the backend cache, and the rendered content will not be displayed on the screen until the manager swaps the front and back caches. Therefore, when the swap is about to occur, the color image and depth image in the backend cache are read and re-rendered to generate images from other viewpoints. The newly generated multi-viewpoint images are then swapped with the front-end cache as backend content. Thus, the 3D image generation device can read the initial color image and initial depth image to be displayed in 3D from the backend cache. The initial color image and initial depth image are associated with each other, meaning that both are converted from the target original image. The target original image is an RGB image, consisting of two images: a conventional RGB three-channel color image containing the color information of the graphic, and a depth image containing depth information. The initial color image is an RGB three-channel color image, including multiple pixels, each pixel is represented by a coordinate value (x, y), each pixel has a pixel value, and the pixel value is RGB color information. The initial depth image is a depth image, including multiple pixels, the pixels of the initial depth image have coordinates corresponding to the target depth image, and each pixel of the initial depth image has a depth value representing depth information.

[0031] 102. Remap the initial depth image to obtain a target depth image.

[0032] In this embodiment, after the 3D image generating device reads the initial depth image to be displayed in 3D from the backend cache, it can remap the initial depth image to obtain a target depth image. Specifically, the initial depth image can be remapped to the target depth image using the following formula:

[0033] I d1 (z)=I d (z) / Dmax*MaxDepth;

[0034] Among them, I d is the initial depth image, I d1 is the target depth image, I d (z) is the depth value of any pixel in the initial depth image, I d1 (z) is the coordinate of the target depth image and I d (z) The depth value of the pixel with the same coordinate, Dmax is the maximum depth value of all pixels in the initial depth image, MaxDepth is an empirical value and can be 100. Of course, it can also be other values, such as 80, 90, and 110, etc., without limitation, as long as it does not exceed the maximum floating-point value. The coordinates of each pixel in the remapped target depth image correspond one-to-one with the coordinates of each pixel in the initial depth image, and the depth value corresponding to the initial depth image is different from the depth value of the target depth image.

[0035] 103. Determine the relative displacement corresponding to the target depth image.

[0036] In this embodiment, the 3D image generation device can determine the relative displacement corresponding to the target depth image. Specifically, the 3D image generation device can determine the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image, where the target original image is the original image corresponding to the initial color image and the initial depth image; determine the rendering position information based on the Y-axis rotation angle and the X-axis rotation angle; and determine the relative displacement based on the initial position information and the rendering position information.

[0037] The following combination Figure 2 For a detailed explanation of how to determine relative displacement, see Figure 2 , Figure 2 A schematic diagram of the angle between the human eye position and the screen provided in an embodiment of the present invention is shown in FIG. Figure 2As shown, 201 is the position of the human eye. In a space with the center of the display screen (the center of the screen is the display screen corresponding to the 3D image) as the origin of the O-XYZ three-dimensional coordinate system, the angle between the projection of the line connecting the user's eye to the center of the screen on the X0Z plane and the positive half-axis of the Z axis is α, and the angle between the projection of the line connecting the user's eye and the center of the screen on the Y0Z plane and the positive half-axis of the Z axis is β. The X-axis is in the same direction as the left-right direction of the display screen, with the positive direction of the X-axis pointing from the center of the left side of the display screen to the center of the right side of the display screen. The Y-axis is in the same direction as the up-down direction of the display screen, with the positive direction of the Y-axis pointing from the upper midpoint of the display screen to the lower midpoint of the display screen. Based on the angles α and β, the distance H between the user's eye and the display screen, and the distance J between the center of the scene corresponding to the target original image and the display screen, the angle a of the scene corresponding to the target original image around the Y axis (i.e., the Y-axis rotation angle corresponding to the target original image) and the angle b of the scene corresponding to the target original image around the X axis (i.e., the X-axis rotation angle corresponding to the target original image) can be calculated. Specifically, the angle a of rotation of the scene corresponding to the target original image around the Y axis can be calculated by the following formula:

[0038] a=arctan(H×tanα / (H+J));

[0039] Wherein, a is the Y-axis rotation angle, H is the distance between the user's eyes and the display screen, α is the angle between the first target projection and the positive Z-axis in the coordinate system, the first target projection is the projection of the target line on the X0Z plane, the target line is the line connecting the user's eyes and the center of the display screen, J is the distance between the center of the scene corresponding to the target original image and the display screen, and the display screen is the screen corresponding to the 3D image;

[0040] The angle b of the scene rotation around the X axis corresponding to the target original image is calculated using the following formula:

[0041] b = arctan(H × tanβ / (H+J));

[0042] Wherein, b is the X-axis rotation angle, β is the angle between the second target projection and the positive half axis of the Z axis in the coordinate system, the second target projection is the projection of the target line on the Y0Z plane, and 0 is the origin of the coordinate system.

[0043] After determining the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image, the 3D image display device can determine the rendering position information according to the Y-axis rotation angle and the X-axis rotation angle. Specifically, the rendering position information can be determined by the following formula:

[0044] Dx=tana*Rd;

[0045] Dy=tanb*Rd;

[0046] Among them, Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant. Rd can be 100 or 1000. Of course, it can also be set according to actual conditions and is not specifically limited.

[0047] After determining the rendering position information, the display device of the 3D image can determine the relative displacement between the rendering position information and the initial position information corresponding to the target original image, such as Figure 3 As shown, Figure 3 A schematic diagram of relative displacement provided in an embodiment of the present invention, wherein the position of camera 301 is C0, which is the initial position information of the target original image. The space where camera 301 is located is defined as a three-dimensional space bounded by the x-axis, y-axis, and z-axis. The coordinates of C0 are (0, 0, z). By changing the position of camera 301, different rendered images are obtained. To maintain the consistency of the size of different rendered images, camera 301 remains unchanged on the z-axis. The changed position of camera 301 is C1, that is, C1 is the rendering position information. The coordinates of C1 are (nx, ny, z). The change in the position of camera 301 is a relative displacement. The relative displacement of camera 301 is determined by the following formula:

[0048] D=C1-C0=(nx,ny,0).

[0049] 104. Render the initial color image based on the relative displacement and the target depth image to obtain a target color image.

[0050] In this embodiment, after determining the relative displacement, the 3D image generation device can render the initial color image based on the relative displacement and the target depth image to obtain the target color image. The following is a detailed description of how to render to obtain the target color image:

[0051] Step A1: determining the initial point cloud corresponding to the target depth image;

[0052] In this step, the 3D image generation device converts the target depth image into an initial point cloud using the following formula:

[0053] P0(x,y,z)=(-w / 2+x,-h / 2+y,I d1 (x,y));

[0054] Among them, 0≤x<w,0≤y<h, P0 is the initial point cloud, P0(x,y,z) is the coordinate of any point in the initial point cloud, w is the width of the target depth image, h is the height of the target depth image, I d1(x,y) is the depth value of the pixel with coordinates (x,y) in the target depth image.

[0055] Step A2: Adjust the coordinates of each point in the initial point cloud by relative displacement to obtain the target point cloud.

[0056] In this step, the relative displacement is added to each point of the initial point cloud to obtain the target point cloud, which is calculated using the following formula:

[0057] P1(x,y,z)=P0(x,y,z)+D;

[0058] Where P0(x, y, z) is the coordinate of any point in the initial point cloud, P1 is the target point cloud, and P1(x, y, z) is the coordinate of the point in the target point cloud obtained by adding D to P0(x, y, z). Since the camera position is unchanged on the z-axis and the D value is (nx, ny, 0), the z value of the target point cloud and the initial point cloud coordinates are the same.

[0059] Step A3: Process the coordinates of each point in the target point cloud to obtain a reference image.

[0060] In this step, the reference image is a depth image that matches the size of the target depth image. The depth value of the reference image is initially assigned to A. The coordinates of each point in the target point cloud are processed using the following formula to obtain the reference direction:

[0061] Z(x,y)=min(Z(IP.x+1,IP.y+1),FltErr);

[0062] Where Z is the reference image, Z(x,y) is the depth value of the pixel with coordinates (x,y) in the reference image, Z(IP.x+1,IP.y+1) is the depth value of the pixel with coordinates (IP.x+1,IP.y+1) in the reference image, FltErr = Aw / 2 / z0, and min is the smaller value of Z(IP.x+1,IP.y+1) and FltErr assigned to Z(x,y).

[0063] IP=LP+Dis*DLP, Dis=(PP-LP)*PN, DLP=-LP, A is the initial depth value of the reference image, w is the width of the target depth image, (x0, y0, z0) is the coordinate of any point in the target point cloud, IP.x is the x value of the IP coordinate, and IP.y is the y value of the IP coordinate.

[0064] The reference image is a depth map of the same size as the target depth image. The initial depth value of the reference image is A, which can be 100,000, 90,000, or 110,000, as long as the value of A is greater than the depth value of the target depth image and less than the maximum floating-point number.

[0065] Step D: Processing the pixels in the initial color image according to the depth values ​​of the pixels in the reference image to obtain a target color image.

[0066] In this step, the target color image is determined by the following formula:

[0067] I c1 (IP.x,IP.y)=I c (x,y)*((Z(x,y)+1)>FltErr);

[0068] Among them, I c is the initial color image, I c1 is the target color image, IP.x is the x value of the IP coordinate, and IP.y is the y value of the IP coordinate. c (x,y) is the pixel value of the pixel with coordinates (x,y) in the initial color image, Z(x,y) is the depth value of the pixel with coordinates (x,y) in the reference image, I c1 (IP.x, IP.y) is the pixel value of the pixel with coordinates (IP.x, IP.y) in the target color image.

[0069] When (Z(x,y)+1)>FltErr, the value of ((Z(x,y)+1)>FltErr) is 1, and the pixel value of the pixel with coordinates (x,y) in the initial depth image is assigned to the pixel with coordinates (IP.x,IP.y) in the target color image.

[0070] When (Z(x,y)+1)>FltErr is not satisfied, the value of ((Z(x,y)+1)>FltErr) is 0, and the pixel value of the pixel point with coordinates (IP.x, IP.y) in the target color image is assigned to 0.

[0071] It should be noted that after determining the target color image, the 3D image generation device can determine whether there is a hole in the target color image, which is described in detail below:

[0072] First, the initial depth image is processed by the following formula to obtain the hole-filled depth image:

[0073] I d2 (IP.x,IP.y)=I d(x,y)*((Z(x,y)+1)>FltErr);

[0074] Among them, I d is the initial depth image, I d2 Fill the depth image with holes, I d (x, y) is the depth value of the pixel with coordinates (x, y) in the initial depth image, I d2 (IP.x, IP.y) is the depth value of the pixel with coordinates (IP.x, IP.y) in the hole-filling depth image. When ((Z(x,y)+1)>FltErr) is 1, the depth value of the pixel with coordinates (x,y) in the initial depth image is assigned to the pixel with coordinates (IP.x, IP.y) in the hole-filling depth image. When ((Z(x,y)+1)>FltErr) is 0, the depth value of the pixel with coordinates (IP.x, IP.y) in the hole-filling depth image is assigned to 0.

[0075] Afterwards, the 3D image generation device determines whether the pixel points in the target color image with the same coordinates as the pixel points in the hole-filling depth image are holes based on the depth values ​​of the pixel points in the hole-filling depth image, and determines the pixel points in the target color image that are holes as target pixels.

[0076] The following describes how to determine whether a target pixel exists in a target color image:

[0077] If I d2 (x,y)≤0, then the pixel with coordinates (x,y) in the target color image is determined to be the target pixel with a hole, where I d2 is the target depth image, I d2 (x, y) is the depth value of the reference pixel point, and the reference pixel point is the pixel point with coordinates (x, y) in the target depth image. That is, when I d2 When (x,y)≤0, the pixel value of the pixel with coordinates (x,y) in the target color image is assigned to 0, and no color value is assigned, so the existence of this pixel is a hole.

[0078] Afterwards, the 3D image generation device determines the hole filling value of the target pixel point according to the coordinate value and depth value of each pixel point in the hole filling depth image, as described in detail below:

[0079] Step B1: Set a preset path for each pixel point in the hole filling depth image to traverse.

[0080] In this embodiment, the preset paths can be set according to actual conditions. For example, the preset paths can be 16, 6, or 5, and there is no specific limitation. The preset paths can be debugged according to the generation of the target 3D image.

[0081] Taking 16 preset paths as an example, the traversal method is described in detail.

[0082] The preset path is represented by Dirs, and 16 traversal search directions are preset: Dirs = (-1, 1), (0, 1), (1, 1), (1, 0), (-1, 2), (1, 2), (2, 1), (2, -1), (-2, 3), (-1, 3), (1, 3), (2, 3), (3, 2), (3, 1), (3, -1), (3, -2). Figure 4 , Figure 4 A schematic diagram of a preset path provided in an embodiment of the present invention, Figure 4 The default path shown is Dirs = (-2, 3).

[0083] Step B2: traverse a preset path based on the coordinates of each pixel in the hole-filled depth image to determine a first target pixel and a second target pixel that meet preset conditions.

[0084] The following combination Figure 4 The following steps describe the traversal of each pixel in the hole-filled depth image. Taking each pixel in the hole-filled depth image as the starting point, the coordinates of each pixel in the hole-filled depth image are (x, y), and traversal of each preset direction is performed through the following steps:

[0085] Step B21: perform negative direction traversal using the following formula:

[0086] FromX=FromX-Dirs[i][0],

[0087] FromY=FromY-Dirs[i][1].

[0088] Until I d2 (FromX, FromY)>0 or one of FromX and FromY exceeds the boundary of the hole-filled depth image.

[0089] Among them, [i] in Dirs[i][0] and Dirs[i][1] represents the preset path of traversal, [0] means that Dirs[i][0] takes the value on the left of the preset path coordinate value, and [1] means that Dirs[i][1] takes the value on the right of the path coordinate value. For example, if the preset path is Dirs=(-2,3), when traversing in the negative direction with (-2,3) as the preset direction (such as Figure 4shown), then FromX=FromX-(-2), FromY=FromY-3.

[0090] Step B22: perform forward traversal using the following formula:

[0091] ToX=ToX+Dirs[i][0],

[0092] ToY=ToY+Dirs[i][1].

[0093] Until I d2 (ToX, ToY)>0 or one of ToX and ToY exceeds the boundary of the hole-filled depth image.

[0094] Among them, [i] in Dirs[i][0] and Dirs[i][1] represents the preset path of traversal, [0] means that Dirs[i][0] takes the value on the left of the preset path coordinate value, and [1] means that Dirs[i][1] takes the value on the right of the path coordinate value. For example, if the preset path is Dirs=(-2,3), when traversing forward with (-2,3) as the preset direction (such as Figure 4 As shown), then ToX=ToX+(-2), ToY=ToY+3.

[0095] Step B23: Determine whether FromX, FromY, ToX and ToY exceed the boundary of the hole filling depth image. If one of them exceeds the boundary,

[0096] FltDis=FLOAT_MAX;

[0097] If FromX, FromY, ToX and ToY do not exceed the boundary of the hole-filled depth image, then

[0098] Among them, FLOAT_MAX is the maximum floating-point value.

[0099] Step B24: After traversing all 16 preset paths, determine the FromX, FromY, ToX and ToY with the smallest value of FltDis, and use (FromX, FromY) as the coordinates of the first target pixel point in the depth image to fill the hole, and use (ToX, ToY) as the coordinates of the second target pixel point in the depth image to fill the hole.

[0100] Step B3: Determine a hole filling value of the target pixel point according to the coordinates of the first target pixel point and the coordinates of the second target pixel point.

[0101] In this embodiment, if the depth values ​​of the first target pixel and the second target pixel satisfy I d2(FromX,FromY)<I d2 (ToX, ToY), then determine FillX = FromX, FillY = FromY; if the depth values ​​of the first target pixel and the second target pixel meet I d2 (FromX,FromY)≥I d2 (ToX, ToY), then determine FillX = ToX, FillY = ToY.

[0102] Among them, I d2 (FromX, FromY) is the depth value of the first target pixel, I d2 (ToX, ToY) is the depth value of the second target pixel.

[0103] The hole filling value is determined by the following formula:

[0104] I c1 (x,y)=I c1 (FillX,FillY);

[0105] Among them, I c1 is the target color image, (x, y) is the coordinate of the target pixel, I c1 (x, y) the hole filling value of the target pixel, I c1 (FillX, FillY) is the pixel value of the pixel with coordinates (FillX, FillY) in the target color image. The formula means that the pixel value of the pixel with coordinates (FillX, FillY) in the target color image is determined as the hole filling value.

[0106] Finally, the hole filling value is filled into the target pixel point to fill the holes in the target color image and obtain a target color image without holes.

[0107] 105. Generate a 3D image according to the target color image and the initial color image.

[0108] In this embodiment, the 3D image generating device scales the target color image and the initial color image respectively to obtain a left color image and a right color image. That is, the 3D image generating device scales the initial color image to a size of (w / 2, h) to obtain a left color image, and scales the target color image to a size of (w / 2, h) to obtain a right color image. Thereafter, the left color image is rewritten into a left half area Rect(0, 0, w / 2, h) of the back-end cache, and the right color image is written into a right half area Rect(w / 2, 0, w / 2, h) of the back-end cache. When it is determined that a 3D image is to be output, the left half area and the right half area of ​​the back-end cache are interleaved to obtain and output a 3D image.

[0109] In summary, it can be seen that in the embodiments provided by the present invention, the initial depth image of the 3D image to be generated is remapped to obtain a target depth image, and the relative displacement corresponding to the initial color image of the 3D image to be generated is determined. Then, the initial color image is rendered using the relative displacement and the target depth image to obtain a target color image, and the target color image is interwoven with the initial color image to generate a 3D image. In this way, a better and clearer 3D image can be obtained, thereby improving the user's viewing experience.

[0110] The present invention is described above from the perspective of a method for generating a 3D image. The present invention will be described below from the perspective of a device for generating a 3D image.

[0111] See also Figure 5 , Figure 5 A schematic diagram of a virtual structure of a 3D image generation device provided in an embodiment of the present invention. The 3D image generation device 400 includes:

[0112] A reading unit 501 is configured to read an initial color image and an initial depth image from a backend cache, wherein the initial color image and the initial depth image are associated with each other;

[0113] A mapping unit 502 is configured to remap the initial depth image to obtain a target depth image;

[0114] A determination unit 503 is configured to determine a relative displacement corresponding to the initial color image;

[0115] a rendering unit 504, configured to render the initial color image based on the relative displacement and the target depth image to obtain a target color image;

[0116] The generating unit 505 is configured to generate a 3D image according to the target color image and the initial color image.

[0117] In one possible design, the determining unit 503 is specifically configured to:

[0118] Determining a Y-axis rotation angle and an X-axis rotation angle corresponding to a target original image, wherein the target original image is an original image corresponding to the initial color image and the initial depth image;

[0119] Determine rendering position information according to the Y-axis rotation angle and the X-axis rotation angle;

[0120] The relative displacement is determined according to the initial position information corresponding to the target original image and the rendering position information.

[0121] In a possible design, the determining unit 503 determines the rendering position information according to the Y-axis rotation angle and the X-axis rotation angle, including:

[0122] The rendering position information is determined by the following formula:

[0123] Dx=tana*Rd;

[0124] Dy=tanb*Rd;

[0125] Wherein, Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant.

[0126] In one possible design, the determining unit 503 determines the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image, including:

[0127] The Y-axis rotation angle is determined by the following formula:

[0128] a=arctan(H×tanα / (H+J));

[0129] Wherein, a is the Y-axis rotation angle, H is the distance between the user's eyes and the display screen, α is the angle between the first target projection and the positive Z-axis in the coordinate system, the first target projection is the projection of the target line on the X0Z plane, the target line is the line connecting the user's eyes and the center of the display screen, and J is the distance between the center of the scene corresponding to the target original image and the display screen, where the display screen is the screen corresponding to the 3D image;

[0130] The X-axis rotation angle is determined by the following formula:

[0131] b = arctan(H × tanβ / (H+J));

[0132] Wherein, b is the X-axis rotation angle, β is the angle between the second target projection and the positive half axis of the Z axis in the coordinate system, the second target projection is the projection of the target line on the Y0Z plane, and 0 is the origin of the coordinate system.

[0133] In one possible design, the generating unit 505 is specifically configured to:

[0134] Respectively scaling the initial color image and the target color image to obtain a left color image and a right color image;

[0135] writing the left color image and the right color image into the backend cache in an associated manner;

[0136] If a display instruction is received, the left color image and the right color image written in the backend buffer are interleaved to generate the 3D image.

[0137] In one possible design, the rendering unit 504 is specifically configured to:

[0138] Determining an initial point cloud corresponding to the target depth image;

[0139] Adjusting the coordinates of each point in the initial point cloud by the relative displacement to obtain a target point cloud;

[0140] Processing the coordinates of each point in the target point cloud to obtain a reference image;

[0141] The pixel points in the initial color image are processed according to the depth values ​​of the pixel points in the reference image to obtain the target color image.

[0142] Figure 6 This is a schematic diagram of the structure of the server of the present invention, as shown in FIG. Figure 6 As shown, the server 600 of this embodiment includes at least one processor 601, at least one network interface 604 or other user interface 603, a memory 606, and at least one communication bus 602. The server 600 optionally includes a user interface 603, including a display, a keyboard, or a pointing device. The memory 605 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 605 stores execution instructions. When the server 600 is running, the processor 601 communicates with the memory 605, and the processor 601 calls the instructions stored in the memory 605 to execute the above-mentioned 3D image generation method. The operating system 606 includes various programs for implementing various basic services and processing tasks based on the hardware.

[0143] The server provided in the embodiment of the present invention can execute the technical solution of the embodiment of the above-mentioned 3D image generation method. Its implementation principle and technical effects are similar and will not be repeated here.

[0144] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the method flow associated with the 3D image generation device in any of the above method embodiments. Accordingly, the computer may be the 3D image generation device described above.

[0145] Embodiments of the present invention further provide a computer program or a computer program product including the computer program. When executed on a computer, the computer program causes the computer to implement the method flow associated with the 3D image generation device in any of the aforementioned method embodiments. Accordingly, the computer may be the aforementioned 3D image generation device.

[0146] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0147] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0148] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0149] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0151] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0153] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a 3D image, characterized in that: include: Reading an initial color image and an initial depth image from a backend cache, wherein the initial color image and the initial depth image are associated with each other; Remapping the initial depth image to obtain a target depth image; Determining a relative displacement corresponding to the target depth image; Rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image; generating a 3D image according to the target color image and the initial color image, Determining the relative displacement corresponding to the target depth image includes: Determine the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image, where the target original image is the original image corresponding to the initial color image and the initial depth image, wherein the center of the display screen is used as the origin of the O-XYZ three-dimensional coordinate system, the X-axis is in the same direction as the left-right direction of the display screen, and the Y-axis is in the same direction as the up-down direction of the display screen. The angle of rotation of the scene corresponding to the target original image around the Y-axis is the Y-axis rotation angle corresponding to the target original image, and the angle of rotation of the scene corresponding to the target original image around the X-axis is the X-axis rotation angle corresponding to the target original image; Determine rendering position information according to the Y-axis rotation angle and the X-axis rotation angle, The relative displacement is determined according to the initial position information corresponding to the target original image and the rendering position information.

2. The method according to claim 1, characterized in that: The determining of the rendering position information according to the Y-axis rotation angle and the X-axis rotation angle includes: The rendering position information is determined by the following formula: ； ； in, is the X-axis coordinate value corresponding to the rendering position information, is the Y-axis coordinate value corresponding to the rendering position information, is the Y-axis rotation angle, is the X-axis rotation angle, is a preset constant.

3. The method according to claim 1, characterized in that Determining the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image includes: The Y-axis rotation angle is determined by the following formula: ； in, is the Y-axis rotation angle, is the distance between the user's eyes and the display screen, is the angle between the first target projection and the positive half axis of the Z axis in the coordinate system, where the first target projection is the projection of the target line on the X0Z plane, where the target line is the line connecting the user's eye and the center of the display screen, is the distance between the center of the scene corresponding to the target original image and the display screen, where the display screen is the screen corresponding to the 3D image; The X-axis rotation angle is determined by the following formula: ； in, is the X-axis rotation angle, is the angle between the second target projection and the positive half axis of the Z axis in the coordinate system, the second target projection is the projection of the target line on the Y0Z plane, and 0 is the origin of the coordinate system.

4. The method according to any one of claims 1 to 3, characterized in that The rendering of the initial color image based on the relative displacement and the target depth image to obtain a target color image includes: Determining an initial point cloud corresponding to the target depth image; Adjusting the coordinates of each point in the initial point cloud by the relative displacement to obtain a target point cloud; Processing the coordinates of each point in the target point cloud to obtain a reference image; The pixel points in the initial color image are processed according to the depth values ​​of the pixel points in the reference image to obtain the target color image.

5. The method according to any one of claims 1 to 3, characterized in that Generating a 3D image according to the target color image and the initial color image includes: Respectively scaling the initial color image and the target color image to obtain a left color image and a right color image; writing the left color image and the right color image into the backend cache in an associated manner; If a display instruction is received, the left color image and the right color image written in the backend buffer are interleaved to generate the 3D image.

6. A 3D image generation device, characterized in that: include: A reading unit, configured to read an initial color image and an initial depth image from a backend cache, wherein the initial color image and the initial depth image are associated with each other; A mapping unit, configured to remap the initial depth image to obtain a target depth image; a determining unit, configured to determine a relative displacement corresponding to the target depth image; a rendering unit, configured to render the initial color image based on the relative displacement and the target depth image to obtain a target color image; a generating unit, configured to generate a 3D image according to the target color image and the initial color image, The determining unit is specifically configured to: Determine the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image, where the target original image is the original image corresponding to the initial color image and the initial depth image, wherein the center of the display screen is used as the origin of the O-XYZ three-dimensional coordinate system, the X-axis is in the same direction as the left-right direction of the display screen, and the Y-axis is in the same direction as the up-down direction of the display screen. The angle of rotation of the scene corresponding to the target original image around the Y-axis is the Y-axis rotation angle corresponding to the target original image, and the angle of rotation of the scene corresponding to the target original image around the X-axis is the X-axis rotation angle corresponding to the target original image; Determine rendering position information according to the Y-axis rotation angle and the X-axis rotation angle, The relative displacement is determined according to the initial position information corresponding to the target original image and the rendering position information.

7. A computer device, characterized in that: include: At least one connected processor, memory and transceiver, wherein the memory is used to store program code, and the processor is used to call the program code in the memory to execute the steps of the 3D image generation method according to any one of claims 1 to 5.

8. A computer storage medium, characterized in that include: The instructions, when executed on a computer, cause the computer to execute the steps of the method for generating a 3D image according to any one of claims 1 to 5.

Images