Method, apparatus, and computer device for generating a 3D image

Abstract

This application provides a method for generating a 3D image and a related device that can obtain a clearer 3D image and improve the user's viewing experience. This method includes the steps of reading an initial color image and an initial depth image associated with each other from a back buffer, remapping the initial depth image to obtain a target depth image, determining a relative displacement corresponding to the initial color image, rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image, and generating a 3D image based on the target color image and the initial color image.

Description

Technical Field

[0001] This application belongs to the field of image processing, and particularly relates to a method, apparatus, and computer device for generating 3D images.

Background Art

[0002] With the development of science and technology, products with 3D display technology are generally applied to people's daily lives. The 3D images displayed by 3D display technology have a strong visual impact and can give consumers a sense of presence.

[0003] Currently, products with 3D display technology usually convert 2D images into 3D images by a view conversion method for stereoscopic display. However, currently, the 3D images obtained by converting 2D images into 3D images by a view conversion method have an incomplete 3D display effect and are prone to blurring, which reduces the viewing experience of users.

Summary of the Invention

[0004] This application provides a method, apparatus, and computer device for generating 3D images, which can obtain clearer 3D images and improve the viewing experience of users.

[0005] The method for generating a 3D image according to the first aspect of this application includes: reading an initial color image and an initial depth image associated with each other from a back-end buffer; remapping the initial depth image to obtain a target depth image; determining a relative displacement corresponding to the initial color image; rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image; generating a 3D image based on the target color image and the initial color image.

[0006] The apparatus for generating a 3D image according to the second aspect of this application includes: A reading unit for reading out an initial color image and an initial depth image associated with each other from a backend buffer, A mapping unit for remapping the initial depth image to obtain a target depth image, A determination unit for determining a relative displacement corresponding to the initial color image, A rendering unit for rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image, A generation unit for generating a 3D image based on the target color image and the initial color image, and includes.

[0007] The computer device according to the third aspect of the present application includes at least one connected processor, a memory, and a transceiver, the memory is used to store program code, and the processor calls the program code in the memory to execute the steps of the 3D image generation method described in the first aspect above.

[0008] From the above, in the embodiment provided by the present application, the initial depth image to be used to generate a 3D image is remapped to obtain a target depth image, and the relative displacement corresponding to the initial color image to be used to generate a 3D image is determined. Then, based on the relative displacement and the target depth image, the initial color image is rendered to obtain a target color image. And by interleaving the target color image and the initial color image to generate a 3D image, a more excellent and clear 3D image can be obtained, and the viewing experience of the user can be improved.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0010] Hereinafter, in accordance with the drawings of the embodiments of the present application, the technical aspects in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments.

[0011] Hereinafter, a method for generating a 3D image will be described from the perspective of a 3D image generation device. The 3D image generation device may be a server or a service unit in the server, and is not specifically limited.

[0012] In connection with referring to FIG. 1, FIG. 1 is a flowchart of a method for generating a 3D image provided in an embodiment of the present application, and includes the following steps.

[0013] In step 101, an initial color image and an initial depth image are read from the backend buffer.

[0014] In this embodiment, in modern 3D rendering frameworks (OpenGL, Metal, Vulkan, DirectX), dual / multi-buffer technology is used. That is, the image being currently rendered is not directly displayed on the screen in real time. Of course, the rendered content is stored in the backend cache. To display the rendered content on the screen, the manager needs to wait for the front and back buffers to be swapped. Therefore, when the swap is about to occur, the color image and depth image of the backend buffer are read out and re-rendered to generate images from other viewpoints, and the newly generated multi-viewpoint images are swapped with the front-end buffer as backend content. As a result, the 3D image generation device can read the initial color image and initial depth image for 3D display from the backend buffer. The initial color image is associated with the initial depth image. That is, both the initial color image and the initial depth image are converted from the target original image. The target original image is an RGBD image, which includes two images, one is a normal RGB 3-channel color image containing graphic color information, and the other is a depth image containing depth information. The initial color image is an RGB 3-channel color image and includes a plurality of pixel points. Each pixel point is represented by coordinate values (x, y) and has a pixel value. The aforementioned pixel value is RGB color information. The initial depth image is a depth image and includes a plurality of pixel points. The pixel points of the initial depth image have coordinates corresponding to the target depth image. Each pixel point of the initial depth image has a depth value representing depth information.

[0015] In step 102, the initial depth image is remapped to obtain the target depth image.

[0016] In this embodiment, after the 3D image generation device reads the initial depth image to be 3D displayed from the backend buffer, the initial depth image can be remapped to obtain the target depth image. Specifically, the initial depth image can be remapped to the target depth image according to the following formula. [Equation 1] JPEG2025519947000002.jpg9170

[0017] Here, I d is the initial depth image, and I d1 is the target depth image, and I d(z) is the depth value of any pixel point in the initial depth image, and I d 1(z) is the depth value of the pixel point whose coordinates in the target depth image are I d(z) the same as the coordinates, Dmax is the maximum depth value of all pixel points in the initial depth image, MaxDepth is an empirical value, and the value can be set to 100. Of course, other values, such as 80, 90, 110, etc. can also be used, and specifically it is not limited as long as it does not exceed the maximum floating-point value. The coordinates of each pixel point of the remapped target depth image correspond one-to-one to the coordinates of each pixel point in the initial depth image. Moreover, the depth value corresponding to the initial depth image is different from the depth value of the target depth image.

[0018] In step 103, the relative displacement corresponding to the initial color image is determined.

[0019] In this embodiment, the 3D image generation device can determine the relative displacement corresponding to the initial color image. Specifically, the 3D image generation device can determine the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image. The target original image is the original image corresponding to the initial color image and the initial depth image. Based on the Y-axis rotation angle and the X-axis rotation angle, the rendering position information is determined. Then, the relative displacement is determined based on the initial position information and the rendering position information.

[0020] Next, in accordance with FIG. 2, how to determine the relative displacement will be described in detail. Refer to FIG. 2. FIG. 2 is a schematic diagram of the position of a human eye and the angle of a screen provided in an embodiment of the present application. As shown in FIG. 2, 201 is the position of the human eye. In a space where the center of the display screen (the screen center is the display screen corresponding to the 3D image) is the origin of the O-XYZ three-dimensional coordinate system, the angle formed by the projection of the connection line from the human eye to the center of the screen on the XOZ plane and the positive half-axis of the Z-axis is α, and the angle formed by the projection of the connection line between the human eye and the center of the screen on the YOZ plane and the positive half-axis of the Z-axis is β. Here, the X-axis is in the same direction as the left-right direction of the display screen, the positive direction of the X-axis is from the left center of the display screen to the right center of the display screen, the Y-axis is in the same direction as the up-down direction of the display screen, and the positive direction of the Y-axis is from the upper midpoint of the display screen to the lower midpoint of the display screen. Based on the angles α, β, the distance H between the human eye and the display screen, and the distance J between the center of the scene corresponding to the target original image and the display screen, the angle a (i.e., the Y-axis rotation angle corresponding to the target original image) by which the scene corresponding to the target original image rotates around the Y-axis and the angle b (i.e., the X-axis rotation angle corresponding to the target original image) by which the scene corresponding to the target original image rotates around the X-axis can be calculated. Specifically, the angle a by which the scene corresponding to the aforementioned target original image rotates around the Y-axis is calculated by the following formula. [Formula 2] JPEG2025519947000003.jpg7170

[0021] Here, a is the Y-axis rotation angle, H is the distance between the human eye and the display screen, and α is the angle between the first target projection and the positive half-axis of the Z-axis in the coordinate system. The first target projection is the projection of the target connection line on the XOZ plane. The target connection line is the connection line between the human eye and the center of the display screen, J is the distance between the center of the scene corresponding to the target original image and the display screen, and the display screen is the screen corresponding to the 3D image.

[0022] The angle b by which the scene corresponding to the target original image rotates around the X-axis is calculated by the following formula. [Equation 3] JPEG2025519947000004.jpg8170

[0023] Here, b is the X-axis rotation angle, β is the angle between the second target projection and the positive half-axis of the Z-axis in the coordinate system, the second target projection is the projection of the target connection line onto the YOZ plane, and 0 is the origin of the coordinate system.

[0024] After determining the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image, the 3D image display device can determine the rendering position information based on the Y-axis rotation angle and the X-axis rotation angle. Specifically, the rendering position information is determined by the following formula. [Equation 4] JPEG2025519947000005.jpg17170

[0025] Here, Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant. Rd may take a value of 100, or may take a value of 1000, and may also be set according to the actual situation, and is not specifically limited.

[0026] Accordingly, after determining the rendering position information, the 3D image display device can calculate the relative displacement based on the rendering position information and the initial position information corresponding to the target original image. Refer to FIG. 3. FIG. 3 is a schematic diagram of the relative displacement according to an embodiment of the present application. Here, the position of the camera 301 is C0, and C0 is the initial position information of the target original image. Defining the space where the camera 301 is located as a three-dimensional space defined by the x-axis, y-axis, and z-axis, and setting the coordinates of C0 as (0, 0, z), different rendering images can be obtained by changing the position of the camera 301. In order to match the sizes of different rendering images, the camera 301 does not change in the z-axis. The position of the camera 301 after the change is C1. That is, C1 is the aforementioned rendering position information, and the coordinates of C1 are (nx, ny, z). The change in the position of the camera 301 is the aforementioned relative displacement. The relative displacement of the camera 301 is determined as follows in the following formula. [Equation 5] JPEG2025519947000006.jpg9170

[0027] In step 104, based on the relative displacement and the target depth image, the initial color image is rendered to obtain a target color image.

[0028] In this embodiment, after determining the relative displacement, the 3D image generation device renders the initial color image based on the relative displacement and the target depth image to obtain the target color image. Next, a specific description will be given on how to render to obtain the target color image:

[0029] In step A1, an initial point cloud corresponding to the first depth image is determined.

[0030] In this step, the 3D image generation device converts the target depth image into the initial point cloud by the following formula. [Equation 6] JPEG2025519947000007.jpg11170

[0031] Here, 0 ≤ x < w, 0 ≤ y < h, P0 is the initial point cloud, P0(x, y, z) is the coordinate of any one point in the initial point cloud, w is the width of the target depth image, h is the height of the target depth image, and I d1 (x, y) is the depth value of the pixel point with coordinates (x, y) in the target depth image.

[0032] In step A2, the coordinates of each point in the initial point cloud are adjusted by the relative displacement to obtain a target point cloud.

[0033] In this step, the relative displacement is added to each point of the initial point cloud to obtain the aforementioned target point cloud. Specifically, it is calculated by the following formula. [Equation 7] JPEG2025519947000008.jpg10170

[0034] Here, P0(x, y, z) is the coordinate of any one point in the initial point cloud, P1 is the aforementioned target point cloud, and P1(x, y, z) is the coordinate of the point obtained by adding D to P0(x, y, z) in the aforementioned target point cloud. Since the position of the camera does not change on the z-axis and the D value is (nx, ny, 0), the z values of the coordinates of the aforementioned target point cloud and the aforementioned initial point cloud are the same.

[0035] In step A3, the coordinates of each point in the target point cloud are processed to obtain a reference image.

[0036] In this step, the aforementioned reference image is a depth image that matches the size of the initial depth image. The initial substitution value of the depth value of the aforementioned reference image is A. The following formula is used to process the coordinates of each point in the target point cloud to obtain the aforementioned reference image. [Equation 8] JPEG2025519947000009.jpg9170

[0037] Here, Z is the aforementioned reference image. Z(x, y) is the depth value of the pixel point whose coordinates in the aforementioned reference image are (x, y). Z(IP.x + 1, IP.y + 1) is the depth value of the pixel point whose coordinates in the aforementioned reference image are (IP.x + 1, IP.y + 1), FltErr = A - w / 2 / z0, and min is the value obtained by assigning the smaller value between Z(IP.x + 1, IP.y + 1) and FltErr to Z(x, y).

[0038] [Equation 9] JPEG2025519947000010.jpg24170A is the initial depth value of the aforementioned reference depth map, w is the width of the target depth image, (x0, y0, z0) is the coordinate of any point in the target point cloud, IP.x is the x value of the point IP coordinate, and IP.y is the y value of the point IP coordinate.

[0039] Here, the reference image is a depth image that matches the size of the target depth image. The initial depth value of the reference image is A. The value of A may be 100000.0, or 90000.0, or 110000.0, and it is sufficient that the value of A is greater than the depth value of the target depth image and less than the maximum floating-point value, and it is not specifically limited.

[0040] In step D, based on the depth values of the pixel points in the reference image, the pixel points in the initial color image are processed to obtain the target color image.

[0041] In this step, the target color image is determined by the following formula. [Equation 10] JPEG2025519947000011.jpg10170

[0042] Here, I c is the initial color image, I c1 is the first color image, IP.x is the x value of the point IP coordinate, and IP.y is the y value of the point IP coordinate. I c(x, y) is the pixel value of the pixel point with coordinates (x, y) in the initial color image, Z(x, y) is the depth value of the pixel point with coordinates (x, y) in the reference image, and I c1 (IP.x, IP.y) is the pixel value of the pixel point with coordinates (IP.x, IP.y) in the target color image.

[0043] When (Z(x, y)+1)>FltErr is satisfied, if the value of ((Z(x, y)+1)>FltErr) is set to 1, the pixel value of the pixel point with coordinates (x, y) in the aforementioned initial depth image is assigned to the pixel point with coordinates (IP.x, IP.y) in the aforementioned target color image.

[0044] When (Z(x, y)+1)>FltErr is not satisfied, if the value of ((Z(x, y)+1)>FltErr) is set to 0, the pixel value of the pixel point with coordinates (IP.x, IP.y) in the aforementioned target color image is 0.

[0045] Note that after determining the target color image, the 3D image generation device can determine whether there are holes in the target color image. This will be specifically described below.

[0046] First, the initial depth image is processed by the following formula to obtain a hole-filled depth image. [Equation 11] JPEG2025519947000012.jpg8170

[0047] Here, I d is the initial depth image, I d2 is the target depth image, I d (x, y) is the depth value of the pixel point with coordinates (x, y) in the initial depth image, and I d2(IP.x, IP.y) is the depth value of the pixel point with coordinates (IP.x, IP.y) in the hole-filling depth image. When the value of ((Z(x, y) + 1) > FltErr) is 1, the depth value of the pixel point with coordinates (x, y) in the initial depth image is assigned to the pixel point with coordinates (IP.x, IP.y) in the hole-filling depth image. When the value of ((Z(x, y) + 1) > FltErr) is 0, the depth value of the pixel point with coordinates (IP.x, IP.y) in the hole-filling depth image is 0.

[0048] Then, based on the depth value of each pixel point in the hole-filling depth image, the 3D image generating device determines whether the pixel point with the same coordinates as each pixel point of the hole-filling depth image in the target color image is a hole, and determines the pixel points existing as holes in the target color image as target pixel points.

[0049] Next, a method for determining whether the target pixel points exist in the target color image will be described.

[0050] I d2 If (x, y) ≤ 0 is satisfied, the pixel point with coordinates (x, y) in the aforementioned target color image is determined as a target pixel point existing as a hole. Here, I d2 is the aforementioned hole-filling depth image, and I d2 (x, y) is the depth value of the reference pixel point, and the aforementioned reference pixel point is the pixel point with coordinates (x, y) in the hole-filling depth image. That is, I d2 when (x, y) ≤ 0 is satisfied, the pixel value of the image point with coordinates (x, y) in the aforementioned target color image is 0 and there is no coloring value, so the image point exists as a hole.

[0051] Hereinafter, specifically, the 3D image generating device will determine the hole-filling value of the target pixel point based on the coordinate value and depth value of each pixel point in the hole-filling depth image.

[0052] In step B1, a preset path traversed by the aforementioned reference image point is set.

[0053] In this embodiment, the preset path may be provided according to the actual situation. For example, the number of preset paths may be 16, or may be 6, or may be 5, and is not specifically limited, and may be debugged according to the generation situation of the foregoing target 3D image.

[0054] Taking the case where there are 16 preset paths as an example, the traversal method will be specifically described.

[0055] The foregoing preset path is represented by Dirs, and 16 traversal search directions are preset, where Dirs = (-1, 1), (0, 1), (1, 1), (1, 0), (-1, 2), (1, 2), (2, 1), (2, -1), (-2, 3), (-1, 3), (1, 3), (2, 3), (3, 2), (3, 1), (3, -1), (3, -2). Referring to FIG. 4 together, FIG. 4 is a schematic diagram of the preset path provided by the embodiment of the present application. The preset path Dirs shown in FIG. 4 satisfies Dirs = (-2, 3).

[0056] In step B2, based on the coordinates of each image point in the foregoing cavity filling depth image, the preset path is traversed to determine a first target pixel point and a second target pixel point that meet the preset conditions.

[0057] Next, with reference to FIG. 4, the traversal steps of each image point in the cavity filling depth image will be described. Taking each image point in the foregoing cavity filling depth image as a starting point, and setting the coordinates of each image point in the cavity filling depth image as (x, y), the traversal in each preset direction is performed according to the following steps.

[0058] In step B21, according to the following formula, I d2 Perform traversal in the negative direction until (FromX, FromY)>0 or either FromX or FromY exceeds the boundary of the cavity filling depth image. [Formula 12] JPEG2025519947000013.jpg22170

[0059] Here, [i] in Dirs[i][0] and Dirs[i][1] indicates the preset path to be traversed, [0] indicates that Dirs[i][0] takes the value on the left side of the preset path coordinate value, and [1] indicates that Dirs[i][1] takes the value on the right side of the preset path coordinate value. For example, if the preset path is Dirs = (-2, 3), and traversing in the negative direction with (-2, 3) as the preset direction (as shown in Figure 4), JPEG2025519947000014.jpg7170

[0060] In step B22, according to the following formula, I d2 Perform forward traversal until (ToX, ToY) > 0 or either ToX or ToY exceeds the boundary of the hole filling depth image. [Formula 13] JPEG2025519947000015.jpg18170

[0061] Here, [i] in Dirs[i][0] and Dirs[i][1] indicates the preset path to be traversed, [0] indicates that Dirs[i][0] takes the value on the left side of the preset path coordinate value, and [1] indicates that Dirs[i][1] takes the value on the right side of the preset path coordinate value. For example, if the preset path is Dirs = (-2, 3), and traversing in the forward direction with (-2, 3) as the preset direction (as shown in Figure 4), JPEG2025519947000016.jpg7170

[0062] In step B23, determine whether FromX, FromY, ToX, and ToY exceed the boundary of the hole filling depth image. If any one of them exceeds the boundary, [Formula 14] JPEG2025519947000017.jpg9170

[0063] If none of FromX, FromY, ToX, and ToY exceeds the boundary of the aforementioned hole-filling depth image, [Equation 15] JPEG2025519947000018.jpg14170 Here, FLOAT_MAX is the maximum floating-point value.

[0064] In step B24, after traversing all 16 preset paths, a set of FromX, FromY, ToX, and ToY with the minimum value of FltDis is determined, and (FromX, FromY) is set as the coordinates of the first target pixel point in the hole-filling depth image, and (ToX, ToY) is set as the coordinates of the second target pixel point in the hole-filling depth image.

[0065] In step B3, based on the coordinates of the first target pixel point and the coordinates of the second target pixel point, the hole-filling value of the target pixel point is determined.

[0066] JPEG2025519947000019.jpg31170

[0067] Here, I d2 (FromX, FromY) is the depth value of the first target pixel point, and I d2 (ToX, ToY)| is the depth value of the second target pixel point.

[0068] The aforementioned hole-filling value is obtained by the following formula. [Equation 16] JPEG2025519947000020.jpg11170

[0069] Here, I c1 is the first color image, (x, y) are the coordinates of the target pixel point, and I c1 (x, y) is the hole-filling value of the target pixel point, and I c1(FillX, FillY) is the pixel value of the pixel point whose coordinates in the target color image are (FillX, FillY). Equation 16 means determining the pixel value of the pixel point whose coordinates in the target color image are (FillX, FillY) as the hole filling value.

[0070] Finally, fill the target pixel point with the hole filling value to realize filling the holes in the target color image, and obtain a target color image without holes.

[0071] In step 105, a 3D image is generated based on the target color image and the initial color image.

[0072] In this embodiment, the aforementioned 3D image generation device scales the target color image and the initial color image respectively to obtain a left color image and a right color image. That is, the aforementioned 3D image generation device scales the initial color image to size (w / 2, h) to obtain a left color image, and scales the target color image to size (w / 2, h) to obtain a right color image. Then, rewrite the left color image into the left half region Rect(0, 0, w / 2, h) of the back-end buffer, and write the right color image into the right half region Rect(w / 2, 0, w / 2, h) of the back-end buffer. When it is determined that a 3D image should be output, the left half region and the right half region of the back-end buffer are interleaved to obtain and output a 3D image.

[0073] As can be seen from the above description, in the embodiment provided by the present application, remap the initial depth image for which a 3D image is to be generated to obtain a target depth image, and determine the relative displacement corresponding to the initial color image for which a 3D image is to be generated. Then, render the initial color image with the relative displacement and the target depth image to obtain a target color image, and interleave the target color image and the initial color image to generate a 3D image. Thereby, a clearer 3D image can be provided to improve the user's viewing experience.

[0074] The above described the present application from the perspective of the method for generating a 3D image. Hereinafter, the present application will be described from the perspective of the 3D image generation apparatus.

[0075] Referring to FIG. 5, FIG. 5 is a schematic diagram of the virtual structure of the 3D image generation apparatus provided by the embodiment of the present application. The 3D image generation apparatus 400 includes a reading unit 501 for reading out an initial color image and an initial depth image associated with each other from the backend buffer, a mapping unit 502 for remapping the initial depth image to obtain a target depth image, a determination unit 503 for determining the relative displacement corresponding to the initial color image, a rendering unit 504 for rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image, and a generation unit 505 for generating a 3D image from the target color image and the initial color image.

[0076] In one possible design, the determination unit 503 specifically performs the following operations. Determine the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image, which is the original image corresponding to the initial color image and the initial depth image. Determine the rendering position information based on the Y-axis rotation angle and the X-axis rotation angle. Determine the relative displacement based on the initial position information corresponding to the target original image and the rendering position information.

[0077] In one possible design, for the determination unit 503 to determine the rendering position information based on the aforementioned Y-axis rotation angle and the aforementioned X-axis rotation angle, the aforementioned rendering position information is obtained by the following formula. [Equation 17] JPEG2025519947000021.jpg20170Here, Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant.

[0078] In one possible design, the determination unit 503 determines the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image, which is The Y-axis rotation angle is determined by the following formula. [Equation 18] JPEG2025519947000022.jpg9170

[0079] Here, a is the Y-axis rotation angle, H is the distance between the human eye and the display screen, α is the angle formed by the first target projection and the positive half-axis of the Z-axis in the coordinate system. The first target projection is the projection of the target connection line onto the XOZ plane. The target connection line is the line connecting the human eye and the center of the display screen. J is the distance between the center of the scene corresponding to the target original image and the display screen. The display screen is the screen corresponding to the 3D image.

[0080] The x-axis rotation angle is determined by the following formula. [Equation 19] JPEG2025519947000023.jpg8170Here, b is the X-axis rotation angle, β is the angle formed by the second target projection and the positive half-axis of the Z-axis in the coordinate system. The second target projection is the projection of the target connection line onto the Y0Z plane, and 0 is the origin of the coordinate system.

[0081] In one possible design, specifically, the generation unit 505 scales the initial color image and the target color image respectively to obtain a left color image and a right color image, associates and writes the left color image and the right color image to the corresponding backend buffer, Upon receiving the representation instruction, it is used to generate the 3D image by interleaving the left color image and the right color image written into the backend buffer.

[0082] In one possible design, the rendering unit 504 specifically determines an initial point cloud corresponding to the target depth image, adjusts the coordinates of each point in the initial point cloud according to the relative displacement to obtain a target point cloud, processes the coordinates of each point in the target point cloud to obtain a reference image, and is used to process the pixel points in the initial color image based on the depth values of the pixel points in the reference image to obtain the target color image.

[0083] FIG. 6 is a schematic structural diagram of the server of the present application. As shown in FIG. 6, the server 600 of this embodiment includes at least one processor 601, at least one network interface 604 or other user interface 603, a memory 606, and at least one communication bus 602. This server 600 optionally includes a user interface 603 such as a display, keyboard or click device. The memory 605 can also include high-speed RAM memory, or may include non-volatile memory, for example, at least one magnetic disk memory. The memory 605 stores execution instructions. When the server 600 operates, the processor 601 communicates with the memory 605. The processor 601 calls the instructions stored in the memory 605 to execute the above 3D image generation method. The operating system 606 includes various programs and realizes various basic services and tasks according to the hardware.

[0084] The server provided by the embodiments of the present application can execute the technical proposals of the embodiments of the above 3D image generation method, and its implementation principle and technical effects are similar, so they will not be described in detail here.

[0085] For the sake of ease and brevity of description, those skilled in the art should understand that the specific operation processes of the systems, apparatuses, and units described above can refer to the corresponding processes in the embodiments of the above-described methods. Here, it will not be described any further.

[0086] As described above, the above embodiments are only used to illustrate the technical solutions of the present application and do not limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art can modify the technical solutions described in each of the above embodiments or equivalently replace some of the technical features, and these modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A 3D image generation method, comprising: reading, from a back buffer, an initial color image and an initial depth image that are associated with each other; remapping the initial depth image to obtain a target depth image; determining a relative displacement corresponding to the initial color image; rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image; generating a 3D image based on the target color image and the initial color image.

2. Determining the relative displacement corresponding to the target depth image includes: determining a Y-axis rotation angle and an X-axis rotation angle corresponding to a target original image, which is an original image corresponding to the initial color image and the initial depth image; determining rendering position information based on the Y-axis rotation angle and the X-axis rotation angle; determining the relative displacement based on initial position information corresponding to the target original image and the rendering position information.

3. Determining the rendering position information based on the Y-axis rotation angle and the X-axis rotation angle includes: obtaining the rendering position information according to the following formula: where Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant.

4. Determining the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image includes: specifying the Y-axis rotation angle according to the following formula: where α is the Y-axis rotation angle, H is the distance between the human eye and the display screen, α is the angle formed by the first target projection and the positive half-axis of the Z-axis in the coordinate system, the first target projection is the projection of the target connection line onto the XOZ plane, the target connection line is the line connecting the human eye and the center of the display screen, J is the distance between the center of the scene corresponding to the target original image and the display screen, and the display screen is the screen corresponding to the 3D image; determining the X-axis rotation angle according to the following formula: Here, b is the X-axis rotation angle, β is the angle formed by the second target projection and the positive half-axis of the Z-axis in the coordinate system, the second target projection is the projection of the target connection line onto the Y0Z plane, and 0 is the origin of the coordinate system. The 3D image generation method according to claim 2.

5. The step of rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image includes: Determining an initial point cloud corresponding to the target depth image; Adjusting the coordinates of each point in the initial point cloud by the relative displacement to obtain a target point cloud; Processing the coordinates of each point in the target point cloud to obtain a reference image; Processing the pixel points in the initial color image based on the depth values of the pixel points in the reference image to obtain the target color image. The 3D image generation method according to any one of claims 1 to 4.

6. The step of generating a 3D image based on the target color image and the initial color image includes: Scaling the initial color image and the target color image respectively to obtain a left color image and a right color image; Associating and writing the left color image and the right color image to the backend buffer; When receiving a display command, interleaving the left color image and the right color image written in the backend buffer to generate the 3D image. The 3D image generation method according to any one of claims 1 to 4.

7. A 3D image generation device, comprising: A reading unit for reading an initial color image and an initial depth image associated with each other from a backend buffer; A mapping unit for remapping the initial depth image to obtain a target depth image; A determination unit for determining a relative displacement corresponding to the initial color image; A rendering unit for rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image; A generation unit for generating a 3D image based on the target color image and the initial color image. The 3D image generation device is characterized by including the above units.

8. Specifically, the determination unit: Determines the Y-axis rotation angle and the X-axis rotation angle corresponding to a target original image, which is the original image corresponding to the initial color image and the initial depth image. determine rendering position information based on the Y-axis rotation angle and the X-axis rotation angle; The 3D image generation device according to claim 7, which is used to determine the relative displacement based on the initial position information corresponding to the target original image and the rendering position information. **Claim 9** The determination unit further: determine the rendering position information according to the following formula: where Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant. The 3D image generation device according to claim 8. **Claim 10** A computer device, comprising: at least one processor, a memory, and a transceiver connected to each other; the memory is used to store program codes; the processor calls the program codes stored in the memory to execute the steps of the 3D image generation method according to any one of claims 1 to 6.

Images