Method, apparatus, and computer device for generating 3D images

Abstract

This application provides a method for generating a 3D image and a related device that can obtain a clearer 3D image and improve the user's viewing experience. This method includes the steps of reading an initial color image and an initial depth image associated with each other from a back buffer, remapping the initial depth image to obtain a target depth image, determining a relative displacement corresponding to the initial color image, rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image, and generating a 3D image based on the target color image and the initial color image.

Description

[Technical Field]

[0001] This application relates to the field of image processing, and more particularly to methods, apparatus, and computer devices for generating 3D images. [Background technology]

[0002] With the advancement of science and technology, products incorporating 3D display technology are commonly used in people's daily lives. 3D images displayed using 3D technology possess a strong visual impact, providing consumers with a sense of realism.

[0003] Currently, products with 3D display technology typically convert 2D images into 3D images using a view conversion method for stereoscopic display. However, 3D images obtained by converting 2D images using this method often lack a complete 3D effect, frequently resulting in blurring and degrading the user's viewing experience. [Overview of the project]

[0004] This invention provides a method, apparatus, and computer apparatus for generating 3D images, enabling the acquisition of clearer 3D images and improving the user's viewing experience.

[0005] The method for generating a 3D image relating to the first aspect of this application is: The steps include reading the initial color image and initial depth image, which are associated with each other, from the rear buffer, The steps include: remapping the initial depth image to obtain a target depth image; The steps include determining the relative displacement corresponding to the initial color image, The steps include rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image, The process includes the step of generating a 3D image based on the target color image and the initial color image.

[0006] The 3D image generation device relating to the second aspect of this application is A readout unit for reading the initial color image and initial depth image, which are associated with each other, from the rear end buffer, A mapping unit that remaps the initial depth image to obtain a target depth image, A determination unit for determining the relative displacement corresponding to the initial color image, A rendering unit for rendering the initial color image to obtain a target color image based on the relative displacement and the target depth image, The system includes a generation unit for generating a 3D image based on the target color image and the initial color image.

[0007] A computer device relating to the third aspect of the present application comprises at least one connected processor, memory, and transceiver, wherein the memory is used to store program code, and the processor calls the program code in the memory to perform the steps of the 3D image generation method described in the first aspect.

[0008] Based on the above, in the embodiment provided by this application, an initial depth image from which a 3D image is to be generated is remapped to obtain a target depth image, and the relative displacement corresponding to the initial color image from which a 3D image is to be generated is determined. Then, the initial color image is rendered based on the relative displacement and the target depth image to obtain a target color image. Then, by interleaving the target color image and the initial color image to generate a 3D image, a better and sharper 3D image can be obtained, improving the user's viewing experience. [Brief explanation of the drawing]

[0009] [Figure 1] This is a flowchart of a method for generating 3D images according to an embodiment of the present invention. [Figure 2] This is a schematic diagram showing the angle between the human eye and the screen according to an embodiment of the present invention. [Figure 3]It is a schematic diagram showing the relative displacement of a camera when an initial color image according to an embodiment of the present invention is rendered. [Figure 4] It is a schematic diagram of a preset path according to an embodiment of the present application. [Figure 5] It is a schematic diagram of a virtual structure of a 3D image generation device according to an embodiment of the present invention. [Figure 6] It is a schematic diagram of the hardware configuration of a server according to an embodiment of the present application.

Mode for Carrying Out the Invention

[0010] Hereinafter, in accordance with the drawings of the embodiments of the present application, the technical aspects in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments.

[0011] Hereinafter, a method for generating a 3D image will be described from the perspective of a 3D image generation device. The 3D image generation device may be a server or a service unit in the server, and is not specifically limited.

[0012] In connection with referring to FIG. 1, FIG. 1 is a flowchart of a method for generating a 3D image provided in an embodiment of the present application, and includes the following steps.

[0013] In step 101, an initial color image and an initial depth image are read from a backend buffer.

[0014] In this embodiment, modern 3D rendering frameworks (opengl, metal, vulkan, dirextx) employ dual / multi-buffer technology. That is, the image currently being rendered is not displayed on the screen in real time. Of course, the rendered content is stored in a backend buffer (Backend Cache), and in order to display the rendered content on the screen, the system must wait for the manager to exchange the front and rear buffers. Therefore, if an exchange is imminent, the color image and depth image from the backend buffer are read and re-rendered to generate images from other viewpoints, and the newly generated multi-view images are exchanged with the front end buffer as backend content. This allows the 3D image generator to read the initial color image and initial depth image to be displayed in 3D from the backend buffer. The initial color image is associated with the initial depth image. That is, both the initial color image and the initial depth image are converted from a target original image. The target original image is an RGBD image and includes two images: one is a normal RGB 3-channel color image containing graphics color information, and the other is a depth image containing depth information. The initial color image is an RGB 3-channel color image and contains multiple pixel points. Each pixel point is represented by coordinate values ​​(x,y) and has a pixel value. The aforementioned pixel value is RGB color information. The initial depth image is a depth image and contains multiple pixel points. The pixel points of the initial depth image have coordinates corresponding to the target depth image. Each pixel point of the initial depth image has a depth value representing depth information.

[0015] In step 102, the initial depth image is remapped to obtain the target depth image.

[0016] In this embodiment, the 3D image generation device reads the initial depth image to be displayed in 3D from the rear end buffer, and then remaps the initial depth image to obtain the target depth image. Specifically, the initial depth image can be remapped to the target depth image using the following formula. [Formula 1] JPEG0007887509000001.jpg9170

[0017] Here, I d This is the initial depth image, I d1 This is the target depth image, and I d(z) is the depth value of any pixel point in the initial depth image, and I d 1(z) The coordinates in the aforementioned target depth image are I d(z) Dmax is the depth value of a pixel point with the same coordinates as the initial depth image, Dmax is the maximum depth value for all pixel points in the initial depth image, and MaxDepth is an empirical value that can be 100. Of course, other values ​​such as 80, 90, 110, etc., are also acceptable and are not specifically limited as long as they do not exceed the maximum floating-point value. The coordinates of each pixel point in the remapped target depth image correspond one-to-one with the coordinates of each pixel point in the initial depth image. Moreover, the depth values ​​corresponding to the initial depth image are different from the depth values ​​of the target depth image.

[0018] In step 103, the relative displacement corresponding to the initial color image is determined.

[0019] In this embodiment, the 3D image generation device can determine the relative displacement corresponding to the initial color image. Specifically, the 3D image generation device can determine the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image. The target original image is the original image corresponding to the initial color image and the initial depth image. Based on the Y-axis rotation angle and the X-axis rotation angle, rendering position information is determined. Then, the relative displacement is determined based on the initial position information and the rendering position information.

[0020] Next, we will explain in detail how the relative displacement is determined, referring to Figure 2. Please refer to Figure 2. Figure 2 is a schematic diagram of the position of the human eye and the angle of the screen provided in the embodiment of the present invention. As shown in Figure 2, 201 is the position of the human eye, and in a space where the origin of the O-XYZ three-dimensional coordinate system is the center of the display screen (the screen center is the display screen corresponding to the 3D image), the angle between the projection of the connecting line from the human eye to the center of the screen onto the XOZ plane and the positive half-axis of the Z axis is α, and the angle between the projection of the connecting line from the human eye to the center of the screen onto the YOZ plane and the positive half-axis of the Z axis is β. Here, the X axis is in the same direction as the left-right direction of the display screen, and the positive direction of the X axis is from the left center of the display screen to the right center of the display screen, and the Y axis is in the same direction as the up-down direction of the display screen, and the positive direction of the Y axis is from the upper midpoint of the display screen to the lower midpoint of the display screen. Based on angles α and β, the distance H between the human eye and the display screen, and the distance J between the center of the scene corresponding to the target original image and the display screen, the angle a (i.e., the Y-axis rotation angle corresponding to the target original image) and the angle b (i.e., the X-axis rotation angle corresponding to the target original image) of the scene corresponding to the target original image rotating around the Y axis can be calculated. Specifically, the angle a of rotation of the scene corresponding to the target original image around the Y axis is calculated by the following formula. [Formula 2] JPEG0007887509000002.jpg7170

[0021] Here, a is the rotation angle along the Y axis, H is the distance between the human eye and the display screen, and α is the angle between the first target projection and the semi-zero axis of the Z axis in the coordinate system. The first target projection is the projection of the target connection line onto the XOZ plane. The target connection line is the connection line between the human eye and the center of the display screen, J is the distance between the center of the scene corresponding to the target original image and the display screen, and the display screen is the screen corresponding to the 3D image.

[0022] The angle b by which the scene corresponding to the target original image rotates around the X-axis is calculated by the following formula: [Formula 3] JPEG0007887509000003.jpg8170

[0023] Here, b is the rotation angle along the X axis, β is the angle between the second target projection and the semi-axis of the Z axis in the coordinate system, the second target projection is the projection of the target connection onto the YZ plane, and 0 is the origin of the coordinate system.

[0024] The 3D image display device can determine the rendering position information based on the Y-axis rotation angle and the X-axis rotation angle corresponding to the target original image, after determining the Y-axis rotation angle and the X-axis rotation angle. Specifically, the rendering position information is determined by the following formula. [Formula 4] JPEG0007887509000004.jpg17170

[0025] Here, Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant. Rd may take the value of 100, or the value of 1000, or it may be set according to the actual situation, and is not specifically limited.

[0026] As a result, the 3D image display device can calculate the relative displacement based on the rendering position information and the initial position information corresponding to the target original image after determining the rendering position information. Refer to Figure 3. Figure 3 is a schematic diagram of the relative displacement according to an embodiment of the present invention. Here, the position of camera 301 is C0, and C0 is the initial position information of the target original image. If the space in which camera 301 is located is defined as a three-dimensional space defined by the x, y, and z axes, and the coordinates of C0 are (0,0,z), then by changing the position of camera 301, different rendering images are obtained. In order to match the sizes of the different rendering images, camera 301 does not change in the z axis. The position of camera 301 after the change is C1. That is, C1 is the rendering position information described above, and the coordinates of C1 are (nx,ny,z). The change in the position of camera 301 is the relative displacement described above. The relative displacement of camera 301 described above is determined as shown in the following equation. [Formula 5] JPEG0007887509000005.jpg9170

[0027] In step 104, the initial color image is rendered based on the relative displacement and the target depth image to obtain the target color image.

[0028] In this embodiment, the 3D image generation device determines the relative displacement, then renders the initial color image based on the relative displacement and the target depth image to obtain the target color image. Next, the method of rendering to obtain the target color image will be described in detail:

[0029] In step A1, the initial point cloud corresponding to the first depth image is determined.

[0030] In this step, the 3D image generation device converts the target depth image into the initial point cloud using the following formula. [Formula 6] JPEG0007887509000006.jpg11170

[0031] Here, 0 ≤ x < w, 0 ≤ y < h, P0 is the initial point cloud, P0(x, y, z) is the coordinate of any one point in the initial point cloud, w is the width of the target depth image, h is the height of the target depth image, and I d1 (x, y) is the depth value of the pixel point with coordinates (x, y) in the target depth image.

[0032] In step A2, the coordinates of each point in the initial point cloud are adjusted by the relative displacement to obtain a target point cloud.

[0033] In this step, the relative displacement is added to each point of the initial point cloud to obtain the aforementioned target point cloud. Specifically, it is calculated by the following formula. [Equation 7] JPEG0007887509000007.jpg10170

[0034] Here, P0(x, y, z) is the coordinate of any one point in the initial point cloud, P1 is the aforementioned target point cloud, and P1(x, y, z) is the coordinate of the point obtained by adding D to P0(x, y, z) among the aforementioned target point cloud. Since the position of the camera does not change on the z-axis and the D value is (nx, ny, 0), the z values of the coordinates of the aforementioned target point cloud and the aforementioned initial point cloud are the same.

[0035] In step A3, the coordinates of each point in the target point cloud are processed to obtain a reference image.

[0036] In this step, the aforementioned reference image is a depth image that matches the size of the initial depth image. The initial substitution value of the depth value of the aforementioned reference image is A. The coordinates of each point in the target point cloud are processed by the following formula to obtain the aforementioned reference image. [Equation 8] JPEG0007887509000008.jpg9170

[0037] Here, Z is the aforementioned reference image. Z(x, y) is the depth value of the pixel point whose coordinates in the aforementioned reference image are (x, y). Z(IP.x + 1, IP.y + 1) is the depth value of the pixel point whose coordinates in the aforementioned reference image are (IP.x + 1, IP.y + 1), and FltErr = A - w / 2 / z0, min is the value obtained by assigning the smaller value of both Z(IP.x + 1, IP.y + 1) and FltErr to Z(x, y).

[0038] [Equation 9] JPEG0007887509000009.jpg24170A is the initial depth value of the aforementioned reference depth map, w is the width of the target depth image, (x0, y0, z0) are the coordinates of any one point among the target point cloud, IP.x is the x value of the point IP coordinates, and IP.y is the y value of the point IP coordinates.

[0039] Here, the reference image is a depth image that matches the size of the target depth image. The initial depth value of the reference image is A. The value of A may be 100000.0, or 90000.0, or 110000.0, as long as the value of A is greater than the depth value of the target depth image and less than the maximum floating-point value, and it is not specifically limited.

[0040] In step D, based on the depth values of the pixel points in the reference image, the pixel points in the initial color image are processed to obtain the target color image.

[0041] In this step, the target color image is determined by the following formula. [Equation 10] JPEG0007887509000010.jpg10170

[0042] Here, I c is the initial color image, I c1 is the first color image, IP.x is the x value of the point IP coordinates, and IP.y is the y value of the point IP coordinates. I c(x,y) is the pixel value of the pixel point with coordinates (x,y) in the initial color image, and Z(x,y) is the depth value of the pixel point with coordinates (x,y) in the reference image, I c1 (IP.x, IP.y) is the pixel value of the pixel point at coordinates (IP.x, IP.y) in the target color image.

[0043] When (Z(x,y)+1)>FltErr is satisfied, if the value of ((Z(x,y)+1)>FltErr) is set to 1, the pixel value of the pixel point with coordinates (x,y) in the initial depth image is assigned to the pixel point with coordinates (IP.x,IP.y) in the target color image.

[0044] If (Z(x,y)+1)>FltErr is not satisfied, then if the value of ((Z(x,y)+1)>FltErr) is set to 0, the pixel value of the pixel point with coordinates (IP.x,IP.y) in the aforementioned target color image is 0.

[0045] Furthermore, the 3D image generation device can determine whether or not a cavity exists within the target color image after determining the target color image. This will be explained in detail below.

[0046] First, the initial depth image is processed using the following formula to obtain a cavity-filled depth image. [Formula 11] JPEG0007887509000011.jpg8170

[0047] Here, I d This is the initial depth image, I d2 This is the target depth image, and I d (x,y) is the depth value of the pixel point with coordinates (x,y) in the initial depth image, I d2(IP.x,IP.y) is the depth value of the pixel point with coordinates (IP.x,IP.y) in the cavity-filled depth image. If the value of ((Z(x,y)+1)>FltErr) is 1, the depth value of the pixel point with coordinates (x,y) in the initial depth image is assigned to the pixel point with coordinates (IP.x,IP.y) in the cavity-filled depth image, and if the value of ((Z(x,y)+1)>FltErr) is 0, the depth value of the pixel point with coordinates (IP.x,IP.y) in the cavity-filled depth image is 0.

[0048] Subsequently, the 3D image generation device determines, based on the depth value of each pixel point in the cavity-filled depth image, whether or not a pixel point in the target color image has the same coordinates as each pixel point in the cavity-filled depth image is a cavity, and determines the pixel point that exists as a cavity in the target color image as the target pixel point.

[0049] Next, a method for determining whether or not the target pixel point exists in the target color image will be described.

[0050] I d2 If (x,y)≦0 is satisfied, then the pixel point with coordinates (x,y) in the aforementioned target color image is determined to be a target pixel point that exists as a void. Here, I d2 This is the cavity filling depth image mentioned above, I d2 (x,y) is the depth value of the reference pixel point, and the aforementioned reference pixel point is the pixel point with coordinates (x,y) in the cavity-filled depth image. That is, I d2 When (x,y)≦0 is satisfied, the pixel value of the image point at coordinates (x,y) in the aforementioned target color image is 0, and since there is no color value, that image point exists as a void.

[0051] The following will specifically explain how the aforementioned 3D image generation device determines the cavity-filling value of the target pixel point based on the coordinate values ​​and depth values ​​of each pixel point in the cavity-filling depth image.

[0052] In step B1, a preset path is set for the aforementioned reference image points to traverse.

[0053] In this embodiment, the preset paths may be provided according to the actual situation, for example, there may be 16, 6, or 5 preset paths, and are not specifically limited, and may be debugged according to the generation status of the target 3D image as described above.

[0054] Using the example of a case with 16 preset paths, we will specifically describe the traverse method.

[0055] The aforementioned preset paths are represented by Dirs, with 16 traverse search directions pre-set, and are Dirs = (-1,1), (0,1), (1,1), (1,0), (-1,2), (1,2), (2,1), (2,-1), (-2,3), (-1,3), (1,3), (2,3), (3,2), (3,1), (3,-1), (3,-2). Referring also to Figure 4, which is a schematic diagram of the preset paths provided by the embodiment of the present application. The preset path Dirs shown in Figure 4 satisfies Dirs = (-2,3).

[0056] In step B2, based on the coordinates of each image point in the cavity depth image described above, the preset path is traversed to determine the first target pixel point and the second target pixel point that match the pre-set conditions.

[0057] Next, the traverse steps for each image point in the cavity depth image will be explained in accordance with Figure 4. Starting from each image point in the cavity depth image, and with the coordinates of each image point in the cavity depth image being (x,y), traverse in each preset direction is performed according to the following steps.

[0058] In step B21, the following formula is used: d2 Perform a negative traverse until (FromX,FromY) > 0 or either FromX or FromY crosses the boundary of the cavity-filled depth image. [Formula 12] JPEG0007887509000012.jpg22170

[0059] Here, in Dirs[i][0] and Dirs[i][1], [i] indicates the traversing preset path, [0] indicates that Dirs[i][0] takes the value of the left-hand side in the preset path coordinate values, and [1] indicates that Dirs[i][1] takes the value of the right-hand side in the preset path coordinate values. For example, if the preset path is Dirs=(-2,3) and we traverse in the negative direction with (-2,3) as the preset direction (as shown in Figure 4), JPEG0007887509000013.jpg7170

[0060] In step B22, the following equation is used: d2 Perform a forward traverse until (ToX,ToY) > 0 or either ToX or ToY exceeds the boundary of the cavity-filled depth image. [Formula 13] JPEG0007887509000014.jpg18170

[0061] Here, in Dirs[i][0] and Dirs[i][1], [i] indicates the traversing preset path, [0] indicates that Dirs[i][0] takes the value of the left-hand side in the preset path coordinates, and [1] indicates that Dirs[i][1] takes the value of the right-hand side in the preset path coordinates. For example, if the preset path is Dirs=(-2,3) and we traverse it in the forward direction with (-2,3) as the preset direction (as shown in Figure 4), JPEG0007887509000015.jpg7170

[0062] In step B23, it is determined whether FromX, FromY, ToX, and ToY have crossed the boundary of the cavity-filled depth image. If any one of them has crossed the boundary, [Formula 14] JPEG0007887509000016.jpg9170

[0063] If none of FromX, FromY, ToX, or ToY exceeds the boundary of the aforementioned cavity-filled depth image, [Formula 15] JPEG0007887509000017.jpg14170 Here, FLOAT_MAX is the maximum floating-point value.

[0064] In step B24, after traversing all 16 preset paths, a set of FromX, FromY, ToX, and ToY that minimizes the FltDis value is determined, and (FromX, FromY) is set as the coordinates of the first target pixel point in the cavity depth-of-situ image, and (ToX, ToY) is set as the coordinates of the second target pixel point in the cavity depth-of-situ image.

[0065] In step B3, the cavity compensation value for the target pixel point is determined based on the coordinates of the first target pixel point and the coordinates of the second target pixel point.

[0066] JPEG0007887509000018.jpg31170

[0067] Here, I d2 (FromX, FromY) is the depth value of the first target pixel point, I d2 (ToX,ToY)| represents the depth value of the second target pixel point.

[0068] The aforementioned cavity filling value can be calculated using the following formula. [Formula 16] JPEG0007887509000019.jpg11170

[0069] Here, I c1 is the first color image, (x,y) is the coordinates of the target pixel point, and I c1 (x,y) is the cavity compensation value of the target pixel point, I c1(FillX, FillY) is the pixel value of the pixel point in the target color image whose coordinates are (FillX, FillY). Equation 16 means that the pixel value of the pixel point in the target color image whose coordinates are (FillX, FillY) is determined as the cavity fill value.

[0070] Finally, the cavity compensation value is added to the target pixel point to fill in the voids in the target color image and obtain a target color image without voids.

[0071] In step 105, a 3D image is generated based on the target color image and the initial color image.

[0072] In this embodiment, the aforementioned 3D image generation device scales the target color image and the initial color image, respectively, to obtain the left color image and the right color image. That is, the aforementioned 3D image generation device scales the initial color image to size (w / 2, h) to obtain the left color image, and scales the target color image to size (w / 2, h) to obtain the right color image. Subsequently, the left color image is written again to the left half region Rect(0,0,w / 2,h) of the rear end buffer, and the right color image is written to the right half region Rect(w / 2,0,w / 2,h) of the rear end buffer. When it is decided that a 3D image should be output, the left half region and the right half region of the rear end buffer are interleaved to obtain and output the 3D image.

[0073] As can be seen from the above description, in the embodiment provided by this application, an initial depth image from which a 3D image is to be generated is remapped to obtain a target depth image, and the relative displacement corresponding to the initial color image from which the 3D image is to be generated is determined. Then, the initial color image is rendered using the relative displacement and the target depth image to obtain a target color image, and the target color image and the initial color image are interleaved to generate a 3D image. This makes it possible to provide a clearer 3D image and improve the user's viewing experience.

[0074] The present invention has been described above from the perspective of a 3D image generation method. Below, the present invention will be described from the perspective of a 3D image generation apparatus.

[0075] Refer to Figure 5. Figure 5 is a schematic diagram of the virtual structure of a 3D image generation device provided by an embodiment of the present application. The 3D image generation device 400 is A readout unit 501 for reading the initial color image and initial depth image, which are associated with each other, from the rear end buffer, A mapping unit 502 remaps the initial depth image to obtain a target depth image, A determination unit 503 for determining the relative displacement corresponding to the initial color image, A rendering unit 504 for rendering the initial color image to obtain a target color image based on the relative displacement and the target depth image, The system includes a generation unit 505 for generating a 3D image from the target color image and the initial color image.

[0076] In one possible design, the decision unit 503 specifically performs the following operations: Determine the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image, which is the original image corresponding to the initial color image and initial depth image. Rendering position information is determined based on the Y-axis rotation angle and the X-axis rotation angle. The relative displacement is determined based on the initial position information corresponding to the target original image and the rendering position information.

[0077] In one possible design, the determination unit 503 determines rendering position information based on the aforementioned Y-axis rotation angle and the aforementioned X-axis rotation angle. The rendering position information mentioned above is obtained using the following formula. [Formula 17] JPEG0007887509000020.jpg20170 Here, Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant.

[0078] In one possible design, the decision unit 503 determines the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image. The Y-axis rotation angle is determined by the following formula. [Formula 18] JPEG0007887509000021.jpg9170

[0079] Here, a is the rotation angle along the Y axis, H is the distance between the human eye and the display screen, and α is the angle between the first target projection and the semi-zero axis of the Z axis in the coordinate system. The first target projection is the projection of the target connection onto the XOZ plane. The target connection is the line connecting the human eye and the center of the display screen. J is the distance between the center of the scene corresponding to the target original image and the display screen. The display screen is the screen corresponding to the 3D image.

[0080] The x-axis rotation angle is determined by the following formula. [Formula 19] JPEG0007887509000022.jpg8170 Here, b is the rotation angle along the X axis, β is the angle between the second target projection and the semi-zero axis of the Z axis in the coordinate system, the second target projection is the projection of the target connection onto the Y0Z plane, and 0 is the origin of the coordinate system.

[0081] In one possible design, the generating unit 505 is specifically, The initial color image and the target color image are scaled, respectively, to obtain the left color image and the right color image. The left color image and the right color image are written to the rear end buffer in association with each other. When a display command is received, the left color image and the right color image written to the rear end buffer are interleaved and used to generate the 3D image.

[0082] In one possible design, the rendering unit 504 specifically, An initial point cloud corresponding to the aforementioned target depth image is determined, The coordinates of each point in the initial point cloud are adjusted by the relative displacement to obtain the target point cloud. The coordinates of each point in the aforementioned target point cloud are processed to obtain a reference image. This is used to process the pixel points in the initial color image based on the depth values ​​of the pixel points in the reference image in order to obtain the target color image.

[0083] Figure 6 is a schematic diagram of the structure of the server of the present invention. As shown in Figure 6, the server 600 of this embodiment includes at least one processor 601, at least one network interface 604 or other user interface 603, memory 606, and at least one communication bus 602. The server 600 optionally includes a user interface 603 such as a display, keyboard, or click device. The memory 605 may include high-speed RAM memory or non-volatile memory, such as at least one magnetic disk memory. The memory 605 stores execution commands. When the server 600 is operating, the processor 601 and the memory 605 communicate. The processor 601 calls the commands stored in the memory 605 to execute the 3D image generation method described above. The operating system 606 includes various programs and implements various basic operations and hardware-specific tasks.

[0084] The server provided in the embodiment of this application can implement the technical proposal of the embodiment of the 3D image generation method described above, and its implementation principle and technical effects are similar, so no further details are provided here.

[0085] For the sake of ease and conciseness of explanation, those skilled in the art should understand that the specific operating processes of the systems, apparatus, and units described above can be understood by referring to the corresponding processes in the embodiments of the methods described above. No further explanation is provided here.

[0086] As described above, the above embodiments are used solely to illustrate the technical proposal of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art can modify the technical proposals described in the above embodiments or replace some of the technical features with equivalent ones, and such modifications or replacements will not cause the essence of the corresponding technical proposal to deviate from the spirit and scope of the technical proposal of each embodiment of the present application.

Claims

1. A method for generating 3D images, The steps include reading the initial color image and initial depth image, which are associated with each other, from the rear buffer, The steps include: remapping the initial depth image to obtain a target depth image; The steps include determining the relative displacement corresponding to the initial color image, The steps include rendering the initial color image based on the relative displacement and the target depth image to obtain a target color image, The step of generating a 3D image based on the target color image and the initial color image includes, Determining the relative displacement corresponding to the aforementioned target depth image is: The steps include determining the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image, which is the original image corresponding to the initial color image and initial depth image, The steps include determining rendering position information based on the Y-axis rotation angle and the X-axis rotation angle, A 3D image generation method characterized by including the step of determining the relative displacement based on initial position information corresponding to the target original image and the rendering position information.

2. Determining rendering position information based on the Y-axis rotation angle and the X-axis rotation angle is, The rendering position information is obtained using the following formula: The 3D image generation method according to claim 1, wherein Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant.

3. The step of determining the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image includes the following: The Y-axis rotation angle is determined by the following formula: Here, α is the rotation angle along the Y axis, H is the distance between the human eye and the display screen, α is the angle between the first target projection and the semi-zero axis of the Z axis in the coordinate system, the first target projection is the projection of the target connection line onto the XOZ plane, the target connection line is the line connecting the human eye and the center of the display screen, J is the distance between the center of the scene corresponding to the target original image and the display screen, and the display screen is the screen corresponding to the 3D image. The X-axis rotation angle is determined by the following formula: The 3D image generation method according to claim 1, wherein b is the X-axis rotation angle, β is the angle between the second target projection and the positive half-axis of the Z axis in the coordinate system, the second target projection is the projection of the target connection onto the Y0Z plane, and 0 is the origin of the coordinate system.

4. The step of rendering the initial color image based on the relative displacement and the target depth image to obtain the target color image is: The steps include determining an initial point cloud corresponding to the aforementioned target depth image, The steps include adjusting the coordinates of each point in the initial point cloud by the relative displacement to obtain a target point cloud, The steps include: processing the coordinates of each point in the aforementioned target point cloud to obtain a reference image; A 3D image generation method according to claim 1, comprising the step of processing pixel points in an initial color image based on the depth values ​​of pixel points in a reference image to obtain a target color image.

5. The step of generating a 3D image based on the target color image and the initial color image is: The steps include scaling the initial color image and the target color image to obtain the left color image and the right color image, The steps include writing the left color image and the right color image to the trailing buffer in association with each other, A method for generating a 3D image according to claim 1, comprising the step of receiving a display command, interleaving the left color image and the right color image written to the rear end buffer to generate the 3D image.

6. A 3D image generation device, A readout unit for reading the initial color image and initial depth image, which are associated with each other, from the rear end buffer, A mapping unit that remaps the initial depth image to obtain a target depth image, A determination unit for determining the relative displacement corresponding to the initial color image, A rendering unit for rendering the initial color image to obtain a target color image based on the relative displacement and the target depth image, Includes a generation unit for generating a 3D image based on the target color image and the initial color image, The aforementioned decision unit specifically, Determine the Y-axis rotation angle and X-axis rotation angle corresponding to the target original image, which is the original image corresponding to the initial color image and the initial depth image. Based on the Y-axis rotation angle and the X-axis rotation angle, the rendering position information is determined. A 3D image generation apparatus characterized by being used to determine the relative displacement based on initial position information corresponding to the target original image and the rendering position information.

7. The aforementioned decision unit further, The rendering position information is determined by the following formula: The 3D image generation apparatus according to claim 6, wherein Dx is the X-axis coordinate value corresponding to the rendering position information, Dy is the Y-axis coordinate value corresponding to the rendering position information, a is the Y-axis rotation angle, b is the X-axis rotation angle, and Rd is a preset constant.

8. A computer device, It comprises at least one processor, memory, and transceiver connected to each other, The aforementioned memory is used to store program code. The processor is a computer device that calls program code stored in the memory to perform the steps of the 3D image generation method according to any one of claims 1 to 5.

Images