Three-dimensional image generation methods, apparatus, devices, storage media and program products

Abstract

This invention provides a method, apparatus, device, storage medium, and program product for generating three-dimensional images, applied in the field of image processing technology. The method includes: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, obtaining (M-1) disparity vectors, wherein the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in a target dimension; determining a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and an interpolation number determined by user input, wherein the three-dimensional light field encoded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, storage medium, and program product for generating three-dimensional images. Background Technology

[0002] With the rapid development of 3D stereoscopic display technology, people can have an immersive visual experience.

[0003] In existing technologies, in order to generate 3D images, a disparity map sequence can be read or generated first, and then the disparity map sequence can be pixel sampled and rearranged according to the hardware parameters and display principles of the 3D display to obtain a 3D image that can be used for 3D display.

[0004] However, since several dense viewpoint images need to be pre-rendered according to the 3D display's viewing angle and number of viewpoints before rendering 3D images, the rendering efficiency is low, the generation rate is slow, and the transmission bandwidth or storage bandwidth is high, which cannot meet the demand of 3D light field display for fast and high-resolution images. Summary of the Invention

[0005] This invention provides a three-dimensional image generation method, apparatus, device, storage medium, and program product to solve the problems of existing 3D image generation methods, which not only have low rendering efficiency and slow generation rate, but also high transmission or storage bandwidth, failing to meet the demand for fast and high-resolution images in 3D light field displays.

[0006] This invention provides a method for generating a three-dimensional image, comprising: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and an interpolation number determined by user input, wherein the three-dimensional light field encoded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0007] According to a three-dimensional image generation method provided by the present invention, the step of determining a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input includes:

[0008] N three-dimensional light field coded images are determined based on the M sparse primitive images and the number of interpolations determined by user input;

[0009] ;

[0010] in, This indicates the number of interpolations.

[0011] According to a three-dimensional image generation method provided by the present invention, the step of determining N three-dimensional light field coded images based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input includes:

[0012] Based on the encoding rules of the three-dimensional light field encoded image and the parameters of the light control device, the image number n of each sub-pixel in the three-dimensional light field encoded image in the dense primitive image is determined;

[0013] The sequence number of the sparse primitive image is determined based on the image sequence number n and the number M of sparse primitive images. ;

[0014] Based on the image sequence number n and the sequence number of the sparse primitive image and the number of interpolations Determine the interpolation interval of the sparse primitive image ;

[0015] According to the formula Determine the position coordinates of the 3D light field encoded image;

[0016] in, Represents the position coordinates of a 3D light field encoded image; This represents the `code` function; Indicates the sequence number is sparse primitive images; Let represent the (M-1)th disparity vector.

[0017] According to a three-dimensional image generation method provided by the present invention, the target dimension includes at least one of the x-dimensional and y-dimensional dimensions.

[0018] The present invention also provides a three-dimensional image generation device, comprising the following modules: an acquisition module and a processing module;

[0019] The acquisition module is used to acquire M sparse primitive images;

[0020] The processing module is used to determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, to obtain (M-1) disparity vectors, which are used to indicate the displacement of corresponding pixels in the target dimension in the two images; and to determine a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by the user input, which is used to generate a three-dimensional image;

[0021] Where M is an integer greater than 1.

[0022] According to a three-dimensional image generation apparatus provided by the present invention, the processing module is used to determine N three-dimensional light field coded images based on the M sparse primitive images and the number of interpolations determined by user input; ; in, This indicates the number of interpolations.

[0023] According to a three-dimensional image generation apparatus provided by the present invention, the processing module is configured to determine the image index n of each sub-pixel in the three-dimensional light field encoded image within a dense primitive image based on the encoding rules of the three-dimensional light field encoded image and the parameters of the light control device; and to determine the index of the sparse primitive image based on the image index n and the number M of sparse primitive images. According to the image sequence number n and the sequence number of the sparse primitive image. and the number of interpolations Determine the interpolation interval of the sparse primitive image According to the formula Determine the position coordinates of the 3D light field encoded image; where, Represents the position coordinates of a 3D light field encoded image; This represents the `code` function; Indicates the sequence number is sparse primitive images; Let represent the (M-1)th disparity vector.

[0024] According to the present invention, a three-dimensional image generation apparatus is provided, wherein the target dimension includes at least one of the x-dimensional and y-dimensional dimensions.

[0025] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the three-dimensional image generation methods described above.

[0026] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the three-dimensional image generation method as described above.

[0027] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the three-dimensional image generation methods described above.

[0028] The present invention provides a three-dimensional image generation method, apparatus, device, storage medium, and program product, which can acquire M sparse primitive images; determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in the target dimension; and determine a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input, wherein the three-dimensional light field encoded image is used to generate a three-dimensional image; wherein M is an integer greater than 1. Through this scheme, a three-dimensional light field encoded image can be determined based on M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input. Since the three-dimensional light field encoded image can be filled by reusing sparse primitive images, the process of rendering numerous dense intermediate views can be avoided, thereby improving rendering efficiency and reducing the three-dimensional image generation time. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the three-dimensional image generation method provided by the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of the three-dimensional image generation device provided by the present invention;

[0032] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0036] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0037] This application describes some exemplary embodiments for illustrative purposes. It should be understood that this application may be implemented in other ways not specifically shown in the accompanying drawings.

[0038] like Figure 1 As shown, this application provides a method for generating a three-dimensional image, which can be applied to a three-dimensional image generating apparatus. The method may include steps S101-S103:

[0039] S101, The three-dimensional image generation device acquires M sparse primitive images.

[0040] Where M is an integer greater than 1.

[0041] Optionally, the above M sparse primitive images can be represented as .

[0042] S102. The three-dimensional image generation device determines the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, and obtains (M-1) disparity vectors.

[0043] The disparity vector mentioned above is used to indicate the displacement of corresponding pixels in the target dimension in two images.

[0044] Specifically, the 3D image generation device can determine the sparse primitive image. With sparse primitive images disparity vectors between Determine the sparse primitive image With sparse primitive images disparity vectors between This process continues until the sparse primitive image is determined. With sparse primitive images disparity vectors between This yields (M-1) disparity vectors, which can be represented as... .

[0045] Optionally, the target dimension may include at least one of the x-dimension, y-dimension, and z-dimension.

[0046] For example, taking the target dimension as x, the disparity vector can be used to indicate the horizontal displacement of corresponding pixels in two images.

[0047] It should be noted that the pixel arrangement of intermediate dense primitive images between adjacent primitive images can be estimated using disparity vectors, i.e., given a pair of adjacent primitive images in the horizontal direction. The dense primitive image in the middle can be represented as: ,in, This indicates the number of interpolations specified by the user.

[0048] S103, the three-dimensional image generation device determines a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by the user input.

[0049] The three-dimensional light field encoded image is used to generate a three-dimensional image.

[0050] Optionally, the three-dimensional image generation device can determine N three-dimensional light field encoded images based on the M sparse primitive images and the number of interpolations determined by user input; ; in, Indicates the number of interpolations.

[0051] For example, with M = 9, Taking 12 as an example, the number of 3D light field encoded images In other words, the 3D image generation device can determine 105 3D light field encoded images based on 9 sparse primitive images.

[0052] Optionally, the 3D image generation device determines N 3D light field encoded images based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input, including: determining the image index n of each sub-pixel in the 3D light field encoded image within the dense primitive image according to the encoding rules of the 3D light field encoded image and the parameters of the light control device; and determining the index of the sparse primitive image according to the image index n and the number M of sparse primitive images. According to the image sequence number n and the sequence number of the sparse primitive image. and the number of interpolations Determine the interpolation interval of the sparse primitive image According to the formula Determine the position coordinates of the 3D light field encoded image; where, Represents the position coordinates of a 3D light field encoded image; This represents the `code` function; Indicates the sequence number is sparse primitive images; Let represent the (M-1)th disparity vector.

[0053] Specifically, based on the encoding rules of the 3D light field encoded image and the parameters of the light control device, the 3D image generation device can determine the image index n of each sub-pixel in the 3D light field encoded image within the dense primitive image, according to the formula... A 3D image generation device can determine the sequence number of a sparse primitive image. According to the formula A 3D image generation device can determine the interpolation interval of a sparse primitive image. Finally, combining (M-1) disparity vectors, the 3D image generation device can be based on the formula... Determine the position coordinates of the 3D light field encoded image, that is, the sparse primitive image index s and coordinates. The sub-pixels are based on the interpolation interval The image is reused and filled into the corresponding position in the 3D light field encoded image.

[0054] In this embodiment, a three-dimensional light field encoded image can be determined based on M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by the user input. Since the three-dimensional light field encoded image can be filled by reusing sparse primitive images, the process of rendering a large number of dense views in the middle can be avoided, thereby improving rendering efficiency and reducing the generation time of three-dimensional images.

[0055] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0056] The three-dimensional image generation method provided in this application can be executed by a three-dimensional image generation device or a control module for three-dimensional image generation within that device. This application uses a three-dimensional image generation device executing the three-dimensional image generation method as an example to illustrate the three-dimensional image generation device provided in this application.

[0057] It should be noted that the embodiments of this application can divide the three-dimensional image generation device into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. Optionally, the module division in the embodiments of this application is illustrative and is only a logical functional division; other division methods may be used in actual implementation.

[0058] like Figure 2 As shown in the figure, this application embodiment provides a three-dimensional image generation device 200. The three-dimensional image generation device 200 includes: an acquisition module 201 and a processing module 202. The acquisition module 201 is used to acquire M sparse primitive images; the processing module 202 is used to determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, to obtain (M-1) disparity vectors, the disparity vectors being used to indicate the displacement of corresponding pixels in the two images in the target dimension; and to determine a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input, the three-dimensional light field encoded image being used to generate a three-dimensional image; wherein, M is an integer greater than 1.

[0059] Optionally, the processing module 202 is used to determine N three-dimensional light field encoded images based on the M sparse primitive images and the number of interpolations determined by user input; ; in, This indicates the number of interpolations.

[0060] Optionally, the processing module 202 is configured to determine the image index n of each sub-pixel in the dense primitive image in the three-dimensional light field encoded image according to the encoding rules of the three-dimensional light field encoded image and the parameters of the light control device; and to determine the index of the sparse primitive image according to the image index n and the number M of sparse primitive images. According to the image sequence number n and the sequence number of the sparse primitive image. and the number of interpolations Determine the interpolation interval of the sparse primitive image According to the formula Determine the position coordinates of the 3D light field encoded image; where, Represents the position coordinates of a 3D light field encoded image; This represents the `code` function; Indicates the sequence number is sparse primitive images; Let represent the (M-1)th disparity vector.

[0061] Optionally, the target dimension includes at least one of the x-dimensional and y-dimensional dimensions.

[0062] In this embodiment, a three-dimensional light field encoded image can be determined based on M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by the user input. Since the three-dimensional light field encoded image can be filled by reusing sparse primitive images, the process of rendering a large number of dense views in the middle can be avoided, thereby improving rendering efficiency and reducing the generation time of three-dimensional images.

[0063] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a three-dimensional image generation method, which includes: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input, wherein the three-dimensional light field encoded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0064] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the three-dimensional image generation method provided by the above methods, the method including: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, obtaining (M-1) disparity vectors, the disparity vectors being used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input, the three-dimensional light field encoded image being used to generate a three-dimensional image; wherein, M is an integer greater than 1.

[0066] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a three-dimensional image generation method provided by the above methods. The method includes: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and an interpolation number determined by user input, wherein the three-dimensional light field encoded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a three-dimensional image, characterized in that, include: Obtain M sparse primitive images; Determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, which are used to indicate the displacement of corresponding pixels in the target dimension in the two images; A three-dimensional light field encoded image is determined based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input. The three-dimensional light field encoded image is used to generate a three-dimensional image. Where M is an integer greater than 1; The process of determining the 3D light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input includes: N three-dimensional light field coded images are determined based on the M sparse primitive images and the number of interpolations determined by user input; N = M + (M-1)Δ; Wherein, Δ represents the number of interpolations; The process of determining the 3D light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by user input includes: Based on the encoding rules of the three-dimensional light field encoded image and the parameters of the light control device, the image number n of each sub-pixel in the three-dimensional light field encoded image in the dense primitive image is determined; The s ... The interpolation interval ω of the sparse primitive image is determined based on the image sequence number n, the sequence number s of the sparse primitive image, and the number of interpolations Δ. According to the formula Determine the position coordinates of the 3D light field encoded image; Where 3Dimage(i, j, k) represents the position coordinates of the 3D light field encoded image; code(·) represents the code function; Input s Represents the sparse primitive image with index s; d M-1 This represents the (M-1)th disparity vector.

2. The three-dimensional image generation method according to claim 1, characterized in that, The target dimension includes at least one of the x-dimensional and y-dimensional dimensions.

3. A three-dimensional image generation device, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire M sparse primitive images; The processing module is used to determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, to obtain (M-1) disparity vectors, which are used to indicate the displacement of corresponding pixels in the target dimension in the two images; and to determine a three-dimensional light field encoded image based on the M sparse primitive images, the (M-1) disparity vectors, and the number of interpolations determined by the user input, which is used to generate a three-dimensional image; Where M is an integer greater than 1; The processing module is used to determine N three-dimensional light field encoded images based on the M sparse primitive images and the number of interpolations determined by user input; N = M + (M-1)Δ; where Δ represents the number of interpolations; The processing module is used to determine the image index n of each sub-pixel in the dense primitive image in the 3D light field encoded image according to the encoding rules of the 3D light field encoded image and the parameters of the light control device; determine the index s of the sparse primitive image according to the image index n and the number M of sparse primitive images; determine the interpolation interval ω of the sparse primitive image according to the image index n, the index s of the sparse primitive image and the number of interpolations Δ; and determine the interpolation interval ω of the sparse primitive image according to the formula 3Dimage(i,j,k)=code(Input) s (i+ω×d M-1 The input (j, k) function determines the position coordinates of the 3D light field encoded image; where 3Dimage(i, j, k) represents the position coordinates of the 3D light field encoded image; code(·) represents the code function; Input s Represents the sparse primitive image with index s; d M-1 This represents the (M-1)th disparity vector.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the three-dimensional image generation method as described in any one of claims 1 to 2.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the three-dimensional image generation method as described in any one of claims 1 to 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the three-dimensional image generation method as described in any one of claims 1 to 2.

Images