Three-dimensional image generation method and device, equipment, storage medium and program product

Abstract

The invention provides a three-dimensional image generation method and device, equipment, a storage medium and a program product, and is applied to the technical field of image processing. The method comprises the following steps: acquiring M sparse primitive images; a disparity vector between every two adjacent sparse primitive images in the M sparse primitive images is determined, (M-1) disparity vectors are obtained, and the disparity vectors are used for indicating displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field coding image based on the M sparse primitive images, the (M-1) parallax vectors and the interpolation number determined by user input, wherein the three-dimensional light field coding image is used for generating a three-dimensional image; wherein M is an integer greater than 1.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a three-dimensional image generation method, device, equipment, storage medium and program product. Background Art

[0002] With the vigorous development of 3D stereoscopic display technology, people can get an immersive experience visually.

[0003] In the prior art, in order to realize the generation of 3D images, a disparity map sequence may be read or generated first, and then the disparity map sequence may be pixel-sampled and rearranged according to the hardware parameters and display principle of the 3D display, so as to obtain a 3D image that can be used for 3D display.

[0004] However, before rendering a 3D image, it is necessary to pre-render several dense viewpoint images according to the viewing angle and number of viewpoints of the 3D display. Therefore, not only is the rendering efficiency low and the generation rate slow, but the transmission bandwidth or storage bandwidth is also high, which cannot meet the requirements of 3D light field display for fast and high-resolution images. Summary of the invention

[0005] The present invention provides a three-dimensional image generation method, device, equipment, storage medium and program product, which are used to solve the problem that the 3D image generation method in the prior art not only has low rendering efficiency and slow generation rate, but also has high transmission bandwidth or storage bandwidth, and cannot meet the demand of 3D light field display for fast and high-resolution images.

[0006] The present invention provides a three-dimensional image generation method, comprising: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vector is used to indicate the displacement of corresponding pixels in the two images in a target dimension; determining a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and an interpolation number determined by a user input, wherein the three-dimensional light field coded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0007] According to a three-dimensional image generation method provided by the present invention, the three-dimensional light field coded image is determined based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input, comprising: Determine N three-dimensional light field coded images based on the M sparse primitive images and the interpolation number determined by user input; ; in, Indicates the interpolation number.

[0008] According to a three-dimensional image generation method provided by the present invention, the method of determining N three-dimensional light field coded images based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input comprises: Determine the image sequence n of each sub-pixel in the three-dimensional light field coded image in the dense primitive image according to the coding rule of the three-dimensional light field coded image and the parameters of the light control device; Determine the sequence number of the sparse primitive image according to the image sequence number n and the number M of the sparse primitive images ; According to the image sequence number n, the sequence number of the sparse primitive image And the interpolation number Determine the interpolation interval for a sparse primitive image ; According to the formula Determine the position coordinates of the three-dimensional light field encoded image; in, Represents the position coordinates of a three-dimensional light field encoded image; Indicates the code function; Indicates the serial number is Sparse primitive image of ; represents the (M-1)th disparity vector.

[0009] According to a three-dimensional image generation method provided by the present invention, the target dimension includes at least one of an x ​​dimension and a y dimension.

[0010] The present invention also provides a three-dimensional image generation device, comprising the following modules: an acquisition module and a processing module; The acquisition module is used to acquire M sparse primitive images; The processing module is used to determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, where the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in a target dimension; determine a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input, where the three-dimensional light field coded image is used to generate a three-dimensional image; Wherein, M is an integer greater than 1.

[0011] According to a three-dimensional image generation device provided by the present invention, the processing module is used to determine N three-dimensional light field coded images based on the M sparse primitive images and the interpolation number determined by user input; ; in, Indicates the interpolation number.

[0012] According to a three-dimensional image generation device provided by the present invention, the processing module is used to determine the image sequence number n of each sub-pixel in the three-dimensional light field coded image in the dense primitive image according to the coding rule of the three-dimensional light field coded image and the light control device parameter; determine the sequence number of the sparse primitive image according to the image sequence number n and the number M of the sparse primitive images ; According to the image number n, the number of the sparse primitive image And the interpolation number Determine the interpolation interval for a sparse primitive image According to the formula Determine the position coordinates of the three-dimensional light field coded image; wherein, Represents the position coordinates of a three-dimensional light field encoded image; Indicates the code function; Indicates the serial number is Sparse primitive image of ; represents the (M-1)th disparity vector.

[0013] According to a three-dimensional image generating device provided by the present invention, the target dimension includes at least one of an x ​​dimension and a y dimension.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the three-dimensional image generation methods described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the three-dimensional image generating method as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the three-dimensional image generating method as described above is implemented.

[0017] The three-dimensional image generation method, device, equipment, storage medium and program product provided by the present invention can obtain M sparse primitive images; determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, and the disparity vector is used to indicate the displacement of corresponding pixels in the two images in the target dimension; determine a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by the user input, and the three-dimensional light field coded image is used to generate a three-dimensional image; wherein M is an integer greater than 1. Through this scheme, a three-dimensional light field coded image can be determined based on M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by the user input. Since the three-dimensional light field coded image can be filled by multiplexing the sparse primitive images, the process of rendering a large number of dense views in the middle can be avoided, thereby improving the rendering efficiency and reducing the three-dimensional image generation time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of the flow of the three-dimensional image generation method provided by the present invention; Figure 2 It is a structural schematic diagram of a three-dimensional image generating device provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0022] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0023] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.

[0024] The embodiments of the present application describe some exemplary embodiments for the purpose of explanation. It should be understood that the present application can be implemented in other ways that are not specifically shown in the drawings.

[0025] like Figure 1 As shown, the embodiment of the present application provides a three-dimensional image generation method, which can be applied to a three-dimensional image generation device. The three-dimensional image generation method may include S101-S103: S101. A three-dimensional image generating device obtains M sparse primitive images.

[0026] Wherein, the above M is an integer greater than 1.

[0027] Alternatively, the above M sparse primitive images can be expressed as .

[0028] S102 . The three-dimensional image generating apparatus determines a disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M−1) disparity vectors.

[0029] The above disparity vector is used to indicate the displacement of corresponding pixels in two images in the target dimension.

[0030] Specifically, the three-dimensional image generation device can determine the sparse primitive image With sparse primitive images The disparity vector between , determine the sparse primitive image With sparse primitive images The disparity vector between , and so on, until the sparse primitive image is determined With sparse primitive images The disparity vector between , thus obtaining (M-1) disparity vectors, which can be expressed as .

[0031] Optionally, the target dimension may include at least one of an x ​​dimension, a y dimension and a z dimension.

[0032] Exemplarily, taking the target dimension as the x dimension as an example, the disparity vector may be used to indicate the displacement of corresponding pixels in two images in the horizontal direction.

[0033] It should be noted that the pixel arrangement of the intermediate dense primitive images between adjacent primitive images can be estimated by the disparity vector, that is, given a pair of adjacent primitive images in the horizontal direction , where the intermediate dense primitive image can be expressed as: ,in, Indicates the number of interpolations determined by user input.

[0034] S103 . The three-dimensional image generating apparatus determines a three-dimensional light field coded image based on the M sparse primitive images, the (M−1) disparity vectors, and an interpolation number determined by a user input.

[0035] The three-dimensional light field coded image is used to generate a three-dimensional image.

[0036] Optionally, the three-dimensional image generating device may determine N three-dimensional light field coded images based on the M sparse primitive images and the interpolation number determined by user input; ; in, Indicates the number of interpolations.

[0037] For example, M is 9, Taking 12 as an example, the number of 3D light field coded images That is, the 3D image generation device can determine 105 3D light field coded images based on 9 sparse primitive images.

[0038] Optionally, the three-dimensional image generating device determines N three-dimensional light field coded images based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by the user input, including: determining the image sequence number n of each sub-pixel in the three-dimensional light field coded image in the dense primitive image according to the encoding rule of the three-dimensional light field coded image and the light control device parameter; determining the sequence number of the sparse primitive image according to the image sequence number n and the number M of the sparse primitive images ; According to the image number n, the number of the sparse primitive image And the interpolation number Determine the interpolation interval for a sparse primitive image According to the formula Determine the position coordinates of the three-dimensional light field coded image; wherein, Represents the position coordinates of a three-dimensional light field encoded image; Indicates the code function; Indicates the serial number is Sparse primitive image of ; represents the (M-1)th disparity vector.

[0039] Specifically, according to the coding rules of the 3D light field coded image and the parameters of the light control device, the 3D image generation device can determine the image sequence number n of each sub-pixel in the 3D light field coded image in the dense primitive image, according to the formula The three-dimensional image generating device can determine the sequence number of the sparse primitive image According to the formula The three-dimensional image generation device can determine the interpolation interval of the sparse primitive image ; Finally, combined with the (M-1) disparity vectors, the 3D image generation device can be based on the formula Determine the position coordinates of the three-dimensional light field encoded image, that is, the sparse primitive image sequence number s, coordinates The sub-pixel points below are based on the interpolation interval The three-dimensional light field coded images are multiplexed and filled into corresponding positions.

[0040] In an embodiment of the present application, a three-dimensional light field coded image can be determined based on M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input. Since the three-dimensional light field coded image can be filled by reusing the sparse primitive images, the process of rendering a large number of dense views in the middle can be avoided, thereby improving rendering efficiency and reducing the time required to generate a three-dimensional image.

[0041] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0042] The 3D image generation method provided in the embodiment of the present application can be executed by a 3D image generation device or a control module for 3D image generation in the 3D image generation device. The 3D image generation device provided in the embodiment of the present application is described by taking the 3D image generation method executed by the 3D image generation device as an example.

[0043] It should be noted that the embodiment of the present application can divide the functional modules of the three-dimensional image generation device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0044] like Figure 2 As shown, an embodiment of the present application provides a three-dimensional image generating device 200. The three-dimensional image generating device 200 includes: an acquisition module 201 and a processing module 202. The acquisition module 201 is used to acquire M sparse primitive images; the processing module 202 is used to determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images, and obtain (M-1) disparity vectors, and the disparity vector is used to indicate the displacement of corresponding pixels in the two images in the target dimension; based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by the user input, a three-dimensional light field coded image is determined, and the three-dimensional light field coded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0045] Optionally, the processing module 202 is configured to determine N three-dimensional light field coded images based on the M sparse primitive images and an interpolation number determined by a user input; ; in, Indicates the interpolation number.

[0046] Optionally, the processing module 202 is used to determine the image sequence number n of each sub-pixel in the three-dimensional light field coded image in the dense primitive image according to the coding rule of the three-dimensional light field coded image and the light control device parameter; determine the sequence number of the sparse primitive image according to the image sequence number n and the number M of the sparse primitive images. ; According to the image number n, the number of the sparse primitive image And the interpolation number Determine the interpolation interval for a sparse primitive image According to the formula Determine the position coordinates of the three-dimensional light field coded image; wherein, Represents the position coordinates of a three-dimensional light field encoded image; Indicates the code function; Indicates the serial number is Sparse primitive image of ; represents the (M-1)th disparity vector.

[0047] Optionally, the target dimension includes at least one of an x-dimension and a y-dimension.

[0048] In an embodiment of the present application, a three-dimensional light field coded image can be determined based on M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input. Since the three-dimensional light field coded image can be filled by reusing the sparse primitive images, the process of rendering a large number of dense views in the middle can be avoided, thereby improving rendering efficiency and reducing the time required to generate a three-dimensional image.

[0049] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a three-dimensional image generation method, the method comprising: obtaining M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vector is used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors and the interpolation number determined by the user input, wherein the three-dimensional light field coded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0050] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0051] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the three-dimensional image generation method provided by the above methods, the method including: obtaining M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vector is used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input, wherein the three-dimensional light field coded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0052] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the three-dimensional image generation method provided by the above-mentioned methods, the method comprising: acquiring M sparse primitive images; determining the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, wherein the disparity vector is used to indicate the displacement of corresponding pixels in the two images in the target dimension; determining a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input, wherein the three-dimensional light field coded image is used to generate a three-dimensional image; wherein M is an integer greater than 1.

[0053] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0054] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional image generation method, characterized in that: include: Get M sparse primitive images; Determine a disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, where the disparity vector is used to indicate the displacement of corresponding pixels in the two images in the target dimension; Determine a 3D light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and an interpolation number determined by a user input, wherein the 3D light field coded image is used to generate a 3D image; Wherein, M is an integer greater than 1.

2. The three-dimensional image generation method according to claim 1, characterized in that: The determining of a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by a user input comprises: Determine N three-dimensional light field coded images based on the M sparse primitive images and the interpolation number determined by user input; ； in, Indicates the interpolation number.

3. The three-dimensional image generation method according to claim 2, characterized in that: The determining N three-dimensional light field coded images based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input comprises: Determine the image sequence n of each sub-pixel in the three-dimensional light field coded image in the dense primitive image according to the coding rule of the three-dimensional light field coded image and the parameters of the light control device; Determine the sequence number of the sparse primitive image according to the image sequence number n and the number M of the sparse primitive images ; According to the image sequence number n, the sequence number of the sparse primitive image And the interpolation number Determine the interpolation interval for a sparse primitive image ; According to the formula Determine the position coordinates of the three-dimensional light field encoded image; in, Represents the position coordinates of a three-dimensional light field encoded image; Indicates the code function; Indicates the serial number is Sparse primitive image of ; represents the (M-1)th disparity vector.

4. The three-dimensional image generation method according to claim 1, characterized in that: The target dimension includes at least one of an x-dimension and a y-dimension.

5. A three-dimensional image generating device, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire M sparse primitive images; The processing module is used to determine the disparity vector between every two adjacent sparse primitive images in the M sparse primitive images to obtain (M-1) disparity vectors, where the disparity vectors are used to indicate the displacement of corresponding pixels in the two images in a target dimension; determine a three-dimensional light field coded image based on the M sparse primitive images, the (M-1) disparity vectors, and the interpolation number determined by user input, where the three-dimensional light field coded image is used to generate a three-dimensional image; Wherein, M is an integer greater than 1.

6. The three-dimensional image generating device according to claim 5, characterized in that: The processing module is used to determine N three-dimensional light field coded images based on the M sparse primitive images and the interpolation number determined by user input; ; in, Indicates the interpolation number.

7. The three-dimensional image generating device according to claim 6, characterized in that: The processing module is used to determine the image sequence number n of each sub-pixel in the three-dimensional light field coded image in the dense primitive image according to the coding rule of the three-dimensional light field coded image and the light control device parameter; determine the sequence number of the sparse primitive image according to the image sequence number n and the number M of the sparse primitive images ; According to the image number n, the number of the sparse primitive image And the interpolation number Determine the interpolation interval for a sparse primitive image According to the formula Determine the position coordinates of the three-dimensional light field coded image; wherein, Represents the position coordinates of a three-dimensional light field encoded image; Indicates the code function; Indicates the serial number is Sparse primitive image of ; represents the (M-1)th disparity vector.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the three-dimensional image generating method according to any one of claims 1 to 4 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the three-dimensional image generating method according to any one of claims 1 to 4 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the three-dimensional image generating method according to any one of claims 1 to 4 is implemented.

Images