A method and apparatus for generating a multi-view image
By combining the video generation model with lightweight control signals, cross normalization and Gaussian rendering technology, the consistency problem of large generative models in multi-view image generation is solved, and multi-view image generation with high consistency and high fidelity is achieved.
Patent Information
- Application Number
- CN202411695616.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2044-11-25
AI Technical Summary
Generative large models have the problem of poor image consistency in multi-view image generation.
A video generation model is used to generate initial multi-view scene images through lightweight control signals and cross normalization technology, which are optimized in combination with key-value constraints and rendered using the Gaussian rendering method to generate highly consistent multi-view images.
The visual consistency and detail fidelity of multi-view images are improved, and the realism and consistency of the images are enhanced.
Smart Images

Figure CN119625150B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present specification relates to the technical field of computer technology and the technical field of image recognition, and in particular relates to a multi-view image generation method and device. BACKGROUND
[0002] Multi-view image generation is a method of reconstructing a scene and generating a new view image by capturing multiple images of the same scene from different perspectives.
[0003] Traditional multi-view image generation methods rely on image overlap area to calculate parallax and extract depth information, generate point clouds to reconstruct the scene. Traditional multi-view image generation methods can solve the scene reconstruction and new view image generation requirements under the condition of dense image input, but still face problems such as poor reconstruction quality and low robustness. As a new emerging frontier technology, generative large models have strong semantic understanding and result generation capabilities. Applying this technology to multi-view image generation can improve the ability level of the latter in terms of reconstruction quality, robustness, etc., but still faces the problem of poor image consistency.
[0004] Based on this, the present specification provides a multi-view image generation method and device. SUMMARY
[0005] The embodiments of the present specification provide a multi-view image generation method and device, which are used to solve the following technical problem: generative large models applied to multi-view image generation can improve the ability level of the latter in terms of reconstruction quality, robustness, etc., but still face the problem of poor image consistency.
[0006] To solve the above technical problems, the embodiments of the present specification are implemented as follows:
[0007] The embodiments of the present specification provide a multi-view image generation method, which comprises:
[0008] The video generation model generates an initial multi-view scene image of the to-be-processed image through a lightweight control signal and a cross-normalization technology;
[0009] The initial multi-view scene image is optimized through key-value constraints to generate a high-consistency multi-view image;
[0010] The high-consistency multi-view image is rendered by a Gaussian rendering method to generate a high-fidelity high-consistency multi-view image.
[0011] The embodiments of the present specification also provide a multi-view image generation device, which comprises:
[0012] An initialization module, a video generation model generates an initial multi-view scene image of the to-be-processed image through a lightweight control signal and a cross-normalization technology;
[0013] an optimization module, which optimizes the initial multi-view scene image through key-value constraints to generate a high-consistency multi-view image;
[0014] a rendering module, which renders the high-consistency multi-view image through a Gaussian rendering method to generate a high-fidelity high-consistency multi-view image.
[0015] The multi-view image generation method provided by the embodiments of the present specification can generate an initial multi-view scene image of a to-be-processed image through a video generation model through a lightweight control signal and a cross-normalization technology, optimize the initial multi-view scene image through key-value constraints to generate a high-consistency multi-view image, and render the high-consistency multi-view image through a Gaussian rendering method to generate a high-fidelity high-consistency multi-view image, which can ensure that the output image has high consistency and detail fidelity in vision and can improve the realism and consistency of the image. BRIEF DESCRIPTION OF DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present specification or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.
[0017] Figure 1 A system architecture schematic diagram of a multi-view image generation method provided by the embodiments of the present specification;
[0018] Figure 2 A flowchart of a multi-view image generation method provided by the embodiments of the present specification;
[0019] Figure 3 A principle flowchart of a multi-view image generation method provided by the embodiments of the present specification;
[0020] Figure 4 A schematic diagram of a multi-view image generation device provided by the embodiments of the present specification. DETAILED DESCRIPTION
[0021] In order to make the technical solutions in the specification better understood by the person skilled in the art, the technical solutions in the specification will be clearly and completely described below in combination with the drawings in the specification. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the specification embodiments, all other embodiments obtained by the person skilled in the art without creative labor should fall within the scope of protection of the present application.
[0022] Figure 1 A system architecture schematic diagram of a multi-view image generation method provided by the specification embodiment is shown in FIG. 1. As shown in FIG. 1, the system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc. Figure 1
[0023] The terminal devices 101, 102, 103 interact with the server 105 through the network 104 to receive or send messages, etc. Various client applications can be installed on the terminal devices 101, 102, 103. For example, a special program for performing a multi-view image generation method, etc.
[0024] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various special or general electronic devices, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or as a single software or software module.
[0025] The server 105 can be a server providing various services, such as a backend server providing services for the client applications installed on the terminal devices 101, 102, 103. For example, the server can perform a multi-view image generation to display the multi-view image generation result on the terminal devices 101, 102, 103.
[0026] The server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or as a single software or software module.
[0027] Figure 2 A flowchart of a multi-view image generation method provided by an embodiment of the present specification is shown. From a program perspective, the execution subject of the flowchart can be a program loaded on an application server or an application terminal. It can be understood that the method can be executed by any device, equipment, platform, or cluster of equipment having computing and processing capabilities. As shown in the figure, the method comprises the following steps. Figure 1
[0028] Step S201: The video generation model generates an initial multi-view scene image of the to-be-processed image through a lightweight control signal and cross-normalization technology.
[0029] In an embodiment of the present specification, the to-be-processed image is an image of different views of a to-be-processed static object. In a specific embodiment, the to-be-processed image includes at least three views, specifically, a front view, a left side view, and a right side view. Of course, the more angles of the to-be-processed image, the better. When selecting the angle of the to-be-processed image, the angle of the typical part of the to-be-processed image should be included, and if there are multiple angles, it is best to evenly distribute each angle.
[0030] In an embodiment of the present specification, the video generation model is an artificial intelligence model based on deep learning technology. In an embodiment of the present specification, the video generation model can be Zero123, One-2-3-45++, SyncDreamer, ImageDream, iNVS, etc. The specific model of the video generation model is not limited in the present application. The video generation model can be any large model that can realize multi-view image synthesis, as well as other artificial intelligence models.
[0031] In an embodiment of the present specification, the video generation model has a time series diffusion characteristic.
[0032] In an embodiment of the present specification, the video generation model generates an initial multi-view scene image of the to-be-processed image through a lightweight control signal and cross-normalization technology, specifically comprising:
[0033] Embedding the view information corresponding to the to-be-processed image into the initial multi-view image through a lightweight encoder to generate the lightweight control signal, wherein the view information includes camera position and camera rotation degree;
[0034] Based on the lightweight control signal and the video generation model, cross-normalization is performed to obtain the initial multi-view scene image.
[0035] In the embodiments of the present application, the view information is embedded into the generation process through a lightweight Transformer encoder, reducing the computational complexity and improving the generation speed, so that the complexity of the calculation is reduced.
[0036] In the embodiments of the present application, the lightweight encoder is a linear Transformer.
[0037] The attention mechanism of the linear Transformer is:
[0038] Attention(Q, K, V) = σ(QK T )V
[0039] wherein,
[0040] σ represents a normalization function.
[0041] Q represents a query.
[0042] K represents a key.
[0043] V represents a value.
[0044] T represents transposition.
[0045] K T represents key transposition.
[0046] QK T represents dot product operation between the query and the key.
[0047] In the embodiments of the present application, the lightweight encoder uses an exponential linear unit as an activation parameter to increase nonlinearity.
[0048] The similarity calculation formula of the lightweight encoder is:
[0049] σ(Q, K) = φ(Q)·φ(K) T
[0050] wherein,
[0051] σ represents a normalization function.
[0052] σ(Q, K) represents the similarity score between the query and the key.
[0053] φ represents a transformation function.
[0054] φ(Q) represents the transformation function of the query.
[0055] φ(K) T represents the transformation function of the key transposition.
[0056] φ(·) = elu(·) + 1
[0057] wherein,
[0058] elu represents an exponential linear activation function.
[0059] In the embodiments of the present specification, the exponential linear unit is used as an activation parameter, that is, an exponential linear activation function elu, and the design purpose of elu is to overcome the disadvantage that the derivative of ReLU (Rectified Linear Unit) is zero in the negative interval, while maintaining the advantages similar to ReLU, such as non-linear, sparse activation and the like. Unlike ReLU, ELU does not completely turn off the neuron when the input is negative, but has a minimum non-zero response, which helps to alleviate the problem of gradient disappearance.
[0060] In the embodiments of the present specification, the video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross-normalization technology, specifically including:
[0061] Based on the corresponding features of the lightweight control signal, the mean of the corresponding features of the lightweight control signal and the standard deviation of the corresponding features of the lightweight control signal are generated;
[0062] Based on the different layer features of the video generation model, the mean of the different layer features of the video generation model and the standard deviation of the different layer features of the video generation model are generated
[0063] Based on the mean of the corresponding features of the lightweight control signal, the standard deviation of the corresponding features of the lightweight control signal, the mean of the different layer features of the video generation model, the standard deviation of the different layer features of the video generation model and the different layer features of the video generation model, a fine-tuning parameter is generated;
[0064] The fine-tuning parameter is fine-tuned to obtain a normalized multi-view image;
[0065] Wherein, the calculation formula of the fine-tuning parameter is:
[0066]
[0067] Wherein,
[0068] z' represents a fine-tuning parameter;
[0069] z represents the different layer features of the video generation model;
[0070] μ(z) represents the standard deviation of the different layer features of the video generation model;
[0071] σ(z) represents the mean of the different layer features of the video generation model;
[0072] σ(c) represents the mean of the corresponding features of the lightweight control signal;
[0073] μ(c) represents the standard deviation of the corresponding feature of the lightweight control signal.
[0074] Step S203: optimizing the initial multi-view scene image through key-value constraints to generate a high-consistency multi-view image.
[0075] To further enhance the consistency of information between different views, the embodiments of the present specification provide a method of key-value constraints to further optimize the image.
[0076] In the embodiments of the present specification, the optimization of the normalized multi-view image through key-value constraints to generate an optimized multi-view image specifically includes:
[0077] inter-view query and key exchange are performed on the normalized multi-view image, and a consistency loss function is used for constraint to obtain an exchanged multi-view image;
[0078] The exchanged multi-view image is optimized layer by layer using a fusion model to generate the optimized multi-view image.
[0079] Through inter-view query and key exchange, the information between different views can be supplemented to improve the consistency between views. The existence of the consistency loss function can ensure the consistent transmission between different views and improve the consistency of information transmission between multi-views.
[0080] In the embodiments of the present specification, the formula for inter-view query and key exchange of the normalized multi-view image is:
[0081] Q i = K j , K i = Q j
[0082] wherein,
[0083] i represents view i;
[0084] j represents view j;
[0085] Q i represents the query of view i;
[0086] K j represents the key of view j;
[0087] K i represents the key of view i;
[0088] Q j represents the query of view j;
[0089] The expression of the consistency loss function is:
[0090] L consistency =‖Q i -K j ‖ 2 +‖Q j -K i ‖ 2
[0091] wherein,
[0092] L consistency denotes a consistency loss function;
[0093] ‖·‖ 2 denotes the square of the Euclidean distance;
[0094] ‖Q i -K j ‖ 2 denotes the square of the Euclidean distance between the query Q i and the key K j ;
[0095] ‖Q j -K i ‖ 2 denotes the square of the Euclidean distance between the query Q j and the key K i ;
[0096] The fusion model is optimized layer by layer based on the following formula:
[0097] x l+1 = Diffusion(x l ) x a + Control(c) x (1-a)
[0098] wherein,
[0099] x l denotes the feature of the lth layer;
[0100] x l+1 denotes the feature of the (l+1)th layer;
[0101] c denotes a control signal;
[0102] a denotes a fusion parameter;
[0103] Diffusion(x l ) denotes the diffusion influence of the feature of the lth layer;
[0104] Control(c) denotes the influence of the control signal c.
[0105] In the embodiments of the present specification, the existence of the fusion model can integrate the control signals by increasing the value layer by layer, making the feature transmission more coherent, while retaining important control information.
[0106] In the embodiments of the present specification, the fusion parameter a is [0, 1], which is the weight of controlling the diffusion feature and the control signal. When a is close to 0, the model relies more on the control signal; when a is close to 1, the model relies more on the diffusion feature of the current level. In specific embodiments, the determination of the fusion parameter a needs to be determined according to specific business scenarios.
[0107] It should be particularly pointed out that the view i and the view j belong to different views.
[0108] Step S205: using a Gaussian rendering method to render the high-consistency multi-view image to generate a high-fidelity high-consistency multi-view image.
[0109] In the embodiments of the present specification, the Gaussian rendering method is used to render the optimized multi-view image to generate a high-fidelity multi-view image, specifically including:
[0110] The optimized multi-view image is rendered by Gaussian sputtering 3D rendering, and the signal of the initial multi-view image is constrained to generate the high-fidelity multi-view image.
[0111] In the embodiments of the present specification, the signal of the initial multi-view image takes the object-level token as the signal;
[0112] The generation formula of the high-fidelity multi-view image is:
[0113] I'(v) = å i w i ·G(v-v i )·I(v i )
[0114] wherein,
[0115] I'(v) represents the pixel value of the high-fidelity multi-view image;
[0116] w i represents the weight of each Gaussian body;
[0117] G(v-v i ) represents a Gaussian kernel function;
[0118] I(v i ) represents the pixel value of the optimized multi-view image;
[0119] v i is the position of the Gaussian body;
[0120] i represents the view angle i.
[0121] In Gaussian splatting 3D rendering, each object or part of an object in the scene can be represented by one or more Gaussian splats. Each Gaussian splat has its own position, color, intensity, and other attributes such as texture or transparency. When rendering an image, each Gaussian splat contributes a certain pixel value to the final image based on the relationship between its position in three-dimensional space and the viewer's perspective.
[0122] The Gaussian splatting rendering method involves the following steps:
[0123] Determine Gaussian splats: For each object in the scene, determine a set of Gaussian splats, each representing a part of the object and assigned with corresponding attributes (e.g., color, transparency, etc.).
[0124] Calculate weights: Calculate a weight w i for each Gaussian splat, which determines the degree of influence of the Gaussian splat on the image pixel value.
[0125] Apply Gaussian kernel function: Use the Gaussian kernel function G(v-v i ) to calculate the influence of each Gaussian splat on a point v in the image, where v is the pixel coordinate in the image, and v i is the position of the Gaussian splat.
[0126] Synthesize image: Finally, combine the contributions of all Gaussian splats to form the final image pixel value I'(v), i.e., the final color at each pixel is the result of the superposition of all associated Gaussian splats according to the weights.
[0127] To further understand the multi-view image generation method provided by the embodiments of the present specification, specific embodiments will be described below.
[0128] Figure 3 The principle flowchart of the multi-view image generation method provided by the embodiments of the present specification. As Figure 3 shown, the video generation model utilizes its time series diffusion characteristics and cross-normalization processing of the view angle control signal features to generate normalized view angle control signal features. The normalized view angle control signal features utilize cross-attention and the video generation model to jointly output video multi-view scene images. The video multi-view scene images are optimized using multi-level key-value constrained consistent images to generate high-consistency multi-view images. Finally, the high-consistency multi-view images are based on object-level Gaussian rendering to generate high-guarantee high-consistency multi-view images.
[0129] The method for generating multi-view images provided by the embodiments of the present specification can generate an initial multi-view scene image of a to-be-processed image through a video generation model by using a lightweight control signal and a cross-normalization technology, optimize the initial multi-view scene image by using a key-value constraint, generate a high-consistency multi-view image, and render the high-consistency multi-view image by using a Gaussian rendering method to generate a high-fidelity high-consistency multi-view image, so that the output image can have high consistency and detail fidelity in vision, and the image can have high reality and consistency.
[0130] The above describes a method for generating multi-view images. Correspondingly, the present specification also provides a device for generating multi-view images, as shown in Figure 4 . Figure 4 The device for generating multi-view images provided by the embodiments of the present specification includes:
[0131] The initialization module 401 generates an initial multi-view scene image of a to-be-processed image through a video generation model by using a lightweight control signal and a cross-normalization technology.
[0132] The optimization module 403 optimizes the initial multi-view scene image by using a key-value constraint to generate a high-consistency multi-view image.
[0133] The rendering module 405 renders the high-consistency multi-view image by using a Gaussian rendering method to generate a high-fidelity high-consistency multi-view image.
[0134] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.
[0135] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device, electronic device, and non-volatile computer storage medium embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.
[0136] The device, the electronic device, the non-volatile computer storage medium and the method provided by the embodiments of the present specification are corresponding, therefore, the device, the electronic device, the non-volatile computer storage medium also have similar beneficial technical effects with the corresponding method, since the beneficial technical effects of the method have been described in detail above, therefore, the beneficial technical effects of the corresponding device, the electronic device, the non-volatile computer storage medium will not be repeated here.
[0137] It also needs to be explained that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitation, the element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0138] The specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. The specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.
[0139] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.
[0140] The above only describes the embodiments of the present specification and does not limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of claims of the present application.
Claims
1. A method for generating a multi-view image, characterized in that: The generation method comprises: The video generation model generates the initial multi-view scene images of the image to be processed through lightweight control signals and cross normalization technology; The initial multi-view scene image is optimized through key-value constraints to generate a highly consistent multi-view image, specifically comprising: performing inter-view query and key exchange on the initial multi-view scene image, and constraining it through a consistency loss function to obtain an exchanged multi-view image; using a fusion model, layer-by-layer optimization of the exchanged multi-view image to generate the optimized highly consistent multi-view image; the formula for inter-view query and key exchange of the initial multi-view scene image is: Q i =K j ,K i =Q j in, i represents perspective i; j represents the view angle j; Q i represents the query of perspective i; K j represents the key of viewpoint j; K i The key representing perspective i; Q j represents the query from perspective j; The consistency loss function expression is: L consistency =‖Q i -K j ‖ 2 +‖Q j -K i ‖ 2 in, L consistency represents the consistency loss function; ‖·‖ 2 represents the square of the Euclidean distance; ‖Q i -K j ‖ 2 Represents query Q i and key K j The square of the Euclidean distance between them; ‖Q j -K i ‖ 2 Represents query Q j and key K i The square of the Euclidean distance between them; The fusion model is optimized layer by layer based on the following formula: x l+1 =Diffusion(x l )×α+Control(c)×(1-α) in, x l Represents the features of the lth layer; x l+1 Represents the features of the l+1th layer; c represents the control signal; α represents the fusion parameter; Diffusion(x l ) represents the diffusion influence of the features of layer l; Control(c) represents the influence of control signal c; The highly consistent multi-view image is rendered using a Gaussian rendering method to generate a high-fidelity and highly consistent multi-view image.
2. The generation method according to claim 1, wherein: The video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and cross normalization technology, specifically including: Embedding the perspective information corresponding to the image to be processed into the initial multi-view image through a lightweight encoder to generate the lightweight control signal, wherein the perspective information includes a camera position and a camera rotation degree; Based on the lightweight control signal and the video generation model, cross normalization is performed to obtain the initial multi-view scene image.
3. The generation method according to claim 2, wherein: The lightweight encoder is a linear Transformer; The attention mechanism of the linear Transformer is: Attention(Q,K,V)=σ(QK T )V in, σ represents the normalization function; Q stands for query; K represents a key; V represents value; T stands for transpose; K T represents key transposition; QK T Represents the dot product operation between the query and the key; 4. The generation method according to claim 3, wherein: The lightweight encoder uses an exponential linear unit as an activation parameter to increase nonlinearity; The similarity calculation formula of the lightweight encoder is: σ(Q,K)=φ(Q)·φ(K) T in, σ represents the normalization function; σ(Q, K) represents the similarity score between the query and the key; φ represents the transformation function; φ(Q) represents the transformation function of the query; φ(K) T Transformation function representing key transposition; φ(·)=elu(·)+1 in, elu represents the exponential linear activation function.
5. The generation method according to claim 1, wherein: The video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and cross normalization technology, specifically including: generating, based on the corresponding feature of the lightweight control signal, a mean of the corresponding feature of the lightweight control signal and a standard deviation of the corresponding feature of the lightweight control signal; Based on the different layer features of the video generation model, the mean of the different layer features of the video generation model and the standard deviation of the different layer features of the video generation model are generated generating a fine-tuning parameter based on a mean of a corresponding feature of the lightweight control signal, a standard deviation of the corresponding feature of the lightweight control signal, a mean of features at different layers of the video generative model, a standard deviation of features at different layers of the video generative model, and features at different layers of the video generative model; Fine-tuning the fine-tuning parameters to obtain a normalized multi-view image; The calculation formula of the fine-tuning parameter is: in, z′ represents the fine-tuning parameter; z represents different layer features of the video generation model; μ(z) represents the standard deviation of the features at different layers of the video generation model; σ(z) represents the mean of the features of different layers of the video generation model; σ(c) represents the mean of the corresponding feature of the lightweight control signal; μ(c) represents the standard deviation of the corresponding feature of the lightweight control signal.
6. The generation method according to claim 1, wherein: The method of using Gaussian rendering to render the highly consistent multi-view image to generate a high-fidelity and highly consistent multi-view image specifically includes: The high-fidelity and high-consistency multi-view image is rendered by Gaussian sputtering 3D rendering and constrained by the signal of the initial multi-view scene image to generate the high-fidelity and high-consistency multi-view image.
7. The generation method according to claim 6, wherein: The signal of the initial multi-view scene image uses object-level tokens as signals; The generation formula of the high-fidelity and high-consistency multi-view image is: I′(v)=∑ i w i ·G(vv i )·I(v i ) in, I′(v) represents the pixel value of the high-fidelity and high-consistency multi-view image; w i Represents the weight of each Gaussian; G(vv i ) represents the Gaussian kernel function; I(v i ) represents the pixel value of the initial multi-view scene image; v i is the position of the Gaussian body; i represents view angle i.
8. A device for generating multi-view images, characterized in that: The generating device is used to execute the method for generating multi-view images as claimed in claim 1.
Citation Information
Patent Citations
Video generation method, device, medium, and electronic device supporting multi-view watching
CN107343206A
Subway station multi-view vector simulation rendering method and system associated with scene structure
CN115841546A