Multi-view image generation method and device

Abstract

The embodiment of the invention discloses a multi-view image generation method and device, and the method comprises the steps: enabling a video generation model to generate an initial multi-view scene image of a to-be-processed image through a lightweight control signal and a cross normalization technology; the initial multi-view scene image is optimized through key-value constraint, and a high-consistency multi-view image is generated; and rendering the high-consistency multi-view-angle image by adopting a Gaussian rendering method to generate a high-fidelity high-consistency multi-view-angle image.

Description

Technical Field

[0001] The present specification relates to the fields of computer technology and image recognition technology, and in particular to a method and device for generating multi-view images. Background Art

[0002] Multi-view image generation is a method of taking multiple images of the same scene from different perspectives, reconstructing the scene and generating new perspective images.

[0003] Traditional multi-view image generation methods rely on calculating disparity and extracting depth information from overlapping areas of images to generate point clouds to reconstruct scenes. Traditional multi-view image generation methods can solve the needs of scene reconstruction and new view image generation in the case of dense image input, but they still face problems such as poor reconstruction quality and low robustness. As an emerging cutting-edge technology, generative large models have strong semantic understanding and result generation capabilities. Applying this technology to multi-view image generation can improve the latter's capabilities in terms of reconstruction quality and robustness, but it still faces the problem of poor results in terms of image consistency.

[0004] Based on this, this specification provides a method and device for generating multi-view images. Summary of the invention

[0005] The embodiments of this specification provide a method and device for generating a multi-view image, which are used to solve the following technical problems: the application of a generative large model to multi-view image generation can improve the latter's capabilities in terms of reconstruction quality and robustness, but still faces the problem of poor results in terms of image consistency.

[0006] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0007] The embodiment of this specification provides a method for generating a multi-view image, including:

[0008] The video generation model generates the initial multi-view scene image of the image to be processed through lightweight control signals and cross normalization technology;

[0009] By means of key-value constraints, the initial multi-view scene image is optimized to generate a highly consistent multi-view image;

[0010] The high-consistency multi-view image is rendered using a Gaussian rendering method to generate a high-fidelity and high-consistency multi-view image.

[0011] The embodiment of this specification also provides a device for generating a multi-view image, including:

[0012] Initialization module: The video generation model generates the initial multi-view scene image of the image to be processed through lightweight control signals and cross normalization technology;

[0013] An optimization module optimizes the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image;

[0014] The rendering module uses a Gaussian rendering method to render the high-consistency multi-view image to generate a high-fidelity and high-consistency multi-view image.

[0015] The method for generating a multi-view image provided in the embodiment of the present specification generates an initial multi-view scene image of the image to be processed through a video generation model using a lightweight control signal and cross-normalization technology; optimizes the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image; and renders the highly consistent multi-view image using a Gaussian rendering method to generate a high-fidelity, high-consistent multi-view image, which can ensure that the output image has a high degree of visual consistency and detail fidelity, and can improve the realism and consistency of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 A schematic diagram of the system architecture of a method for generating a multi-view image provided in an embodiment of this specification;

[0018] Figure 2 A schematic diagram of a flow chart of a method for generating a multi-view image according to an embodiment of this specification;

[0019] Figure 3 A principle flow chart of a method for generating a multi-view image provided in an embodiment of this specification;

[0020] Figure 4 A schematic diagram of a multi-view image generation device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0022] Figure 1 The following is a schematic diagram of the system architecture of a method for generating a multi-view image provided in an embodiment of this specification. Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0023] The terminal devices 101, 102, 103 interact with the server 105 via the network 104 to receive or send messages, etc. Various client applications may be installed on the terminal devices 101, 102, 103, such as dedicated programs for performing multi-view image generation methods, etc.

[0024] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various dedicated or general electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or they may be implemented as a single software or software module.

[0025] The server 105 may be a server that provides various services, such as a backend server that provides services for client applications installed on the terminal devices 101, 102, and 103. For example, the server may generate a multi-view image so as to display the generated result of the multi-view image on the terminal device server 101, 102, and 103.

[0026] The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server 105 is software, it may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or as a single software or software module.

[0027] Figure 2 The following is a flowchart of a method for generating a multi-view image provided in an embodiment of the present specification. From a program perspective, the execution subject of the process can be a program installed on an application server or an application terminal. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 1 As shown, the method includes:

[0028] Step S201: The video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross normalization technique;

[0029] In the embodiments of the present specification, the images to be processed are images of different perspectives of the static object to be processed. In a specific embodiment, the images to be processed include at least three perspectives, specifically, a front image, a left side image, and a right side image. Of course, the more angles of the images to be processed, the better. When selecting the angles of the images to be processed, the angles of typical parts of the images to be processed should be included, and if there are multiple angles, it is best if the angles are evenly distributed.

[0030] In the embodiments of this specification, the video generation model is an artificial intelligence model based on deep learning technology. In the embodiments of this specification, the video generation model can be Zero123, One-2-3-45++, SyncDreamer, ImageDream, iNVS, etc. The specific model of the video generation model is not limited in this application. The video generation model can be any large model that can realize multi-view image synthesis, as well as other artificial intelligence models.

[0031] In the embodiment of the present specification, the video generation model has a temporal diffusion characteristic.

[0032] In the embodiment of this specification, the video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross normalization technology, specifically including:

[0033] By means of a lightweight encoder, the viewing angle information corresponding to the image to be processed is embedded into the initial multi-view image to generate the lightweight control signal, wherein the viewing angle information includes a camera position and a camera rotation degree;

[0034] Based on the lightweight control signal and the video generation model, cross normalization is performed to obtain the initial multi-view scene image.

[0035] In the embodiments of this specification, the perspective information is embedded into the generation process through a lightweight Transformer encoder, which reduces the computational complexity and increases the generation speed, thereby reducing the computational complexity.

[0036] In the embodiment of this specification, the lightweight encoder is a linear Transformer;

[0037] The attention mechanism of the linear Transformer is:

[0038] Attention(Q,K,V)=σ(QK T )V

[0039] in,

[0040] σ represents the normalization function;

[0041] Q stands for query;

[0042] K stands for key;

[0043] V represents value;

[0044] T stands for transpose;

[0045] K T represents key transposition;

[0046] QK T represents the dot product operation between the query and the key;

[0047] In the embodiment of this specification, the lightweight encoder uses an exponential linear unit as an activation parameter to increase nonlinearity;

[0048] The similarity calculation formula of the lightweight encoder is:

[0049] σ(Q, K) = φ(Q)·φ(K) T

[0050] in,

[0051] σ represents the normalization function;

[0052] σ(Q, K) represents the similarity score between the query and the key;

[0053] φ represents the transformation function;

[0054] φ(Q) represents the transformation function of the query;

[0055] φ(K) T Transformation function representing key transposition;

[0056] φ(·)=elu(·)+1

[0057] in,

[0058] elu represents the exponential linear activation function.

[0059] In the embodiments of this specification, the exponential linear unit is used as the activation parameter, i.e., the exponential linear activation function elu. The purpose of designing elu is to overcome the disadvantage of ReLU (Rectified Linear Unit) that the derivative is zero in the negative interval, while maintaining the advantages similar to ReLU, such as nonlinearity, sparse activation and other characteristics. Unlike ReLU, ELU does not completely shut down the neuron when the input is negative, but has a minimum non-zero response, which helps alleviate the problem of gradient disappearance.

[0060] In the embodiment of this specification, the video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross normalization technology, specifically including:

[0061] Based on the corresponding feature of the lightweight control signal, generating a mean value of the corresponding feature of the lightweight control signal and a standard deviation of the corresponding feature of the lightweight control signal;

[0062] Based on the different layer features of the video generation model, the mean of the different layer features of the video generation model and the standard deviation of the different layer features of the video generation model are generated

[0063] Generate a fine-tuning parameter based on a mean of a corresponding feature of the lightweight control signal, a standard deviation of the corresponding feature of the lightweight control signal, a mean of features of different layers of the video generation model, a standard deviation of features of different layers of the video generation model, and features of different layers of the video generation model;

[0064] Fine-tuning the fine-tuning parameters to obtain a normalized multi-view image;

[0065] The calculation formula of the fine-tuning parameter is:

[0066]

[0067] in,

[0068] z′ represents the fine-tuning parameter;

[0069] z represents different layer features of the video generation model;

[0070] μ(z) represents the standard deviation of the features at different layers of the video generation model;

[0071] σ(z) represents the mean of the features of different layers of the video generation model;

[0072] σ(c) represents the mean of the corresponding feature of the lightweight control signal;

[0073] μ(c) represents the standard deviation of the corresponding feature of the lightweight control signal.

[0074] Step S203: Optimizing the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image.

[0075] In order to further enhance the consistency of information between different perspectives, the embodiments of this specification provide a key-value constraint method to further optimize the image.

[0076] In the embodiment of this specification, the normalized multi-view image is optimized by key-value constraints to generate an optimized multi-view image, specifically including:

[0077] Performing query and key exchange between perspectives on the normalized multi-perspective image, and constraining it through a consistency loss function to obtain an exchanged multi-perspective image;

[0078] The exchanged multi-view images are optimized layer by layer using a fusion model to generate the optimized multi-view images.

[0079] Through querying and key exchange between perspectives, the information between different perspectives can complement each other and improve the consistency between perspectives. The existence of the consistency loss function can ensure consistent transmission between different perspectives and improve the consistency of information transmission between multiple perspectives.

[0080] In the embodiment of this specification, the formula for querying and exchanging keys between perspectives of the normalized multi-perspective image is:

[0081] Q i =K j ,K i =Q j

[0082] in,

[0083] i represents viewpoint i;

[0084] j represents the view angle j;

[0085] Q i represents the query of perspective i;

[0086] K j represents the key of view j;

[0087] K i The key representing the perspective i;

[0088] Q j represents the query of viewpoint j;

[0089] The consistency loss function expression is:

[0090] L consistency =‖Q i -K j ‖ 2 +‖Q j -K i ‖ 2

[0091] in,

[0092] L consistency represents the consistency loss function;

[0093] ‖·‖ 2 represents the square of Euclidean distance;

[0094] ‖Q i -K j ‖ 2 Represents the query Q i and key K j The square of the Euclidean distance between them;

[0095] ‖Q j -K i ‖ 2 Represents the query Q j and key K i The square of the Euclidean distance between them;

[0096] The fusion model is optimized layer by layer based on the following formula:

[0097] x l+1 =Diffusion(x l )×α+Control(c)×(1-α)

[0098] in,

[0099] x l Represents the features of the lth layer;

[0100] x l+1 Represents the features of the l+1th layer;

[0101] c represents the control signal;

[0102] α represents the fusion parameter;

[0103] Diffusion(x l ) represents the diffusion influence of the features of layer l;

[0104] Control(c) represents the influence of the control signal c.

[0105] In the embodiments of this specification, the existence of the fusion model can integrate the control signal by increasing the value layer by layer, making the feature transmission more coherent while retaining important control information.

[0106] In the embodiment of this specification, the fusion parameter α is [0,1], which is the weight of the control diffusion feature and the control signal. When α is close to 0, the model relies more on the control signal; when α is close to 1, the model relies more on the diffusion feature of the current level. In a specific embodiment, the determination of the fusion parameter α needs to be determined according to the specific business scenario.

[0107] It should be noted that perspective i and perspective j belong to different perspectives.

[0108] Step S205: using a Gaussian rendering method to render the high-consistency multi-view image to generate a high-fidelity and high-consistency multi-view image.

[0109] In the embodiment of this specification, the Gaussian rendering method is used to render the optimized multi-view image to generate a high-fidelity multi-view image, specifically including:

[0110] The optimized multi-view image is rendered by Gaussian sputtering 3D rendering, and is constrained by the signal of the initial multi-view image to generate the high-fidelity multi-view image.

[0111] In the embodiment of this specification, the signal of the initial multi-view image uses object-level token as a signal;

[0112] The generation formula of the high-fidelity multi-view image is:

[0113] I′(v)=∑ i w i ·G(vv i )·I(v i )

[0114] in,

[0115] I′(v) represents the pixel value of the high-fidelity multi-view image;

[0116] w i Represents the weight of each Gaussian;

[0117] G(vv i ) represents the Gaussian kernel function;

[0118] I(v i ) represents the pixel value of the optimized multi-view image;

[0119] v i is the position of the Gaussian body;

[0120] i represents view angle i.

[0121] In Gaussian splat 3D rendering, each object or part of an object in the scene can be represented by one or more Gaussian splats. Each Gaussian has its own position, color, intensity, and other properties, such as texture or transparency. When rendering an image, each Gaussian contributes a certain pixel value to the final image based on the relationship between its position in three-dimensional space and the viewer's perspective.

[0122] The Gaussian Splatter rendering method involves the following steps:

[0123] Determine Gaussian bodies: For each object in the scene, determine a set of Gaussian bodies, each of which represents a part of the object and is assigned corresponding attributes (such as color, transparency, etc.).

[0124] Calculate weights: Calculate a weight w for each Gaussian i , this weight determines the influence of the Gaussian on the image pixel value.

[0125] Apply Gaussian kernel function: Use Gaussian kernel function G(vv i ) to calculate the influence of each Gaussian on a point v in the image, where v is the pixel coordinate in the image, v i is the position of the Gaussian body.

[0126] Synthesized image: Finally, the contributions of all Gaussians are combined to form the final image pixel value I′(v), that is, the final color at each pixel is the result of the weighted superposition of all associated Gaussians.

[0127] In order to further understand the method for generating a multi-view image provided by the embodiments of this specification, it will be described below in conjunction with specific embodiments.

[0128] Figure 3 The following is a flow chart of the principle of the method for generating a multi-view image provided in the embodiment of this specification. Figure 3 As shown in the figure, with the image to be processed as input, the video generation model uses its temporal diffusion characteristics and the view control signal features for cross-normalization processing to generate normalized view control signal features. The normalized view control signal features use cross attention and the video generation model to jointly output the video multi-view scene image. The video multi-view scene image is optimized using the consistency image of multi-level key-value constraints to generate a highly consistent multi-view image; finally, the highly consistent multi-view image is rendered based on object-level Gaussian. Generate a highly guaranteed and highly consistent multi-view image.

[0129] The method for generating a multi-view image provided in the embodiment of the present specification generates an initial multi-view scene image of the image to be processed through a video generation model using a lightweight control signal and cross-normalization technology; optimizes the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image; and renders the highly consistent multi-view image using a Gaussian rendering method to generate a high-fidelity, high-consistent multi-view image, which can ensure that the output image has a high degree of visual consistency and detail fidelity, and can improve the realism and consistency of the image.

[0130] The above content describes in detail a method for generating a multi-view image. Correspondingly, this specification also provides a device for generating a multi-view image. Figure 4 shown. Figure 4 A schematic diagram of a multi-view image generation device provided in an embodiment of this specification, the generation device comprising:

[0131] Initialization module 401, the video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross normalization technique;

[0132] An optimization module 403 optimizes the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image;

[0133] The rendering module 405 renders the high-consistency multi-view image by using a Gaussian rendering method to generate a high-fidelity high-consistency multi-view image.

[0134] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0136] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0137] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0138] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0140] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for generating a multi-view image, characterized in that: The generation method comprises: The video generation model generates the initial multi-view scene image of the image to be processed through lightweight control signals and cross normalization technology; By means of key-value constraints, the initial multi-view scene image is optimized to generate a highly consistent multi-view image; The high-consistency multi-view image is rendered using a Gaussian rendering method to generate a high-fidelity and high-consistency multi-view image.

2. The generation method according to claim 1, characterized in that: The video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross normalization technology, specifically including: By means of a lightweight encoder, the viewing angle information corresponding to the image to be processed is embedded into the initial multi-view image to generate the lightweight control signal, wherein the viewing angle information includes a camera position and a camera rotation degree; Based on the lightweight control signal and the video generation model, cross normalization is performed to obtain the initial multi-view scene image.

3. The generation method according to claim 2, characterized in that: The lightweight encoder is a linear Transformer; The attention mechanism of the linear Transformer is: Attention(Q,K,V)=σ(QK T )V in, σ represents the normalization function; Q stands for query; K represents a key; V represents value; T stands for transpose; K T represents key transposition; QK T Represents a dot product operation between a query and a key.

4. The generation method according to claim 3, characterized in that: The lightweight encoder uses an exponential linear unit as an activation parameter to increase nonlinearity; The similarity calculation formula of the lightweight encoder is: σ(Q,K)=φ(Q)·φ(K) T in, σ represents the normalization function; σ(Q, K) represents the similarity score between the query and the key; φ represents the transformation function; φ(Q) represents the transformation function of the query; φ(K) T Transformation function representing key transposition; φ(·)=elu(·)+1 in, elu represents the exponential linear activation function.

5. The generation method according to claim 1, characterized in that: The video generation model generates an initial multi-view scene image of the image to be processed through a lightweight control signal and a cross normalization technology, specifically including: Based on the corresponding feature of the lightweight control signal, generating a mean value of the corresponding feature of the lightweight control signal and a standard deviation of the corresponding feature of the lightweight control signal; Based on the different layer features of the video generation model, the mean of the different layer features of the video generation model and the standard deviation of the different layer features of the video generation model are generated Generate a fine-tuning parameter based on a mean of a corresponding feature of the lightweight control signal, a standard deviation of the corresponding feature of the lightweight control signal, a mean of features of different layers of the video generation model, a standard deviation of features of different layers of the video generation model, and features of different layers of the video generation model; Fine-tuning the fine-tuning parameters to obtain a normalized multi-view image; The calculation formula of the fine-tuning parameter is: in, z' represents the fine-tuning parameter; z represents different layer features of the video generation model; μ(z) represents the standard deviation of the features at different layers of the video generation model; σ(z) represents the mean of the features of different layers of the video generation model; σ(c) represents the mean of the corresponding feature of the lightweight control signal; μ(c) represents the standard deviation of the corresponding feature of the lightweight control signal.

6. The generation method according to claim 1, characterized in that: The step of optimizing the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image specifically includes: Performing query and key exchange between perspectives on the initial multi-perspective scene image, and constraining it through a consistency loss function to obtain an exchanged multi-perspective image; The exchanged multi-view images are optimized layer by layer using a fusion model to generate the optimized highly consistent multi-view images.

7. The generation method according to claim 6, characterized in that: The formula for querying and exchanging keys between perspectives of the initial multi-perspective scene image is: Q i =K j ,K i =Q j in, i represents viewpoint i; j represents the view angle j; Q i represents the query of perspective i; K j represents the key of view j; K i The key representing the perspective i; Q j represents the query of viewpoint j; The consistency loss function expression is: L consistency =||Q i -K j || 2 +||Q j -K i || 2 in, L consistency represents the consistency loss function; ||·|| 2 represents the square of Euclidean distance; ||Q i -K j || 2 Represents the query Q i and key K j The square of the Euclidean distance between them; ||Q j -K i || 2 Represents the query Q j and key K i The square of the Euclidean distance between them; The fusion model is optimized layer by layer based on the following formula: x l+1 =Diffusion(x l )×α+Control(c)×(1-α) in, x l Represents the features of the lth layer; x l+1 Represents the features of the l+1th layer; c represents the control signal; α represents the fusion parameter; Diffusion(x l ) represents the diffusion influence of the features of layer l; Control(c) represents the influence of the control signal c.

8. The generation method according to claim 1, characterized in that: The method of using Gaussian rendering to render the high-consistency multi-view image to generate a high-fidelity high-consistency multi-view image specifically includes: The high-fidelity and high-consistency multi-view image is rendered by Gaussian sputtering 3D rendering and constrained by the signal of the initial multi-view scene image to generate the high-fidelity and high-consistency multi-view image.

9. The generation method according to claim 8, characterized in that: The signal of the initial multi-view image uses an object-level token as a signal; The generation formula of the high-fidelity and high-consistency multi-view image is: I'(v)=∑ i w i ·G(vv i )·I(v i ) in, I'(v) represents the pixel value of the high-fidelity and high-consistency multi-view image; w i Represents the weight of each Gaussian; G(vv i ) represents the Gaussian kernel function; I(v i ) represents the pixel value of the initial multi-view scene image; v i is the position of the Gaussian body; i represents view angle i.

10. A device for generating a multi-view image, characterized in that: The generating device comprises: Initialization module: The video generation model generates the initial multi-view scene image of the image to be processed through lightweight control signals and cross normalization technology; An optimization module optimizes the initial multi-view scene image through key-value constraints to generate a highly consistent multi-view image; The rendering module uses a Gaussian rendering method to render the high-consistency multi-view image to generate a high-fidelity and high-consistency multi-view image.

Images