Space element video production method and system based on 3D Gaussian splashing

Abstract

The invention discloses a space element video production method and system based on 3D Gaussian splashing, and the method comprises the following steps: S10, carrying out media decoding: obtaining multimedia information through setting different input modes, and building a multimedia data model; s20, media preview setting is carried out, the decoded multimedia information is obtained, setting is carried out according to preview requirements, and the preview requirements comprise real-time playing and frame-by-frame preview; s30, performing media processing for the first time: setting the decoded multimedia data and spatial element data according to processing requirements after obtaining the decoded multimedia data and spatial element data, and selecting a spatial mark point in the obtained preview picture; s40, carrying out frame extraction preprocessing and modeling; s50, space element reconstruction three-dimensional rendering: converting an input space element model (such as an OBJ format) into a second 3D Gaussian model through a Gaussian splashing technology, wherein the second 3D Gaussian model is called a space element model; s60, performing spatial calculation rendering; s70, performing media processing for the second time; and S80, carrying out media coding.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer graphics, and in particular relates to a method and system for producing space element videos based on 3D Gaussian splashing. Background Art

[0002] 3D Gaussian splatting is an advanced algorithm for 3D data processing and rendering. Its main principle is to process points in 3D space through Gaussian functions to generate smooth and natural visual effects. Compared with traditional 3D data processing algorithms, on the one hand, 3D Gaussian splatting technology can avoid the problems of edge sharpening or detail loss caused by traditional interpolation and surface fitting algorithms; on the other hand, in the case of multimedia video production combined with 3D models, a large number of video synthesis, tracking matching, and image superposition of 3D models processed by 3D Gaussian splatting algorithms will be involved.

[0003] Spatial elements refer to the basic building blocks with geometric characteristics and physical properties in three-dimensional space. These spatial elements include not only traditional three-dimensional models such as OBJ, FBX, GLB, etc., but also various objects that can be expressed and interacted in three-dimensional space such as scenes, objects, characters, etc.

[0004] At present, domestic multimedia producers mainly rely on overseas video post-production software such as Adobe After Effects, Nuke, Davinci Resolve and related plug-ins when creating videos based on spatial elements. They combine multiple video sources, spatial elements and other media elements through video synthesis technology to generate a continuous video output. However, in this process, in order to ensure that the relative position of the spatial elements and the video screen remains constant, the video needs to be processed frame by frame or interpolated, which takes a long time. Summary of the invention

[0005] In view of the above problems, the present invention provides a method and system for producing spatial element videos based on 3D Gaussian splashing, which are used to solve the problem of high integration of spatial elements and videos in a dynamic environment.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A first aspect of the present invention provides a method for producing a spatial element video based on 3D Gaussian splashing, comprising the following steps:

[0008] S10, performing media decoding: acquiring multimedia information by setting different input modes and establishing a multimedia data model; wherein the different input modes include: parsing and acquiring video frame information according to the uploaded video file; and parsing and acquiring basic three-dimensional data according to the uploaded spatial element file; decoding the input video data into frame-by-frame image data and raw audio data; each video material corresponds to a decoding channel, and the content or information on the channel is kept unchanged;

[0009] S20, performing media preview settings: obtaining the decoded multimedia information and performing settings according to preview requirements, wherein the preview requirements include real-time playback and frame-by-frame preview;

[0010] S30, performing the first media processing: after obtaining the decoded multimedia data and spatial element data, setting them according to the processing requirements, selecting a spatial marker point in the obtained preview image, wherein the spatial marker point refers to a marker point in the multimedia image content for placing the spatial element; wherein the processing requirements include mixing, color correction, special effect addition, color matching and light effect adjustment;

[0011] S40, frame extraction preprocessing and modeling: extracting and preprocessing the input video frame by frame to restore the real three-dimensional spatial structure of the video scene, establishing a three-dimensional model of the video through Gaussian splashing technology, and generating a first 3D Gaussian model corresponding to the input video space. The 3D Gaussian model is composed of spatial element point clouds or voxels, which is called a video space model;

[0012] S50, spatial element reconstruction three-dimensional rendering: convert the input spatial element model (such as OBJ format) into a second 3D Gaussian model, called the spatial element model, through the Gaussian splash technology; select the embedding position of the spatial element from the video spatial model; embed the spatial element into the selected spatial coordinates and generate a fused spatial model; render and display the result through a three-dimensional engine; for each input point or voxel, calculate the Gaussian function value in the surrounding space; where a point refers to a discrete point data in the three-dimensional space, a large number of spatial points constitute a point cloud, and a voxel refers to the smallest unit data represented by a discrete point in the three-dimensional space; adjust the parameters of the Gaussian function according to the density of the point and the characteristics of the surrounding environment; calculate the weight value of each point according to the Gaussian function value and the attributes of the point for subsequent interpolation or surface reconstruction; use the calculated weight value to interpolate or surface fit the three-dimensional space to generate a continuous and smooth three-dimensional surface or volume representation;

[0013] S60, perform spatial calculation and rendering: based on the media data after the first media processing, obtain the processed media image data, calculate the depth information according to the two-dimensional image, and obtain the video camera trajectory; calculate the spatial position and posture of the fused space model, the space element model, and the video space model; perform smoothing and density calculation on the three-dimensional point cloud data; render the camera trajectory image sequence corresponding to the fused space model, the space element model, and the video space model based on the camera trajectory; calculate the difference between the image sequences of the above three models to generate a mask image sequence; reversely calculate and generate a virtual camera of the corresponding position according to the trajectory image, and bind each angle of the fused space model, the space element model, and the video space model to the corresponding virtual camera;

[0014] S70, performing a second media processing: obtaining a positional relationship between the 3D Gaussian model and the marker point; and a rendering result of the rendered 3D Gaussian model; setting the multimedia data and the rendered image sequence result part as a time axis synthesis, synthesizing according to a required time axis, and outputting the media video data after the time axis synthesis;

[0015] S80, perform media encoding: encode the processed video data by different encoding methods, wherein the different encoding methods include: compressing and encoding the video data according to the specified encoding format and parameters; compressing and encoding the audio data according to the specified encoding format and parameters; packaging the compressed and encoded multimedia data into a specified file format.

[0016] In a possible implementation, in S20, if the media preview is set to real-time playback, rendering is performed according to the required resolution and frame rate, and a real-time preview effect is output;

[0017] In a possible implementation, in S20, if the media preview is set to preview frame by frame, the media is displayed frame by frame according to the timeline control, and a frame preview effect is output.

[0018] In a possible implementation, the step of selecting the embedding position of the spatial element from the video space model in S50 specifically includes: obtaining and selecting the point cloud or voxel data of the spatial element in the video space model corresponding to the input video; the spatial coordinates of the selected point cloud or voxel represent the position where the spatial element is finally presented in the media result, that is, the embedding position of the spatial element.

[0019] A second aspect of the present invention provides a spatial element video production system based on 3D Gaussian splashing, comprising:

[0020] The media decoding module is used to obtain multimedia information and establish a multimedia data model by setting different input methods; the different input methods include: parsing and obtaining video frame information according to the uploaded video file; and parsing and obtaining basic three-dimensional data according to the uploaded spatial element file; decoding the input video data into frame-by-frame image data and audio raw data; each video material corresponds to a decoding channel, and the content or information on the channel is kept unchanged;

[0021] A media preview setting module is used to obtain the decoded multimedia information and set it according to the preview requirements, which include real-time playback and frame-by-frame preview;

[0022] The media processing module is used to perform the first media processing. After obtaining the decoded multimedia data and spatial element data, it is set according to the processing requirements and selects the spatial mark point in the obtained preview picture, where the spatial mark point refers to the mark point for placing the spatial element in the multimedia picture content; the processing requirements include mixing, color correction, special effect addition, color matching and light effect adjustment;

[0023] The frame extraction preprocessing and modeling module is used to extract and preprocess the input video frame by frame to restore the real three-dimensional spatial structure of the video scene, establish a three-dimensional model of the video through the Gaussian splashing technology, and generate a first 3D Gaussian model corresponding to the input video space. The 3D Gaussian model is composed of spatial element point clouds or voxels, which is called a video space model;

[0024] The spatial element reconstruction three-dimensional rendering module is used to convert the input spatial element model (such as OBJ format) into a second 3D Gaussian model, called the spatial element model, through the Gaussian splash technology; select the embedding position of the spatial element from the video spatial model; embed the spatial element into the selected spatial coordinates and generate a fused spatial model; render and display the results through a three-dimensional engine; for each input point or voxel, calculate the Gaussian function value in the surrounding space; where a point refers to a discrete point data in the three-dimensional space, a large number of spatial points constitute a point cloud, and a voxel refers to the smallest unit data represented by a discrete point in the three-dimensional space; adjust the parameters of the Gaussian function according to the density of the point and the characteristics of the surrounding environment; calculate the weight value of each point according to the Gaussian function value and the attributes of the point for subsequent interpolation or surface reconstruction; use the calculated weight value to interpolate or surface fit the three-dimensional space to generate a continuous and smooth three-dimensional surface or volume representation;

[0025] A spatial calculation and rendering module is used to obtain processed media image data based on the media data after the first media processing, calculate the depth information according to the two-dimensional image, and obtain the video camera trajectory; calculate the spatial position and posture of the fused space model, the space element model, and the video space model; perform smoothing and density calculation on the three-dimensional point cloud data; render the camera trajectory image sequence corresponding to the fused space model, the space element model, and the video space model based on the camera trajectory; calculate the difference between the image sequences of the above three models to generate a mask image sequence; reversely calculate and generate a virtual camera of the corresponding position according to the trajectory image, and bind each angle of the fused space model, the space element model, and the video space model to the corresponding virtual camera;

[0026] The media processing module is used to further perform a second media processing, obtain the positional relationship between the 3D Gaussian model and the marker point; and the rendered 3D Gaussian model rendering result; set the multimedia data and the rendered image sequence result part as timeline synthesis, synthesize according to the required timeline, and output the media video data after the timeline synthesis;

[0027] The media encoding module is used to encode the processed multimedia data through different encoding methods, where the different encoding methods include: compressing and encoding the video data according to the specified encoding format and parameters; compressing and encoding the audio data according to the specified encoding format and parameters; and packaging the compressed and encoded multimedia data into a specified file format.

[0028] In a possible implementation, in the media preview setting module, if the media preview is set to real-time playback, rendering is performed according to the required resolution and frame rate, and a real-time preview effect is output;

[0029] In a possible implementation, in the media preview setting module, if the media preview is set to frame-by-frame preview, frame-by-frame display is performed according to the timeline control, and a frame preview effect is output.

[0030] In a possible implementation, the selection of the embedding position of the spatial element from the video space model in the spatial element reconstruction three-dimensional rendering module specifically includes: acquiring and selecting the point cloud or voxel data of the spatial element in the video space model corresponding to the input video; the spatial coordinates of the selected point cloud or voxel represent the final position of the spatial element presented in the media result, that is, the embedding position of the spatial element.

[0031] The present invention has the following beneficial effects:

[0032] (1) It can achieve a high degree of integration between 3D models and video files, and end-to-end link transmission from data acquisition to final rendering, which can be used in a variety of application scenarios such as film and television special effects, virtual reality, augmented reality, etc., adding a new technical implementation method for 3D film and television post-production special effects. Compared with traditional 3D post-production, the present invention has more advantages in integration and can provide a more immersive viewing experience;

[0033] (2) The 3D model embedded in the video supports model angle adaptation under dynamic viewing angle changes through precise spatial calculation, so that the relative position of the model and the video environment under any lens remains constant, thereby meeting different production requirements. Compared with the traditional 3D model embedding method that requires frame-by-frame lens calibration and interpolation for smooth transition, this method realizes the automation of the process based on AI algorithm, greatly reducing production costs and improving production efficiency;

[0034] (3) Based on Gaussian splash technology and multiple rendering modes, it can maintain high-quality visual effects under different lighting and environmental conditions, enhance the user's immersive viewing experience, and is compatible with traditional 3D rendering modes and workflows. It has strong scalability and reduces the migration cost brought about by using new methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of a method for producing a spatial element video based on 3D Gaussian splashing according to an embodiment of the present invention;

[0036] Figure 2 This is a principle block diagram of a spatial element video production system based on 3D Gaussian splashing according to another embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] See also Figure 1 , which is a flowchart of a method for producing a spatial element video based on 3D Gaussian splashing according to an embodiment of the present invention, includes the following steps:

[0039] S10, performing media decoding: acquiring multimedia information by setting different input modes and establishing a multimedia data model; wherein the different input modes include: parsing and acquiring video frame information according to the uploaded video file; and parsing and acquiring basic three-dimensional data according to the uploaded spatial element file (such as a three-dimensional model, etc.); decoding the input video data into frame-by-frame image data and raw audio data; each video material corresponds to a decoding channel, and the content or information on the channel is kept unchanged;

[0040] S20, performing media preview settings: obtaining the decoded multimedia information and performing settings according to preview requirements, wherein the preview requirements include real-time playback and frame-by-frame preview;

[0041] S30, performing the first media processing: after obtaining the decoded multimedia data and spatial element data, setting them according to the processing requirements, selecting a spatial marker point in the obtained preview image, wherein the spatial marker point refers to a marker point in the multimedia image content for placing the spatial element; wherein the processing requirements include mixing, color correction, special effect addition, color matching and light effect adjustment;

[0042] S40, frame extraction preprocessing and modeling: extracting and preprocessing the input video frame by frame to restore the real three-dimensional spatial structure of the video scene, establishing a three-dimensional model of the video through Gaussian splashing technology, and generating a first 3D Gaussian model corresponding to the input video space. The 3D Gaussian model is composed of spatial element point clouds or voxels, which is called a video space model;

[0043] S50, spatial element reconstruction three-dimensional rendering: convert the input spatial element model (such as OBJ format) into a second 3D Gaussian model, called the spatial element model, through the Gaussian splash technology; select the embedding position of the spatial element from the video spatial model; embed the spatial element into the selected spatial coordinates and generate a fused spatial model; render and display the result through a three-dimensional engine; for each input point or voxel, calculate the Gaussian function value in the surrounding space; where a point refers to a discrete point data in the three-dimensional space, a large number of spatial points constitute a point cloud, and a voxel refers to the smallest unit data represented by a discrete point in the three-dimensional space; adjust the parameters of the Gaussian function according to the density of the point and the characteristics of the surrounding environment; calculate the weight value of each point according to the Gaussian function value and the attributes of the point for subsequent interpolation or surface reconstruction; use the calculated weight value to interpolate or surface fit the three-dimensional space to generate a continuous and smooth three-dimensional surface or volume representation;

[0044] S60, perform spatial calculation and rendering: based on the media data after the first media processing, obtain the processed media image data, calculate the depth information according to the two-dimensional image, and obtain the video camera trajectory; calculate the spatial position and posture of the fused space model, the space element model, and the video space model; perform smoothing and density calculation on the three-dimensional point cloud data; render the camera trajectory image sequence corresponding to the fused space model, the space element model, and the video space model based on the camera trajectory; calculate the difference between the image sequences of the above three models to generate a mask image sequence; reversely calculate and generate a virtual camera of the corresponding position according to the trajectory image, and bind each angle of the fused space model, the space element model, and the video space model to the corresponding virtual camera;

[0045] S70, performing a second media processing: obtaining a positional relationship between the 3D Gaussian model and the marker point; and a rendering result of the rendered 3D Gaussian model; setting the multimedia data and the rendered image sequence result part as a time axis synthesis, synthesizing according to a required time axis, and outputting the media video data after the time axis synthesis;

[0046] S80, perform media encoding: encode the processed video data by different encoding methods, wherein the different encoding methods include: compressing and encoding the video data according to the specified encoding format and parameters; compressing and encoding the audio data according to the specified encoding format and parameters; packaging the compressed and encoded multimedia data into a specified file format.

[0047] In a specific application example, in S20, if the media preview is set to real-time playback, it is rendered according to the required resolution and frame rate, and the real-time preview effect is output. If the media preview is set to frame-by-frame preview, it is displayed frame by frame according to the timeline control, and the frame preview effect is output.

[0048] In a specific application example, selecting the embedding position of the spatial element from the video space model in S50 specifically includes: obtaining and selecting the point cloud or voxel data of the spatial element in the video space model corresponding to the input video; the spatial coordinates of the selected point cloud or voxel represent the final position of the spatial element in the media result, that is, the embedding position of the spatial element.

[0049] For method embodiments, see Figure 2 , which is a principle block diagram of a spatial element video production system based on 3D Gaussian splashing according to another embodiment of the present invention, includes a media decoding module 101, a media preview setting module 102, a media processing module 103, a frame extraction preprocessing and modeling module 104, a spatial element reconstruction 3D rendering module 105, a spatial calculation rendering module 106, and a media encoding module 107, wherein,

[0050] The media decoding module 101 is used to obtain multimedia information and establish a multimedia data model by setting different input modes; wherein the different input modes include: parsing and obtaining video frame information according to the uploaded video file; and parsing and obtaining basic three-dimensional data according to the uploaded spatial element file; decoding the input video data into frame-by-frame image data and raw audio data; each video material corresponds to a decoding channel, and the content or information on the channel is kept unchanged;

[0051] The media preview setting module 102 is used to obtain the decoded multimedia information and set it according to the preview requirements, which include real-time playback and frame-by-frame preview;

[0052] The media processing module 103 is used to perform the first media processing, and after obtaining the decoded multimedia data and spatial element data, it is set according to the processing requirements, and a spatial marker point is selected in the obtained preview picture, wherein the spatial marker point refers to the marker point for placing the spatial element in the multimedia picture content; wherein the processing requirements include mixing, color correction, special effect addition, color matching and light effect adjustment;

[0053] The frame extraction preprocessing and modeling module 104 is used to extract and preprocess the input video frame by frame to restore the real three-dimensional spatial structure of the video scene, establish a three-dimensional model of the video through the Gaussian splashing technology, and generate a first 3D Gaussian model corresponding to the input video space. The first 3D Gaussian model is composed of spatial element point clouds or voxels, which is called a video space model;

[0054] The spatial element reconstruction three-dimensional rendering module 105 is used to convert the input spatial element model (such as OBJ format) into a second 3D Gaussian model, called the spatial element model, through the Gaussian splash technology; select the embedding position of the spatial element from the video spatial model; embed the spatial element into the selected spatial coordinates and generate a fused spatial model; render and display the result through a three-dimensional engine; for each input point or voxel, calculate the Gaussian function value in the surrounding space; wherein a point refers to a discrete point data in the three-dimensional space, a large number of spatial points constitute a point cloud, and a voxel refers to the smallest unit data represented by a discrete point in the three-dimensional space; adjust the parameters of the Gaussian function according to the density of the point and the characteristics of the surrounding environment; calculate the weight value of each point according to the Gaussian function value and the attributes of the point, for subsequent interpolation or surface reconstruction; use the calculated weight value to interpolate or surface fit the three-dimensional space to generate a continuous and smooth three-dimensional surface or volume representation;

[0055] The spatial calculation and rendering module 106 is used to obtain the processed media image data based on the media data after the first media processing, calculate the depth information according to the two-dimensional image, and obtain the video camera trajectory; calculate the spatial position and posture of the fused space model, the spatial element model, and the video space model; perform smoothing and density calculation on the three-dimensional point cloud data; render the camera trajectory image sequence corresponding to the fused space model, the spatial element model, and the video space model based on the camera trajectory; calculate the difference between the image sequences of the above three models to generate a mask image sequence; reversely calculate and generate a virtual camera of the corresponding position according to the trajectory image, and bind each angle of the fused space model, the spatial element model, and the video space model to the corresponding virtual camera;

[0056] The media processing module 103 is used to further perform a second media processing, obtain the positional relationship between the 3D Gaussian model and the marker point; and the rendered 3D Gaussian model rendering result; set the multimedia data and the rendered image sequence result part as a time axis synthesis, synthesize according to the required time axis, and output the media video data after the time axis synthesis;

[0057] The media encoding module 107 is used to encode the processed multimedia data using different encoding methods, wherein the different encoding methods include: compressing and encoding the video data according to the specified encoding format and parameters; compressing and encoding the audio data according to the specified encoding format and parameters; and packaging the compressed and encoded multimedia data into a specified file format.

[0058] In a specific application example, in the media preview setting module 102, if the media preview is set to real-time playback, it is rendered according to the required resolution and frame rate, and the real-time preview effect is output; if the media preview is set to frame-by-frame preview, it is displayed frame by frame according to the timeline control, and the frame preview effect is output.

[0059] In a specific application example, the selection of the embedding position of the spatial element from the video space model in the spatial element reconstruction three-dimensional rendering module specifically includes: obtaining and selecting the point cloud or voxel data of the spatial element in the video space model corresponding to the input video; the spatial coordinates of the selected point cloud or voxel represent the final position of the spatial element presented in the media result, that is, the embedding position of the spatial element.

[0060] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A method for producing a spatial element video based on 3D Gaussian splashing, characterized in that: The following steps are involved: S10, performing media decoding: acquiring multimedia information by setting different input methods and establishing a multimedia data model; The different input methods include: parsing the uploaded video file to obtain video frame information; parsing the uploaded spatial element file to obtain basic three-dimensional data; decoding the input video data into frame-by-frame image data and raw audio data; each video material corresponds to a decoding channel, and the content or information on the channel is kept unchanged; S20, performing media preview settings: obtaining the decoded multimedia information and performing settings according to preview requirements, wherein the preview requirements include real-time playback and frame-by-frame preview; S30, performing the first media processing: after obtaining the decoded multimedia data and spatial element data, setting them according to the processing requirements, selecting a spatial marker point in the obtained preview image, wherein the spatial marker point refers to a marker point in the multimedia image content for placing the spatial element; wherein the processing requirements include mixing, color correction, special effect addition, color matching and light effect adjustment; S40, frame extraction preprocessing and modeling: extracting and preprocessing the input video frame by frame to restore the real three-dimensional spatial structure of the video scene, establishing a three-dimensional model of the video through Gaussian splashing technology, and generating a first 3D Gaussian model corresponding to the input video space. The 3D Gaussian model is composed of spatial element point clouds or voxels, which is called a video space model; S50, spatial element reconstruction three-dimensional rendering: convert the input spatial element model (such as OBJ format) into a second 3D Gaussian model, called the spatial element model, through the Gaussian splash technology; select the embedding position of the spatial element from the video spatial model; embed the spatial element into the selected spatial coordinates and generate a fused spatial model; render and display the result through a three-dimensional engine; for each input point or voxel, calculate the Gaussian function value in the surrounding space; where a point refers to a discrete point data in the three-dimensional space, a large number of spatial points constitute a point cloud, and a voxel refers to the smallest unit data represented by a discrete point in the three-dimensional space; adjust the parameters of the Gaussian function according to the density of the point and the characteristics of the surrounding environment; calculate the weight value of each point according to the Gaussian function value and the attributes of the point for subsequent interpolation or surface reconstruction; use the calculated weight value to interpolate or surface fit the three-dimensional space to generate a continuous and smooth three-dimensional surface or volume representation; S60, perform spatial calculation and rendering: based on the media data after the first media processing, obtain the processed media image data, calculate the depth information according to the two-dimensional image, and obtain the video camera trajectory; calculate the spatial position and posture of the fused space model, the space element model, and the video space model; perform smoothing and density calculation on the three-dimensional point cloud data; render the camera trajectory image sequence corresponding to the fused space model, the space element model, and the video space model based on the camera trajectory; calculate the difference between the image sequences of the above three models to generate a mask image sequence; reversely calculate and generate a virtual camera of the corresponding position according to the trajectory image, and bind each angle of the fused space model, the space element model, and the video space model to the corresponding virtual camera; S70, performing a second media processing: obtaining a positional relationship between the 3D Gaussian model and the marker point; and a rendering result of the rendered 3D Gaussian model; setting the multimedia data and the rendered image sequence result part as a time axis synthesis, synthesizing according to a required time axis, and outputting the media video data after the time axis synthesis; S80, perform media encoding: encode the processed video data by different encoding methods, wherein the different encoding methods include: compressing and encoding the video data according to the specified encoding format and parameters; compressing and encoding the audio data according to the specified encoding format and parameters; packaging the compressed and encoded multimedia data into a specified file format.

2. The method for producing a spatial element video based on 3D Gaussian splashing according to claim 1, characterized in that: In S20, if the media preview is set to real-time playback, rendering is performed according to the required resolution and frame rate, and the real-time preview effect is output.

3. The method for producing a spatial element video based on 3D Gaussian splashing according to claim 1, characterized in that: In S20, if the media preview is set to preview frame by frame, the media is displayed frame by frame according to the timeline control, and the frame preview effect is output.

4. The method for producing a spatial element video based on 3D Gaussian splashing according to claim 1, characterized in that: The step of selecting the embedding position of the spatial element from the video space model in S50 specifically includes: obtaining and selecting the point cloud or voxel data of the spatial element in the video space model corresponding to the input video; the spatial coordinates of the selected point cloud or voxel represent the position where the spatial element is finally presented in the media result, that is, the embedding position of the spatial element.

5. A spatial element video production system based on 3D Gaussian splashing, characterized in that: include: The media decoding module is used to obtain multimedia information by setting different input methods and establish a multimedia data model; The different input methods include: parsing the uploaded video file to obtain video frame information; parsing the uploaded spatial element file to obtain basic three-dimensional data; decoding the input video data into frame-by-frame image data and raw audio data; each video material corresponds to a decoding channel, and the content or information on the channel is kept unchanged; A media preview setting module is used to obtain the decoded multimedia information and set it according to the preview requirements, which include real-time playback and frame-by-frame preview; The media processing module is used to perform the first media processing. After obtaining the decoded multimedia data and spatial element data, it is set according to the processing requirements and selects the spatial mark point in the obtained preview picture, where the spatial mark point refers to the mark point for placing the spatial element in the multimedia picture content; the processing requirements include mixing, color correction, special effect addition, color matching and light effect adjustment; The frame extraction preprocessing and modeling module is used to extract and preprocess the input video frame by frame to restore the real three-dimensional spatial structure of the video scene, establish a three-dimensional model of the video through the Gaussian splashing technology, and generate a first 3D Gaussian model corresponding to the input video space. The 3D Gaussian model is composed of spatial element point clouds or voxels, which is called a video space model; The spatial element reconstruction three-dimensional rendering module is used to convert the input spatial element model (such as OBJ format) into a second 3D Gaussian model, called the spatial element model, through the Gaussian splash technology; select the embedding position of the spatial element from the video spatial model; embed the spatial element into the selected spatial coordinates and generate a fused spatial model; render and display the results through a three-dimensional engine; for each input point or voxel, calculate the Gaussian function value in the surrounding space; where a point refers to a discrete point data in the three-dimensional space, a large number of spatial points constitute a point cloud, and a voxel refers to the smallest unit data represented by a discrete point in the three-dimensional space; adjust the parameters of the Gaussian function according to the density of the point and the characteristics of the surrounding environment; calculate the weight value of each point according to the Gaussian function value and the attributes of the point for subsequent interpolation or surface reconstruction; use the calculated weight value to interpolate or surface fit the three-dimensional space to generate a continuous and smooth three-dimensional surface or volume representation; A spatial calculation and rendering module is used to obtain processed media image data based on the media data after the first media processing, calculate the depth information according to the two-dimensional image, and obtain the video camera trajectory; calculate the spatial position and posture of the fused space model, the space element model, and the video space model; perform smoothing and density calculation on the three-dimensional point cloud data; render the camera trajectory image sequence corresponding to the fused space model, the space element model, and the video space model based on the camera trajectory; calculate the difference between the image sequences of the above three models to generate a mask image sequence; reversely calculate and generate a virtual camera of the corresponding position according to the trajectory image, and bind each angle of the fused space model, the space element model, and the video space model to the corresponding virtual camera; The media processing module is used to further perform a second media processing, obtain the positional relationship between the 3D Gaussian model and the marker point; and the rendered 3D Gaussian model rendering result; set the multimedia data and the rendered image sequence result part as timeline synthesis, synthesize according to the required timeline, and output the media video data after the timeline synthesis; The media encoding module is used to encode the processed multimedia data through different encoding methods, where the different encoding methods include: compressing and encoding the video data according to the specified encoding format and parameters; compressing and encoding the audio data according to the specified encoding format and parameters; and packaging the compressed and encoded multimedia data into a specified file format.

6. The spatial element video production system based on 3D Gaussian splashing as claimed in claim 5, characterized in that: In the media preview setting module, if the media preview is set to real-time playback, it is rendered according to the required resolution and frame rate, and the real-time preview effect is output.

7. The spatial element video production system based on 3D Gaussian splashing as claimed in claim 5, characterized in that: In the media preview setting module, if the media preview is set to frame-by-frame preview, it will be displayed frame by frame according to the timeline control and the frame preview effect will be output.

8. The spatial element video production system based on 3D Gaussian splashing as claimed in claim 1, characterized in that: The selection of the embedding position of the spatial element from the video space model in the spatial element reconstruction three-dimensional rendering module specifically includes: obtaining and selecting the point cloud or voxel data of the spatial element in the video space model corresponding to the input video; the spatial coordinates of the selected point cloud or voxel represent the final position of the spatial element presented in the media result, that is, the embedding position of the spatial element.

Images