Encoding method and decoding method for 3DGS model, and related apparatus

Abstract

The present application discloses an encoding method and a decoding method for a 3DGS model, and a related apparatus, so as to improve encoding and decoding effects of the 3DGS model. The encoding method comprises: acquiring a 3DGS model; and encoding a plurality of Gaussian spheres in the 3DGS model, obtaining encoded data of the 3DGS model, and encoding the encoded data into a bitstream. The bitstream further comprises indication information of each Gaussian sphere, the indication information being used for indicating a mode for encoding or decoding the corresponding Gaussian sphere. The decoding method comprises: receiving a bitstream; and, on the basis of indication information of each Gaussian sphere, decoding encoded data of the Gaussian sphere, so as to obtain a 3DGS model.

Description

Encoding methods, decoding methods and related devices for 3DGS models

[0001] This application claims priority to Chinese patent application filed on January 20, 2025, with application number 202510089693.4 and entitled "Encoding method, decoding method and related apparatus for 3DGS model", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, and in particular to an encoding method, decoding method and related apparatus for a 3DGS model. Background Technology

[0003] With the rapid development of technology and the wave of digital transformation, digital twin technology has shown great potential in many fields, such as 3D visualization based on digital twins. 3D visualization based on digital twins refers to processing spatial information from the real world to obtain a 3D model, such as a 3D Gaussian splatting (3DGS) model, and then transmitting and rendering the 3DGS model to achieve its visualization. To achieve better 3D visualization effects, 3DGS models typically need to include a large amount of data. Since transmitting and storing large amounts of data presents difficulties, it is necessary to encode and decode the data included in the 3DGS model.

[0004] However, current encoding and decoding methods for 3DGS models have poor encoding and decoding effects. Summary of the Invention

[0005] This application provides an encoding method, a decoding method, and related apparatus for 3DGS models, which are used to provide an encoding and decoding method for 3DGS models to improve the encoding and decoding effect of 3DGS models.

[0006] To achieve the above objectives, the embodiments of this application provide the following technical solutions:

[0007] Firstly, a method for encoding 3DGS models is provided. This method can be executed by an encoding device, or by a component of the encoding device, such as the processor, chip, or chip system of the encoding device. It can also be implemented by a logic module or software that can realize all or part of the functions of the encoding device, such as a client or server application in the encoding device.

[0008] The method includes: acquiring a 3DGS model; then, encoding multiple Gaussian spheres in the 3DGS model to obtain encoded data of the 3DGS model, and encoding the encoded data into a bitstream. The multiple Gaussian spheres include a first Gaussian sphere and a second Gaussian sphere.

[0009] The bitstream also includes first indication information and second indication information. The first indication information indicates the method of encoding or decoding the first Gaussian sphere, and the second indication information indicates the method of encoding or decoding the second Gaussian sphere. That is, after encoding multiple Gaussian spheres in the 3DGS model to obtain the encoded data of the 3DGS model, the indication information of the multiple Gaussian spheres also needs to be encoded into the bitstream so that the indication information can be used in the subsequent decoding stage to determine the method of encoding or decoding each Gaussian sphere.

[0010] Thus, by carrying first and second indication information in the bitstream, the encoding or decoding method of the first Gaussian sphere and the encoding or decoding method of the second Gaussian sphere can be indicated in the subsequent decoding stage.

[0011] In some possible implementations, the first indication information differs from the second indication information. Because the first and second indication information differ, it indicates that the encoding method of the first Gaussian sphere is different from that of the second Gaussian sphere, and consequently, the decoding method of the first Gaussian sphere is also different from that of the second Gaussian sphere. This allows for the use of different encoding and decoding methods for different Gaussian spheres, effectively improving the encoding and decoding performance.

[0012] In some possible implementations, the method further includes: grouping the multiple Gaussian spheres into N Gaussian sphere groups, where N is a positive integer greater than 1. That is, the multiple Gaussian spheres belong to N Gaussian sphere groups.

[0013] In this system, the encoding methods for the N Gaussian sphere groups are different; that is, different Gaussian sphere groups correspond to different encoding methods, with each Gaussian sphere group corresponding to a different encoding method. Thus, using different encoding methods for different groups of Gaussian spheres effectively improves the flexibility of the encoding.

[0014] In some possible implementations, the first Gaussian sphere and the second Gaussian sphere belong to different groups within the N Gaussian sphere groups, such as the first Gaussian sphere belonging to the first Gaussian sphere group and the second Gaussian sphere belonging to the second Gaussian sphere group. This means that the first Gaussian sphere and the second Gaussian sphere correspond to different encoding methods.

[0015] In some possible implementations, the bitstream further includes a range for each of the N Gaussian sphere groups, whereby the range is used to determine the group to which the first Gaussian sphere belongs based on first indication information, and to determine the group to which the second Gaussian sphere belongs based on second indication information. For example, the range of each of the N Gaussian sphere groups can be represented by an index interval. It is worth noting that the index intervals of different groups do not overlap. This allows the index interval of each group to be used to uniquely identify the corresponding group.

[0016] The first indication information indicates the sequence number of the first Gaussian sphere in its group, and the second indication information indicates the sequence number of the second Gaussian sphere in its group. For example, different numerical sequence numbers can be used in the indication information for each Gaussian sphere to represent different groups. This allows each Gaussian sphere to be uniquely identified by its sequence number.

[0017] In some possible implementations, the method further includes: sorting the Gaussian spheres in each of the N Gaussian sphere groups to obtain the sequence number of each Gaussian sphere in its respective group.

[0018] In this system, Gaussian spheres belonging to the same group have consecutive ordinal numbers within that group. This consecutive numbering simplifies the sorting process; simply arranging the Gaussian spheres in numerical order yields the ordinal number of each sphere within its group and the range of that group (e.g., an interval of numbers), thus improving sorting efficiency. Furthermore, consecutive ordinal numbers make data maintenance and management more convenient. They clearly represent each Gaussian sphere, facilitating subsequent identification of its group based on its ordinal number.

[0019] In the above embodiments, a method is provided to indicate the group to which a Gaussian ball belongs based on the sequence number of the Gaussian ball and the range of each Gaussian ball group, so that the Gaussian ball's sequence number and the range of each Gaussian ball group can be used to determine the group to which the Gaussian ball belongs in the subsequent decoding stage, and then the decoding method corresponding to the group to which the group belongs can be used to perform decoding.

[0020] In some possible implementations, the first indication information is used to indicate the group to which the first Gaussian ball belongs, and the second indication information is used to indicate the group to which the second Gaussian ball belongs. For example, different character flag bits can be used in the indication information of each Gaussian ball to represent different groups. The flag bit is typically an integer variable, such as 0 or 1. Taking the first Gaussian ball group and the second Gaussian ball group as examples, 0 can be used to represent the first Gaussian ball group, and 1 can be used to represent the second Gaussian ball group. Thus, through the indication information of each Gaussian ball, the group to which each Gaussian ball belongs can be directly determined, and the encoding or decoding method corresponding to the Gaussian ball can be determined based on the group to which the Gaussian ball belongs.

[0021] In some possible implementations, the bitstream also includes grouping information for N Gaussian sphere groups. This grouping information includes the number of groups, the number of Gaussian spheres in each group, and the encoding information for each group. This allows a subsequent decoding device to decode multiple Gaussian spheres based on the grouping information of the N Gaussian sphere groups, thereby obtaining multiple Gaussian spheres.

[0022] In some possible implementations, encoding multiple Gaussian spheres includes: performing a first quantization on the feature data of a first Gaussian sphere; and performing a second quantization on the feature data of a second Gaussian sphere. The compression ratio of the first quantization is lower than that of the second quantization.

[0023] In the above implementation, a low compression ratio encoding method is used for the Gaussian spheres in the first Gaussian sphere group. A high compression ratio encoding method is used for the Gaussian spheres in the second Gaussian sphere group. For example, the first Gaussian sphere group may be a group with a greater impact on the rendering effect, while the second Gaussian sphere group may be a group with a smaller impact on the rendering effect. Thus, by using different encoding methods for the Gaussian spheres in different Gaussian sphere groups, Gaussian spheres with a greater impact on the rendering effect (i.e., relatively important Gaussian spheres) use a low compression ratio, high bit rate, and low loss encoding method, while Gaussian spheres with a smaller impact on the rendering effect (i.e., relatively unimportant Gaussian spheres) use a high compression ratio, low bit rate, and high loss encoding method. This achieves two goals: firstly, it allows for more effective compression of the 3DGS model while maintaining rendering quality, improving the compression ratio of the 3DGS model; secondly, it reduces the transmission time of the 3DGS model, thus reducing the time spent loading model data from memory during rendering, thereby improving rendering speed.

[0024] In some possible implementations, the feature data of the second Gaussian sphere includes first feature data and second feature data. Accordingly, the feature data of the second Gaussian sphere is subjected to a second quantization, which includes: a third quantization of the first feature data and a fourth quantization of the second feature data.

[0025] The second quantization includes a third quantization of the first feature data and a fourth quantization of the second feature data. The third quantization can be scalar quantization. Accordingly, performing the third quantization on the first feature data includes performing scalar quantization on the first feature data.

[0026] In some possible implementations, the fourth quantization may include scalar quantization and vector quantization. Accordingly, performing the fourth quantization on the second feature data includes: performing scalar quantization on the second feature data and performing vector quantization on the result of the scalar quantization.

[0027] In the above implementation, for the Gaussian spheres in the second Gaussian sphere group, after scalar quantization of the second feature data (such as higher-order sphere covariances) of the Gaussian spheres, vector quantization is further performed, which can further compress the second feature data, thereby achieving further compression of the 3DGS model and improving the compression rate of the 3DGS model.

[0028] In some possible implementations, the fourth quantization may include scalar quantization and vector quantization. Accordingly, performing the fourth quantization on the second feature data includes: performing scalar quantization on the second feature data and performing product quantization on the result of the scalar quantization.

[0029] In the above implementation, for the Gaussian spheres in the second Gaussian sphere group, after scalar quantization of the second feature data (such as higher-order sphere coefficients) of the Gaussian spheres, product quantization is further performed, which can further compress the second feature data, thereby achieving further compression of the 3DGS model and improving the compression rate of the 3DGS model.

[0030] In some possible implementations, the feature data of the second Gaussian sphere includes both first and second feature data, but the bitstream does not contain the second feature data. This means that the second feature data is discarded during the encoding stage. Thus, by discarding the second feature data, further compression of the 3DGS model can be achieved, thereby improving the compression ratio of the 3DGS model.

[0031] Secondly, a decoding method for 3DGS models is provided. This method can be executed by a decoding device, or by a component of the decoding device, such as the processor, chip, or chip system of the decoding device. It can also be implemented by a logic module or software that can realize all or part of the functions of the decoding device, such as the client or server of the application in the decoding device.

[0032] The method includes receiving a bitstream. The bitstream includes encoded data of a 3DGS model, which includes multiple Gaussian spheres, including a first Gaussian sphere and a second Gaussian sphere. The bitstream also includes first indication information and second indication information. The first indication information indicates the encoding or decoding method for the first Gaussian sphere, and the second indication information indicates the encoding or decoding method for the second Gaussian sphere. The first and second indication information are different. Thus, by carrying the first and second indication information in the bitstream, the encoding or decoding method for both the first and second Gaussian spheres can be indicated during the decoding stage.

[0033] The encoded data of the first Gaussian sphere is decoded according to the first instruction information to obtain the first Gaussian sphere. The encoded data of the second Gaussian sphere is decoded according to the second instruction information to obtain the second Gaussian sphere.

[0034] In the above technical solution, during the decoding stage, the first Gaussian sphere can be decoded according to the encoding or decoding method indicated by the first indication information to obtain the first Gaussian sphere, and the second Gaussian sphere can be decoded according to the encoding or decoding method indicated by the second indication information to obtain the second Gaussian sphere. Thus, by carrying the first and second indication information in the bitstream, the amount of information included in the bitstream is increased, as is the amount of information that can be referenced during the decoding stage, thereby further and effectively improving the decoding performance.

[0035] In some possible implementations, multiple Gaussian spheres belong to N Gaussian sphere groups, where N is a positive integer greater than 1. The decoding methods for the N Gaussian sphere groups are different; that is, different Gaussian sphere groups correspond to different decoding methods, with each Gaussian sphere group corresponding to one decoding method. Thus, using different decoding methods for different groups of Gaussian spheres can improve the flexibility of decoding.

[0036] In some possible implementations, the first Gaussian sphere and the second Gaussian sphere belong to different groups within the N Gaussian sphere groups, such as the first Gaussian sphere belonging to the first Gaussian sphere group and the second Gaussian sphere belonging to the second Gaussian sphere group. This means that the decoding methods corresponding to the first Gaussian sphere and the second Gaussian sphere are different.

[0037] In some possible implementations, the bitstream includes the range of each of the N Gaussian sphere groups. The method further includes: determining the group to which a first Gaussian sphere belongs based on first indication information and the range of each of the N Gaussian sphere groups; and determining the group to which a second Gaussian sphere belongs based on second indication information and the range of each of the N Gaussian sphere groups.

[0038] The first indication information indicates the sequence number of the first Gaussian sphere in its group, and the second indication information indicates the sequence number of the second Gaussian sphere in its group. For example, different numerical sequence numbers can be used in the indication information for each Gaussian sphere to represent different groups. This allows each Gaussian sphere to be uniquely identified by its sequence number.

[0039] For example, the range of each Gaussian sphere group in the N Gaussian sphere groups can be represented by an index interval. It is worth noting that the index intervals of different groups do not overlap. In this way, the index interval of each group can be used to uniquely identify the corresponding group.

[0040] Among multiple Gaussian spheres, those belonging to the same group have consecutive ordinal numbers within that group. This consecutive numbering simplifies data maintenance and management, clearly representing each Gaussian sphere and facilitating subsequent identification of its group based on its ordinal number.

[0041] The method further includes: determining the group to which the first Gaussian sphere belongs based on its index in the group and the range of each of the N Gaussian sphere groups; and determining the group to which the second Gaussian sphere belongs based on its index in the group and the range of each of the N Gaussian sphere groups.

[0042] In the above embodiments, a method is provided to determine the group to which a Gaussian ball belongs based on the Gaussian ball's serial number and the range of each Gaussian ball group. This method can determine the group to which a Gaussian ball belongs by using the Gaussian ball's serial number and the range of each Gaussian ball group, and then use the decoding method corresponding to the group to perform decoding.

[0043] In some possible implementations, the method further includes: determining the group to which the first Gaussian sphere belongs based on first indication information; and determining the group to which the second Gaussian sphere belongs based on second indication information.

[0044] For example, different character flags can be used to represent different groups in the indication information of each Gaussian sphere. Taking the first Gaussian sphere group and the second Gaussian sphere group as examples, 0 can be used to refer to the first Gaussian sphere group and 1 can be used to refer to the second Gaussian sphere group.

[0045] In the above implementation, the group to which each Gaussian ball belongs can be directly determined by the indication information of each Gaussian ball, and then the encoding or decoding method corresponding to the Gaussian ball can be determined according to the group to which the Gaussian ball belongs.

[0046] In some possible implementations, the bitstream also includes grouping information for N Gaussian sphere groups. The method further includes decoding multiple Gaussian spheres based on the grouping information of the N Gaussian sphere groups to obtain multiple Gaussian spheres. The grouping information includes the number of groups, the number of Gaussian spheres in each group, and the encoding information for each group.

[0047] In some possible implementations, decoding the encoded data of the first Gaussian sphere according to the first instruction information to obtain the first Gaussian sphere, and decoding the encoded data of the second Gaussian sphere according to the second instruction information to obtain the second Gaussian sphere, includes:

[0048] Based on the first instruction information, the encoded data of the first Gaussian sphere is subjected to the first inverse quantization to obtain the first Gaussian sphere.

[0049] Based on the second instruction information, the encoded data of the second Gaussian sphere is subjected to a second inverse quantization to obtain the second Gaussian sphere.

[0050] In the above embodiments, the decoding methods of the first Gaussian sphere and the second Gaussian sphere can be quickly determined by using the first indication information and the second indication information carried in the bit stream. Then, the first Gaussian sphere is decoded by using the decoding method of the first Gaussian sphere, i.e., the first inverse quantization, and the second Gaussian sphere is decoded by using the decoding method of the second Gaussian sphere, i.e., the second inverse quantization, which can effectively improve the decoding efficiency.

[0051] In some possible implementations, the encoded data of the second Gaussian sphere includes encoded data of the first feature data and encoded data of the second feature data. Accordingly, the encoded data of the second Gaussian sphere undergoes a second inverse quantization, including:

[0052] The encoded data of the first feature data is subjected to a third inverse quantization, and the encoded data of the second feature data is subjected to a fourth inverse quantization.

[0053] In some possible implementations, the encoded data of the second feature data undergoes a fourth inverse quantization, including:

[0054] The encoded data of the second feature data is subjected to inverse vector quantization, and the result of inverse vector quantization is subjected to inverse scalar quantization.

[0055] In some possible implementations, the encoded data of the second feature data undergoes a fourth inverse quantization, including:

[0056] The encoded data of the second feature data is subjected to inverse product quantization, and the result of inverse product quantization is subjected to inverse scalar quantization.

[0057] Thirdly, a 3DGS model encoding device is provided for implementing any of the methods provided in the first aspect. This 3DGS model encoding device includes modules, units, or means corresponding to the aforementioned methods. The actions performed by these modules, units, or means can be implemented in hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the aforementioned functions.

[0058] In one possible implementation, the device may include an acquisition module and an encoding module; wherein:

[0059] The acquisition module is used to acquire a 3DGS model, which includes multiple Gaussian spheres, including a first Gaussian sphere and a second Gaussian sphere.

[0060] The encoding module is used to encode multiple Gaussian spheres to obtain encoded data of the 3DGS model, and to encode the encoded data into a bitstream. The bitstream also includes first indication information and second indication information. The first indication information is used to indicate the method of encoding or decoding the first Gaussian sphere, and the second indication information is used to indicate the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different.

[0061] Fourthly, a 3DGS model decoding device is provided to implement any of the methods provided in the second aspect above. The 3DGS model decoding device includes modules, units, or means that implement the methods described above. The actions performed by these modules, units, or means can be implemented in hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0062] In one possible implementation, the device may include a receiving module and a decoding module; wherein:

[0063] The receiving module is used to receive the bit stream, which includes the encoded data of a 3DGS model. The 3DGS model includes multiple Gaussian spheres, including a first Gaussian sphere and a second Gaussian sphere. The bit stream also includes first indication information and second indication information. The first indication information is used to indicate the method of encoding or decoding the first Gaussian sphere, and the second indication information is used to indicate the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different.

[0064] The decoding module is used to decode the encoded data of the first Gaussian sphere according to the first instruction information to obtain the first Gaussian sphere, and to decode the encoded data of the second Gaussian sphere according to the second instruction information to obtain the second Gaussian sphere.

[0065] Fifthly, an encoding device is provided, comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store computer-executable instructions; and the processor being used to invoke the computer-executable instructions, thereby implementing the method of the first aspect above or any implementation thereof.

[0066] The encoding device in the fifth aspect can be any of the encoding devices in the first aspect, or an apparatus containing the encoding device, or an apparatus included in the encoding device, such as a chip. For example, the encoding device can be a server.

[0067] In a sixth aspect, a decoding device is provided, comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store computer-executed instructions; and the processor being used to invoke the computer-executed instructions to implement the method of the second aspect above or any implementation thereof.

[0068] The decoding device in the sixth aspect can be any of the decoding devices in the first aspect, or an apparatus containing the decoding device, or an apparatus included in the decoding device, such as a chip. For example, the decoding device can be a terminal.

[0069] In a seventh aspect, an encoding device is provided, including a processing circuit for implementing the method of the first aspect or any implementation thereof.

[0070] Eighthly, a decoding device is provided, including a processing circuit for implementing the method of the second aspect above or any implementation thereof.

[0071] Ninthly, a chip is provided, the chip comprising: a processor and an interface circuit; the interface circuit for receiving computer execution instructions; and the processor for executing the computer execution instructions to perform the methods of the first aspect, the second aspect, or any implementation thereof described above.

[0072] In a tenth aspect, a computer-readable storage medium is provided, comprising computer-executable instructions that, when executed on an encoding device, cause the encoding device to perform the method described in the first aspect or any implementation thereof, and when executed on a decoding device, cause the decoding device to perform the method described in the second aspect or any implementation thereof.

[0073] In an eleventh aspect, a computer-readable storage medium is provided, including a bitstream, the bitstream including encoded data of a 3DGS model, the 3DGS model including a plurality of Gaussian spheres, the plurality of Gaussian spheres including a first Gaussian sphere and a second Gaussian sphere, the bitstream also including first indication information and second indication information, the first indication information being used to indicate the method of encoding or decoding the first Gaussian sphere, the second indication information being used to indicate the method of encoding or decoding the second Gaussian sphere, the first indication information and the second indication information being different.

[0074] In a twelfth aspect, a bitstream is provided, the bitstream including encoded data of a 3DGS model, the 3DGS model including multiple Gaussian spheres, the multiple Gaussian spheres including a first Gaussian sphere and a second Gaussian sphere, the bitstream also including first indication information and second indication information, the first indication information being used to indicate the method of encoding or decoding the first Gaussian sphere, the second indication information being used to indicate the method of encoding or decoding the second Gaussian sphere, the first indication information and the second indication information being different.

[0075] In a thirteenth aspect, a computer program product is provided, comprising computer-executable instructions that, when executed on an encoding device, cause the encoding device to perform the method described in the first aspect or any implementation thereof, and when executed on a decoding device, cause the decoding device to perform the method described in the second aspect or any implementation thereof.

[0076] In a fourteenth aspect, an apparatus for storing a bitstream is provided, the apparatus comprising: a receiver and at least one storage medium, the receiver being configured to receive a bitstream generated according to the encoding method described in the first aspect above, and the at least one storage medium being configured to store the bitstream.

[0077] In a fifteenth aspect, an apparatus for transmitting a bitstream is provided, the apparatus comprising: a transmitter and a receiver, the receiver being configured to receive a bitstream generated according to the encoding method described in the first aspect above, and the transmitter being configured to transmit the bitstream to an end-side device via a transmission medium.

[0078] In a sixteenth aspect, an apparatus for transmitting a bitstream is provided, the apparatus comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store a bitstream generated according to the encoding method described in the first aspect above, the transmitter being used to obtain the bitstream from the storage medium and transmit the bitstream to an end-side device via a transmission medium.

[0079] In a seventeenth aspect, a system for distributing bitstreams is provided, the system comprising: at least one storage medium for storing bitstreams generated according to the encoding method described in the first aspect above; and a streaming media device for obtaining a target bitstream from the at least one storage medium and sending the target bitstream to an end-side device, wherein the streaming media device includes a content server or a content distribution server.

[0080] The technical effects of any of the implementation methods in aspects three through seventeen can be found in the technical effects of the corresponding implementation methods in aspects one or two, and will not be repeated here.

[0081] All possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0082] Figure 1 is a schematic diagram of the system architecture of a 3DGS model encoding and decoding method provided in an embodiment of this application;

[0083] Figure 2 is a schematic diagram of the system architecture of another 3DGS model encoding and decoding method provided in the embodiments of this application;

[0084] Figure 3 is a schematic diagram of the system architecture of another 3DGS model encoding and decoding method provided in the embodiments of this application;

[0085] Figure 4 is a schematic diagram of the business process for encoding and decoding a 3DGS model provided in an embodiment of this application;

[0086] Figure 5 is a schematic diagram of the hardware structure of a terminal provided in an embodiment of this application;

[0087] Figure 6 is a schematic diagram of the hardware structure of a server provided in an embodiment of this application;

[0088] Figure 7 is a flowchart illustrating a 3DGS model encoding and decoding method provided in an embodiment of this application;

[0089] Figure 8 is a flowchart illustrating another 3DGS model encoding and decoding method provided in an embodiment of this application;

[0090] Figure 9 is a schematic flowchart of an example encoding / decoding method based on flag bits provided in an embodiment of this application;

[0091] Figure 10 is a schematic flowchart of an example of a sequence number-based encoding / decoding method provided in an embodiment of this application;

[0092] Figure 11 is a schematic diagram of the structure of an encoding device for a 3DGS model provided in an embodiment of this application;

[0093] Figure 12 is a schematic diagram of the structure of a decoding device for a 3DGS model provided in an embodiment of this application. Detailed Implementation

[0094] In the description of this application, unless otherwise stated, "multiple" means two or more. At least one of the following or similar expressions refer to any combination of these terms, including any combination of single or plural terms. For example, at least one of a, b, and / or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0095] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0096] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0097] It is understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, throughout the specification, various embodiments do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0098] It is understood that some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current solution on which they are based, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in certain scenarios. Correspondingly, the apparatus given in the embodiments of this application can also implement these features or functions, which will not be elaborated here.

[0099] In this application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be mutually referenced. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. The following embodiments of this application do not constitute a limitation on the scope of protection of this application.

[0100] The following provides an exemplary description of the application scenarios of the embodiments of this application.

[0101] With the rapid development of technology and the wave of digital transformation, digital twin technology has shown great potential in many fields, such as 3D visualization based on digital twins.

[0102] Digital twins refer to the creation of virtual models in the digital world that correspond to real-world entities (such as physical objects). For example, digital twin technology can be applied to product display scenarios on e-commerce platforms, virtual house viewing scenarios, virtual game scenarios, six degrees of freedom (6DoF) photo scenarios, and 6DoF video scenarios. 6DoF includes three rotational degrees of freedom and three translational degrees of freedom. Rotational degrees of freedom typically refer to rotation around the X, Y, and Z axes, while translational degrees of freedom typically refer to movement along the X, Y, and Z axes. Through 6DoF photos or 6DoF videos, users can be provided with an interaction method based on six degrees of freedom, enabling them to interact more naturally and realistically in a virtual environment.

[0103] Digital twins can typically take various forms of representation, such as 3DGS models, meshes, point clouds, and neural radiance fields. Among these, 3DGS models are currently considered the optimal and most widely used form of digital twin representation. 3DGS is a technique based on Gaussian distribution used to achieve efficient, high-quality, and real-time 3D scene reconstruction and rendering. In 3DGS, a set of Gaussian spheres with color characteristics is typically used to simulate the scene, and then quickly projected onto the image through rasterization, thus achieving high-quality real-time rendering. In essence, a Gaussian sphere refers to a 3D shape or structure based on Gaussian distribution, used to simulate objects in the scene. The properties of Gaussian distribution enable precise capture and rendering of details and textures, smoothly representing the shape and texture of objects, thereby providing high-quality rendering effects.

[0104] Digital twin-based 3D visualization refers to processing spatial information from the real world to obtain a 3DGS model, and then transmitting and rendering this 3DGS model to achieve its visual display. Typically, differentiable rendering technology is used to process photographs of a scene (such as an environment or an object) from multiple perspectives to obtain digital twin assets expressed as 3DGS models. Using 3DGS models to represent digital twin assets offers faster rendering speeds (e.g., enabling real-time rendering on the device) and higher rendering quality.

[0105] Currently, to achieve better 3D visualization effects, 3DGS models typically require a large amount of data. A 3DGS model can be viewed as a set of Gaussian spheres with color characteristics. The feature data contained in each Gaussian sphere are shown in Table 1, namely, center point, scaling information, spatial orientation information (rotation), opacity, diffuse color (DC), and higher-order spherical covariance coefficients.

[0106] Table 1

[0107] The center point is used to indicate the location of the center point of the Gaussian sphere, represented as (X, Y, Z). See Table 1. The feature size of the center point is (4×3), indicating that the number of coefficients at the center point is 3, and each coefficient occupies 4 bits.

[0108] Scale information is used to indicate the scale of the Gaussian sphere on each axis, defining the size of the Gaussian sphere in three-dimensional space, as represented by (scale). x scale y scale z (See Table 1). The feature size of the scale information is (4×3), which means that there are 3 coefficients for the scale information and each coefficient occupies 4 bits.

[0109] Spatial orientation information, used to indicate the orientation of a Gaussian sphere, can be represented using quaternions. Understandably, rotating the Gaussian sphere can result in different orientations. Quaternions are an efficient way to represent rotations, indicating the rotation of the Gaussian sphere relative to the coordinate system. A quaternion consists of a real part and three imaginary units, and can be represented as (a+bi+cj+dk), where a, b, c, and d are real numbers, and i, j, and k are imaginary numbers. Referring to Table 1, the feature size of the spatial orientation information is (4×4), the number of coefficients representing the spatial orientation information is 4, and each coefficient occupies 4 bits.

[0110] Opacity describes the degree to which a Gaussian sphere allows light to pass through in three-dimensional space. Opacity is typically represented by a floating-point number between 0 and 1, where 0 represents complete transparency and 1 represents complete opacity. Referring to Table 1, the feature size of opacity is (4×1), the number of coefficients representing opacity is 1, and each coefficient occupies 4 bits.

[0111] Diffuse color (also known as the 0th-order spherical covariance) is used to indicate the color of light when it is diffusely reflected off the surface of a Gaussian sphere, simulating the effect of light reflection in the real world. Referring to Table 1, the feature size of the diffuse color is (4×3), indicating that there are 3 coefficients for the diffuse color, and each coefficient occupies 4 bits.

[0112] Higher-order spherical coefficients are used to indicate direction-related color information. Understandably, the color of a Gaussian sphere differs depending on its orientation. Each color channel (R, G, B) of the Gaussian sphere has independent higher-order spherical coefficients, such as first-order, second-order, and third-order coefficients. Typically, there are 3 first-order coefficients, 5 second-order coefficients, and 7 third-order coefficients, totaling 45 (i.e., 15 × 3) higher-order spherical coefficients. Based on the total number of features (236) in a Gaussian sphere as shown in Table 1, it can be calculated that higher-order spherical coefficients account for the largest proportion of data, reaching approximately 76%. Referring to Table 1, the feature size of the higher-order spherical coefficients is (4 × 15 × 3), indicating that the number of coefficients in each higher-order spherical coefficient is 15 × 3, and each coefficient occupies 4 bits.

[0113] Because 3DGS models typically contain a large amount of data—for example, a scene with a background and characters might be several hundred megabytes (MB) in size—downloading 3DGS models on the client-side can result in excessively long download times due to the sheer volume of data. Furthermore, the frequent reading of 3DGS model data from memory during client-side rendering also limits rendering speed. Therefore, considering the difficulties in transmitting and storing large amounts of data, it is necessary to encode and decode the data contained in the 3DGS model.

[0114] In the encoding and decoding scheme of 3DGS models, data quantization processing is usually involved. The following is an introduction to three quantization methods provided by relevant technologies.

[0115] (1) Scalar quantization (SQ).

[0116] Scalar quantization is a commonly used encoding method in numerical compression, which refers to the individual quantization processing of data (usually scalars, i.e., quantities that only have magnitude and no direction). For example, in some embodiments, scalar quantization is used to quantize a floating-point number (such as float32) into a smaller data type (such as 8 bits), thereby reducing data transmission and storage requirements.

[0117] Taking the feature data contained in the Gaussian sphere shown in Table 1 as an example, the feature data contained in the Gaussian sphere are all represented using floating-point numbers such as float32. By performing scalar quantization on the feature data contained in the Gaussian sphere, the corresponding feature data can be stored using a reasonable bit depth. Here, bit depth refers to the number of bits used in scalar quantization, also known as bit width.

[0118] For example, taking the conversion of the value of feature A into n bits based on scalar quantization as an example, the scalar quantization process can be as follows: based on the original value of feature A, the maximum value of feature A, the minimum value of feature A, the bit depth and the following scalar quantization formula (1), the value of feature A is converted to obtain the scalar quantization result of feature A, that is, n bits.

[0119] In the formula, qA represents the scalar quantization result of feature A; A represents the original value of feature A; Amax represents the maximum value of feature A; Amin represents the minimum value of feature A; n represents the bit depth; and round represents rounding the value to an integer.

[0120] When it is necessary to decode the scalar quantization result of feature A to obtain the original value of feature A, the scalar quantization result of feature A can be converted based on the scalar quantization result of feature A, the maximum value of feature A, the minimum value of feature A, the bit depth, and the following inverse scalar quantization formula (2) to obtain the original value of feature A.

[0121] During the encoding stage of the 3DGS model, scalar quantization can be used to quantize the feature data of the 3DGS model to obtain a reasonable bit depth. This allows for compression of the 3DGS model's size without affecting subsequent rendering effects, thereby reducing the requirements for model data transmission and storage. During the decoding stage of the 3DGS model, the scalar quantization result can be inversely quantized to obtain the original data.

[0122] (2) Vector quantization (VQ).

[0123] Vector quantization is also a commonly used encoding method in numerical compression. It uses the index of the codeword in the code table that is closest to the input vector (a quantity that has both magnitude and direction) to replace the input vector for transmission and storage.

[0124] In some embodiments, vector quantization can be implemented by clustering a dataset containing multiple vectors to obtain multiple clusters, and generating a code table that includes these clusters. Then, the cluster in the code table that is closest to the vector is used to represent the vector, and the index of the cluster is used as the vector quantization result. Here, the clusters in the code table can be called codewords.

[0125] For example, for a dataset containing N vectors, such as S = {A i |i∈[0,N]}, the dataset is divided into M clusters through clustering, resulting in C={V j |j∈[0,M]}. Here, set C can be called a code table. For the original data A...i It can be used in the code table with A i The closest code word V j Remove the approximate representation and use index j as A. i The vector quantization result is used to represent A i When it is necessary to base it on A i The vector quantization result is decoded to obtain the original data A. i In this case, the codeword V can be obtained by inverse vector quantization, that is, by looking up the codeword V from the code table based on index j. j As the decoded A i .

[0126] In the encoding stage of the 3DGS model, vector quantization can be used to quantize the feature data of the 3DGS model to obtain an index with a reasonable bit depth. This further reduces the size of the 3DGS model, thereby reducing the transmission and storage requirements of the model data. In the decoding stage of the 3DGS model, inverse vector quantization (i.e., a lookup table operation) can be performed on the vector quantization result of the 3DGS model to obtain codewords to replace the original data.

[0127] (3) Product quantization (PQ).

[0128] Product quantization is a commonly used coding method in numerical compression. It involves dividing a high-dimensional space into multiple low-dimensional subspaces, where each subspace is independently quantized using vector quantization to generate its own code table. In essence, product quantization is a coding method derived from vector quantization.

[0129] For example, a high-dimensional vector can be divided into several low-dimensional sub-vectors, and vector quantization can be applied independently to each low-dimensional sub-vector. For instance, a high-dimensional vector of length 15 can be divided into two groups: the first 7 dimensions and the last 8 dimensions. Vector quantization can then be applied to each group to obtain two code tables. When decoding the original data based on the product quantization result is required, inverse product quantization can be used. That is, the low-dimensional sub-vectors of each group can be inversely vector-quantized, and then the low-dimensional sub-vectors can be concatenated to form the high-dimensional vector.

[0130] In the encoding stage of the 3DGS model, product quantization can be used to quantize the feature data of the 3DGS model to obtain indexes with reasonable bit depths for several low-dimensional sub-vectors. This further reduces the size of the 3DGS model, thereby reducing the transmission and storage requirements of the model data. In the decoding stage of the 3DGS model, inverse product quantization can be performed on the product quantization results. This involves performing a lookup operation on several low-dimensional sub-vectors in each group to obtain codewords to replace each low-dimensional sub-vector, and then concatenating these low-dimensional sub-vectors to obtain codewords to replace the high-dimensional vectors.

[0131] However, in related technologies, all Gaussian spheres use a uniform encoding method, which is inflexible and produces poor encoding results. For example, if it is necessary to improve the compression ratio of the 3DGS model, a low bitrate quantization method must be used. This may cause some Gaussian spheres that are important for rendering to be over-compressed, resulting in a loss of rendering quality. If it is necessary to ensure the rendering quality of the 3DGS model, a high bitrate quantization method must be used. This may result in a low compression ratio of the 3DGS model, leading to a longer transmission time for the 3DGS model and a longer time to load data from memory during rendering, thus affecting the rendering speed.

[0132] In view of this, embodiments of this application provide an encoding and decoding method for a 3DGS model. In the encoding stage, multiple Gaussian spheres of the 3DGS model are encoded to obtain encoded data of the 3DGS model, and this encoded data is encoded into a bitstream. Furthermore, the bitstream carries first and second indication information to indicate the encoding or decoding method of the first and second Gaussian spheres. Since the first and second indication information are different, it indicates that the encoding and decoding methods of the first and second Gaussian spheres are different. This allows for different encoding and decoding methods to be used for different Gaussian spheres, effectively improving the encoding and decoding performance. Furthermore, in the decoding stage, the encoded data of the first Gaussian sphere can be decoded according to the encoding or decoding method indicated by the first indication information to obtain the first Gaussian sphere, and the encoded data of the second Gaussian sphere can be decoded according to the encoding or decoding method indicated by the second indication information to obtain the second Gaussian sphere. Thus, by carrying the first and second indication information in the bitstream, the amount of information included in the bitstream is increased, as is the amount of information that can be referenced during the decoding stage, which can further and effectively improve the decoding effect.

[0133] To facilitate understanding of the embodiments of this application, the following points will be explained before introducing the embodiments of this application.

[0134] 1. In the embodiments of this application, "instruction" can include direct instruction and indirect instruction, as well as explicit instruction and implicit instruction. The information indicated by a certain piece of information is called the information to be instructed. In specific implementation, there are many ways to indicate the information to be instructed, such as, but not limited to, directly indicating the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly indicate the information to be instructed by indicating other information, where there is a relationship between the other information and the information to be instructed. It can also indicate only a part of the information to be instructed, while the other parts of the information to be indicated are known or pre-agreed.

[0135] 2. In the embodiments of this application, the descriptions such as "in the case of", "if" and "if" all refer to the fact that the device (e.g., electronic device) will make corresponding processing under certain objective circumstances. They are not time limits, nor do they require the device (e.g. electronic device) to have a judgment action when it is implemented, nor do they mean that there are other limitations.

[0136] Furthermore, the system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0137] Furthermore, the actions, terms, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are merely examples, and other names may be used in specific implementations without limitation.

[0138] The system architecture of the embodiments of this application will be described below as an example.

[0139] The 3DGS model encoding and decoding method provided in this application can be applied to 3D visualization scenarios based on digital twins, such as product display scenarios, virtual house viewing scenarios, virtual game scenarios, 6DoF photo scenarios, and 6DoF video scenarios. Specifically, the 3DGS model encoding and decoding method provided in this application can be applied to the encoding and decoding process of 3DGS models in 3D visualization scenarios based on digital twins.

[0140] In some embodiments, the 3DGS model encoding and decoding method provided in this application can be applied to the system architecture shown in FIG1. ​​For example, FIG1 is a schematic diagram of a system architecture for a 3DGS model encoding and decoding method provided in this application. Referring to FIG1, the system architecture may include: an encoding device 101 and a decoding device 102.

[0141] The encoding device 101 is used to encode the 3DGS model. Optionally, the encoding device 101 may have a built-in encoder, such as a 3DGS encoder. The decoding device 102 is used to decode the 3DGS model. Optionally, the decoding device 102 may have a built-in decoder, such as a 3DGS decoder.

[0142] The 3DGS model encoding and decoding method provided in this application embodiment can be completed based on the cooperation of encoding device 101 and decoding device 102. The corresponding process may include: encoding device 101 acquiring a 3DGS model, the 3DGS model including multiple Gaussian spheres, including a first Gaussian sphere and a second Gaussian sphere. Encoding device 101 encodes the multiple Gaussian spheres to obtain encoded data of the 3DGS model, and encodes the encoded data into a bitstream. The bitstream further includes first indication information and second indication information. The first indication information indicates the method of encoding or decoding the first Gaussian sphere, and the second indication information indicates the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different. Decoding device 102 receives the bitstream. Decoding device 102 decodes the encoded data of the first Gaussian sphere according to the first indication information to obtain the first Gaussian sphere, and decodes the encoded data of the second Gaussian sphere according to the second indication information to obtain the second Gaussian sphere.

[0143] In some embodiments of this application, the encoding device 101 and the decoding device 102 are two separately deployed devices. Referring to FIG1, the encoding device 101 can be a server, and the decoding device 102 can be a terminal. In some embodiments, the server and the terminal can be connected via a wired network or a wireless network.

[0144] For example, in a 6DoF photo scenario, taking encoding device 101 as the server and decoding device 102 as the terminal, a user can operate on the terminal to acquire multiple images of a target scene (such as the same environment or the same object) captured from multiple perspectives, and upload these multiple images of the target scene to the server. The server receives the multiple images of the target scene from the terminal, and can reconstruct a 3DGS model of the target scene based on these multiple images. Then, the server uses the 3DGS model encoding method provided in this application embodiment to encode the 3DGS model to obtain an encoded bitstream, and transmits the bitstream to the terminal via the network. The terminal receives the bitstream, uses the decoding method provided in this application embodiment to decode the encoded data of the 3DGS model, and then performs rendering based on the decoded data.

[0145] For example, Figure 2 is a schematic diagram of the system architecture of another 3DGS model encoding and decoding method provided in an embodiment of this application. Referring to Figure 2, the encoding device 101 can be a terminal, such as an encoding terminal, and the decoding device 102 can also be a terminal, such as a decoding terminal. In some embodiments, the encoding terminal and the decoding terminal can communicate with each other through a physical data cable connection, Bluetooth connection, network connection, or other means.

[0146] For example, in a 6DoF photo scene, taking encoding device 101 as the encoding terminal and decoding device 102 as the decoding terminal, a user can operate on the encoding terminal to acquire multiple images of the target scene captured from multiple perspectives, and reconstruct a 3DGS model of the target scene based on these multiple images. Alternatively, a user can operate on the decoding terminal to acquire multiple images of the target scene captured from multiple perspectives, and send these multiple images of the target scene to the encoding terminal, which then reconstructs a 3DGS model of the target scene based on these multiple images. Furthermore, the encoding terminal uses the 3DGS model encoding method provided in this application embodiment to encode the 3DGS model to obtain an encoded bitstream, and transmits the bitstream to the decoding terminal. The decoding terminal receives the bitstream, uses the decoding method provided in this application embodiment to decode the encoded data of the 3DGS model, and then renders the data based on the decoded data.

[0147] In other embodiments of this application, the encoding device 101 and the decoding device 102 may also be deployed on the same device. For example, FIG3 is a schematic diagram of the system architecture of another 3DGS model encoding and decoding method provided in an embodiment of this application. Referring to FIG3, the encoding device 101 and the decoding device 102 may be an encoder and a decoder deployed on a terminal, or the encoding device 101 and the decoding device 102 may be an encoder and a decoder deployed on a server. In some embodiments, the encoder and decoder may be connected via a bus.

[0148] For example, in a 6DoF photo scene, taking the encoding device 101 as an encoder deployed on the terminal and the decoding device 102 as a decoder deployed on the terminal, the user can operate on the terminal to acquire multiple images of the target scene captured from multiple perspectives, and send these multiple images of the target scene to the encoder. The encoder receives the multiple images of the target scene and can reconstruct a 3DGS model of the target scene based on these images. Then, the encoder uses the 3DGS model encoding method provided in this application embodiment to encode the 3DGS model to obtain an encoded bitstream, and transmits the bitstream to the decoder via a bus. The decoder receives the bitstream, uses the decoding method provided in this application embodiment to decode the encoded data of the 3DGS model, and then performs rendering based on the decoded data.

[0149] Taking encoding device 101 and decoding device 102 as two separately deployed devices as an example, in some embodiments, the system architecture may further include: a transmission device for transmitting the bitstream, i.e., transmitting the data included in the bitstream. For example, encoding device 101 can transmit the bitstream through the transmission device. In some embodiments, the system architecture may further include: a distribution device for distributing the bitstream, i.e., distributing the data included in the bitstream. For example, during the transmission of the bitstream, the distribution device can distribute the bitstream to decoding device 102. In some embodiments, the system architecture may further include: a storage device for storing the bitstream, i.e., storing the data included in the bitstream. For example, after receiving the bitstream, decoding device 102 can store the data included in the bitstream in the storage device. Then, when decoding is needed, the data included in the bitstream can be retrieved from the storage device.

[0150] The following describes the encoding and decoding business process provided in the embodiments of this application based on a 3DGS encoder and a 3DGS decoder.

[0151] For example, Figure 4 is a schematic diagram of the business process for encoding and decoding a 3DGS model according to an embodiment of this application. Referring to Figure 4, the 3DGS encoder may include a Gaussian sphere grouping module 401 and a group-based encoding module 402. The 3DGS decoder may include a Gaussian sphere grouping recognition module 403 and a group-based decoding module 404.

[0152] Accordingly, the encoding and decoding business process may include:

[0153] ① Gaussian sphere grouping. The corresponding process is as follows: input multiple Gaussian spheres (i.e., the original Gaussian spheres) in the 3DGS model into the Gaussian sphere grouping module 401, and group the multiple Gaussian spheres through the Gaussian sphere grouping module 401 to obtain N Gaussian sphere groups.

[0154] ② Encoding. The corresponding process is as follows: input the N Gaussian spheres into the group encoding module 402, and encode the Gaussian spheres according to their respective groups to obtain the encoded data of the 3DGS model.

[0155] ③ Encoding into the bitstream. The corresponding process is as follows: the encoded data, indication information, and grouping information of the 3DGS model are encapsulated together and encoded into the bitstream. The indication information indicates the encoding or decoding method of the corresponding Gaussian sphere. The grouping information includes information used to assist decoding and rendering.

[0156] ④ Transmission and storage. For example, the bitstream is transmitted through transmission equipment, which means transmitting the encoded data, indication information, and packet information of the 3DGS model. Similarly, the bitstream is stored through storage equipment, which means storing the encoded data, indication information, and packet information of the 3DGS model.

[0157] ⑤ Read the bitstream. The corresponding process is: to parse the bitstream using a 3DGS decoder to obtain encoded data, indication information, and grouping information, etc.

[0158] ⑥ Identify groups. The corresponding process is as follows: the Gaussian sphere group identification module 403 identifies the group to which each Gaussian sphere belongs based on the instruction information.

[0159] ⑦ Decoding. The corresponding process is as follows: input the group to which each Gaussian sphere belongs and the encoded data of each Gaussian sphere into the group decoding module 404. The group decoding module 404 decodes the encoded data of the Gaussian sphere according to the decoding method corresponding to the group to which each Gaussian sphere belongs, and obtains the Gaussian sphere (i.e., the reconstructed Gaussian sphere).

[0160] It is worth noting that the decoding process in this embodiment can be executed concurrently with the rendering calculation of the 3DGS model by the graphics processing unit (GPU). That is, when the GPU performs rendering calculations on the 3DGS model, for the encoded data of the 3DGS model, when a specific feature data is needed, the GPU directly reads the encoded data of that feature data, uses a decoder to decode the encoded data of that feature data to obtain the original feature data, and then the GPU performs subsequent rendering calculations based on the decoded data. Compared to decoding all the encoded data of all Gaussian spheres of the 3DGS model before rendering calculations, although the relevant features need to be decoded each time, the amount of data that needs to be loaded each time is reduced, which can reduce the time for the GPU's computing unit to load data from memory, thereby speeding up the rendering process.

[0161] For the terminals involved in Figures 1 to 3 above, exemplarily, the terminal can be at least one of the following devices: smartphone, smart wearable device (e.g., smartwatch, smart glasses, etc.), printer, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, laptop computer, vehicle terminal, etc. This application embodiment does not limit this. In one example of this application, a schematic diagram of the terminal's hardware structure is shown in Figure 5. Figure 5 is a schematic diagram of the hardware structure of a terminal provided in an embodiment of this application.

[0162] Referring to Figure 5, terminal 500 may include a processor 510, an external memory interface 520, an internal memory 521, an encoder 522, a decoder 523, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, antenna 1, antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a sensor module 580, a camera 590, and a display screen 591. The sensor module 580 may include a pressure sensor 580A, a gyroscope sensor 580B, an accelerometer sensor 580C, a proximity sensor 580D, a touch sensor 580E, etc.

[0163] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the terminal 500. In other embodiments of this application, the terminal 500 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0164] Processor 510 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0165] The terminal 500 implements display functions through a GPU, a display screen 591, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 591 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 510 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0166] The controller can serve as the nerve center and command center of the terminal 500. Based on the instruction opcode and timing signals, the controller can generate operation control signals to control the fetching and execution of instructions.

[0167] The processor 510 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. This memory can store instructions or data that the processor 510 has just used or that are used repeatedly. If the processor 510 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 510, and thus improves the efficiency of the system.

[0168] The external storage interface 520 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal 500. The external storage card communicates with the processor 510 through the external storage interface 520 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0169] The internal memory 521 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 510 and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data, and can be pre-loaded into the RAM for direct reading and writing by the processor 510. In this embodiment, the internal memory 521 is used to store computer execution instructions. The processor 510 executes various functional applications and data processing of the terminal 500 by running the instructions stored in the internal memory 521.

[0170] For example, in a scenario where a 3DGS model is encoded based on terminal 500, when processor 510 calls and executes the instructions or program code stored in internal memory 521, it can implement the 3DGS model encoding method provided in this application embodiment. Similarly, in a scenario where a 3DGS model is decoded based on terminal 500, when processor 510 calls and executes the instructions or program code stored in internal memory 521, it can implement the 3DGS model decoding method provided in this application embodiment.

[0171] Encoder 522 is used to convert raw data (such as audio, video, text, etc.) into an encoding format suitable for transmission or storage. In some embodiments of this application, in scenarios where 3DGS models are encoded based on terminal 500, terminal 500 can call encoder 522 to execute the 3DGS model encoding process.

[0172] Decoder 523 is used to restore the encoded data to its original data format for display on display screen 591. In some embodiments of this application, in scenarios where 3DGS model decoding is performed based on terminal 500, terminal 500 can invoke decoder 523 to execute the 3DGS model decoding process.

[0173] USB port 530 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 530 can be used to connect a charger to charge terminal 500, and can also be used for data transfer between terminal 500 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0174] The charging management module 540 receives charging input from a charger, which can be either a wireless or wired charger. The power management module 541 connects to the battery 542, the charging management module 540, and the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540, providing power to the processor 510, internal memory 521, external memory, display 591, camera 590, and wireless communication module 560, etc.

[0175] The wireless communication function of terminal 500 can be implemented through antenna 1, antenna 2, mobile communication module 550, wireless communication module 560, modem processor and baseband processor.

[0176] The mobile communication module 550 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the terminal 500. The wireless communication module 560 can provide wireless communication solutions, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies, for use on the terminal 500.

[0177] In some embodiments, the antenna 1 of the terminal 500 is coupled to the mobile communication module 550, and the antenna 2 is coupled to the wireless communication module 560, so that the terminal 500 can communicate with the network and other devices through wireless communication technology.

[0178] Terminal 500 implements display functions through a GPU, display screen 591, and other components. The GPU is a microprocessor for image processing, connected to the display screen 591 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 510 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0179] The display screen 591 is used to display images, videos, etc. The display screen 591 includes a display panel. In some embodiments, the terminal 500 may include one or more display screens 591.

[0180] Terminal 500 can perform shooting functions through an ISP, camera 590, video codec, GPU, display 591, and application processor. Camera 590 is used to capture still images or videos.

[0181] Terminal 500 can implement audio functions, such as music playback and recording, through audio module 570 and application processor.

[0182] The pressure sensor 580A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 580A may be disposed on the display screen 591.

[0183] The gyroscope sensor 580B can be used to determine the motion attitude of the terminal 500. In some embodiments, the angular velocity of the terminal 500 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 580B.

[0184] The accelerometer 580C can detect the magnitude of acceleration of the terminal 500 in various directions (generally three axes). When the terminal 500 is stationary, it can detect the magnitude and direction of gravity.

[0185] A distance sensor 580D is used to measure distance. The terminal 500 can measure distance via infrared or laser. In some embodiments, during a shooting scene, the terminal 500 can utilize the distance sensor 580D to measure distance for rapid focusing.

[0186] Touch sensor 580E, also known as a "touch panel," can be located on display screen 591. The touch sensor 580E and display screen 591 together form a touchscreen, also known as a "touch screen." Touch sensor 580E detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 591. In other embodiments, touch sensor 580E may also be located on the surface of terminal 500, in a different position than display screen 591.

[0187] It should be noted that the structure shown in Figure 5 does not constitute a limitation on the terminal. In addition to the components shown in Figure 5, the terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0188] Regarding the servers involved in Figures 1 and 3 above, exemplarily, the server can be an independent physical server, such as a general-purpose server, a graphics processing unit (GPU) server, a data processing unit (DPU) server, an artificial intelligence (AI) server, etc., or a server cluster or distributed file system composed of multiple physical servers. This application embodiment does not limit this. In one example of this application, a schematic diagram of the server's hardware structure is shown in Figure 6. Figure 6 is a schematic diagram of the hardware structure of a server provided in an embodiment of this application. Referring to Figure 6, the server 600 may include: a processor 601, a memory 602, an encoder 603, a decoder 604, a communication interface 605, and a bus 606. The processor 601, the memory 602, and the communication interface 605 can be connected via the bus 606.

[0189] The processor 601 is the control center of the server 600. It can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0190] As an example, processor 601 may include one or more CPUs, such as CPU0 and CPU1 shown in Figure 6.

[0191] The memory 602 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0192] In one possible implementation, memory 602 can exist independently of processor 601. Memory 602 can be connected to processor 601 via bus 606 and is used to store data, instructions, or program code. In another possible implementation, memory 602 can be integrated with processor 601.

[0193] When processor 601 calls and executes instructions or program code stored in memory 602, it can implement the 3DGS model encoding and decoding method provided in the embodiments of this application. For example, in a scenario where 3DGS models are encoded based on server 600, processor 601 can implement the 3DGS model encoding method provided in the embodiments of this application by calling and executing instructions or program code stored in memory 602. Similarly, in a scenario where 3DGS models are decoded based on server 600, processor 601 can implement the 3DGS model decoding method provided in the embodiments of this application by calling and executing instructions or program code stored in memory 602.

[0194] Encoder 603 is used to convert raw data (such as audio, video, text, etc.) into an encoding format suitable for transmission or storage. In some embodiments of this application, in scenarios where 3DGS models are encoded based on server 600, server 600 can call encoder 603 to execute the encoding process of the 3DGS model.

[0195] Decoder 604 is used to restore the encoded data to its original data format for display on the display screen associated with server 600. In some embodiments of this application, in scenarios where 3DGS model decoding is performed based on server 600, server 600 can invoke decoder 604 to perform the 3DGS model decoding process.

[0196] The communication interface 605 is used for the server 600 to connect with other devices via a communication network, which can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 605 may include a receiving unit for receiving data and a transmitting unit for sending data.

[0197] Bus 606 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 6, but this does not indicate that there is only one bus or one type of bus.

[0198] It should be noted that the structure shown in Figure 6 does not constitute a limitation on the server 600. In addition to the components shown in Figure 6, the server 600 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0199] For ease of understanding, the following description, in conjunction with the above system architecture and accompanying drawings, provides an exemplary illustration of the 3DGS model encoding and decoding method provided in the embodiments of this application.

[0200] It is understood that in the embodiments of this application, the encoding device or the decoding device may perform some or all of the steps in the embodiments of this application. These steps or operations are merely examples. The embodiments of this application may also perform other operations or variations of various operations.

[0201] Figure 7 is a flowchart illustrating a 3DGS model encoding / decoding method according to an embodiment of this application. In some possible implementations, the 3DGS model encoding / decoding method can be completed by the encoding device and decoding device in the above system architecture. Referring to Figure 7, taking the interaction flow between the encoding device and the decoding device as an example, the method includes the following steps S701-S704.

[0202] S701, Encoding device acquires 3DGS model.

[0203] The 3DGS model includes multiple Gaussian spheres.

[0204] In some embodiments, the 3DGS model can be a virtual model obtained by 3D reconstruction based on multiple images of the target scene. Accordingly, the process by which the encoding device acquires the 3DGS model can be: acquiring multiple images of the target scene, performing 3D reconstruction based on the multiple images of the target scene, and obtaining the 3DGS model.

[0205] In this context, "target scene" refers to a scene in the real world, such as a specific environment or object. Multiple images of the target scene can be obtained by taking pictures of the target scene from different perspectives.

[0206] For example, in some embodiments, the multiple images of the target scene may be images pre-stored in the encoding device. The process by which the encoding device acquires the multiple images of the target scene may be: acquiring the pre-stored multiple images of the target scene from the encoding device (such as the encoding device's memory).

[0207] For example, in some other embodiments, the multiple images of the target scene can be images uploaded by other devices, such as a decoding device. Taking a decoding device as an example, if the decoding device uploads multiple images of the target scene to the encoding device, the process of the encoding device acquiring the multiple images of the target scene can be: the encoding device receives the multiple images of the target scene from the decoding device.

[0208] The following describes the process of the decoding device uploading multiple images of the target scene to the encoding device.

[0209] The decoding device can run software programs that support 3D model reconstruction. By inputting multiple images of the target scene into the software program, it can output a 3DGS model of the target scene based on the multiple images of the target scene, thereby providing users with a highly realistic and interactive virtual model based on the 3DGS model.

[0210] In some embodiments, multiple images of the target scene can be obtained by triggering a shooting control within the software program. For example, the software program interface can provide a shooting control used to trigger a shooting function to obtain multiple images of the target scene. Exemplarily, when a user needs to reconstruct a 3D model of the target scene, they can trigger the shooting control in the software program interface. The decoding device, in response to the triggering operation of the shooting control, calls its camera to capture images, obtaining multiple images of the target scene. The triggering operation can be a click, double-click, long-press, or other operation; this embodiment does not limit the specific type of operation.

[0211] In some embodiments, multiple images of the target scene can be obtained by triggering a selection control within the software program. For example, the software program's interface can provide a selection control used to trigger the selection of multiple images from multiple candidate images to obtain multiple images of the target scene. Exemplarily, when a user needs to reconstruct a 3D model of the target scene, they can trigger the selection control in the software program's interface. The decoding device, in response to the triggering operation of the selection control, displays multiple candidate images stored locally. At this time, the user can perform a selection operation among the displayed candidate images, and the decoding device, in response to the selection operation of multiple images of the target scene, obtains multiple images of the target scene. The triggering operation can be a click operation, a double-click operation, a long-press operation, or other operations; this application embodiment does not limit this.

[0212] It is worth noting that in other embodiments, the decoding device may also employ other possible implementations to acquire multiple images of the target scene. This application does not limit this approach.

[0213] Furthermore, in some embodiments, the decoding device sends multiple images of the target scene to the encoding device. For example, the software program interface may also provide an upload control, which is used to trigger the decoding device to send multiple images of the target scene to the encoding device. Exemplarily, after the decoding device obtains multiple images of the target scene, the user can trigger the upload control, and the decoding device, in response to the triggering operation of the upload control, sends the multiple images of the target scene to the encoding device.

[0214] S702: The encoding device encodes multiple Gaussian spheres in the 3DGS model to obtain the encoded data of the 3DGS model, and then encodes the encoded data into the bitstream.

[0215] The bitstream includes the encoded data of the 3DGS model. Encoded data refers to the data obtained by encoding the feature data of each Gaussian sphere in the 3DGS model.

[0216] In some embodiments, the plurality of Gaussian spheres includes a first Gaussian sphere and a second Gaussian sphere, and the first Gaussian sphere and the second Gaussian sphere have different encoding methods.

[0217] In some embodiments, multiple Gaussian spheres belong to N Gaussian sphere groups, where N is a positive integer greater than 1. Each of the N Gaussian sphere groups has a different encoding method, meaning that each Gaussian sphere group corresponds to one encoding method. Similarly, each of the N Gaussian sphere groups has a different decoding method, meaning that each Gaussian sphere group corresponds to one decoding method.

[0218] For example, the first Gaussian sphere and the second Gaussian sphere belong to different groups among N Gaussian sphere groups, such as the first Gaussian sphere belonging to the first Gaussian sphere group and the second Gaussian sphere belonging to the second Gaussian sphere group. This means that the encoding methods for the first Gaussian sphere and the second Gaussian sphere are different, and the decoding methods for the first Gaussian sphere and the second Gaussian sphere are different.

[0219] In this embodiment, the encoding methods for Gaussian spheres in different groups are different. That is, when the encoding device encodes each Gaussian sphere in the 3DGS model, it can encode the Gaussian spheres according to the encoding method corresponding to the group to which each Gaussian sphere belongs.

[0220] Furthermore, the bitstream also includes indication information for each Gaussian sphere to indicate the encoding or decoding method of each Gaussian sphere. For example, taking a first Gaussian sphere and a second Gaussian sphere as an example, the bitstream also includes first and second indication information. The first indication information indicates the encoding or decoding method for the first Gaussian sphere. The second indication information indicates the encoding or decoding method for the second Gaussian sphere. The first and second indication information are different. Thus, by carrying the indication information for each Gaussian sphere in the bitstream, the encoding or decoding method of each Gaussian sphere can be indicated during the decoding stage.

[0221] In some embodiments of this application, the indication information of each Gaussian sphere is used to indicate the group to which each Gaussian sphere belongs. For example, taking a first Gaussian sphere and a second Gaussian sphere comprising multiple Gaussian spheres as an example, the first indication information is used to indicate the group to which the first Gaussian sphere belongs. The second indication information is used to indicate the group to which the second Gaussian sphere belongs. Thus, by using the indication information of each Gaussian sphere, the group to which each Gaussian sphere belongs can be determined, and consequently, the encoding or decoding method corresponding to the Gaussian sphere can be determined based on the group to which the Gaussian sphere belongs.

[0222] For example, different character flags can be used to represent different groups in the indication information of each Gaussian sphere. Taking the first Gaussian sphere group and the second Gaussian sphere group as examples, 0 can be used to refer to the first Gaussian sphere group and 1 can be used to refer to the second Gaussian sphere group.

[0223] In other embodiments of this application, the bitstream includes a range for each of N Gaussian sphere groups, which is used to determine the group to which each Gaussian sphere belongs based on the indication information of each Gaussian sphere. For example, taking a first Gaussian sphere and a second Gaussian sphere as examples, the range is used to determine the group to which the first Gaussian sphere belongs based on first indication information, and to determine the group to which the second Gaussian sphere belongs based on second indication information.

[0224] For example, different numerical serial numbers can be used to represent different groups in the indication information of each Gaussian sphere. In this way, the serial number of each Gaussian sphere can be used to uniquely identify the corresponding Gaussian sphere.

[0225] For example, the range of each Gaussian sphere group in the N Gaussian sphere groups can be represented by an index interval. It is worth noting that the index intervals of different groups do not overlap. Thus, the index interval of each group can be used to uniquely identify the corresponding group. Understandably, in some embodiments, the encoding device can sort the Gaussian spheres in each of the N Gaussian sphere groups to obtain the index of each Gaussian sphere within its respective group.

[0226] It is worth noting that Gaussian spheres belonging to the same group have consecutive ordinal numbers within that group. This consecutive numbering simplifies the sorting process; simply arranging the Gaussian spheres in numerical order yields the ordinal number of each sphere within its group and the range of that group (e.g., an interval of numbers), thus improving sorting efficiency. Furthermore, consecutive ordinal numbers also facilitate data maintenance and management, clearly representing each Gaussian sphere and making it easier to identify its group based on its ordinal number.

[0227] Taking the first and second Gaussian sphere groups as examples, (1, 10) can be used to represent the range of the first Gaussian sphere group, and (11, 20) can be used to represent the range of the second Gaussian sphere group. Taking a first and second Gaussian sphere group comprising multiple Gaussian spheres as an example, the first indication information is used to indicate the sequence number of the first Gaussian sphere within its group, such as sequence number 5. The second indication information is used to indicate the sequence number of the second Gaussian sphere within its group, such as sequence number 15.

[0228] The two embodiments described above provide two ways to express the indication information of the Gaussian sphere. It is worth noting that in other possible implementations, the indication information of the Gaussian sphere can also be expressed in other ways, and this application does not limit such expressions.

[0229] S703, the decoding device receives the bit stream.

[0230] S704. The decoding device decodes the encoded data of the first Gaussian sphere according to the first instruction information to obtain the first Gaussian sphere, and decodes the encoded data of the second Gaussian sphere according to the second instruction information to obtain the second Gaussian sphere.

[0231] The technical solution provided in this application, during the encoding stage, encodes multiple Gaussian spheres of a 3DGS model to obtain encoded data of the 3DGS model, and then encodes the encoded data into a bitstream. Furthermore, the bitstream carries first and second indication information to indicate the encoding or decoding methods of the first and second Gaussian spheres. Since the first and second indication information are different, it indicates that the encoding and decoding methods of the first and second Gaussian spheres are different. This allows for different encoding and decoding methods to be used for different Gaussian spheres, effectively improving the encoding and decoding performance. Furthermore, during the decoding stage, the encoded data of the first Gaussian sphere can be decoded according to the encoding or decoding method indicated by the first indication information to obtain the first Gaussian sphere, and the encoded data of the second Gaussian sphere can be decoded according to the encoding or decoding method indicated by the second indication information to obtain the second Gaussian sphere. Thus, by carrying the first and second indication information in the bitstream, the amount of information included in the bitstream is increased, as is the amount of information that can be referenced during the decoding stage, which can further and effectively improve the decoding effect.

[0232] In some embodiments of this application, the decoding methods for Gaussian spheres in different groups are different. That is, when the decoding device decodes each Gaussian sphere in the 3DGS model, it can decode the Gaussian sphere according to the decoding method corresponding to the group to which each Gaussian sphere belongs.

[0233] For the multiple Gaussian spheres in the 3DGS model shown in Figure 7, during the encoding stage, the encoding device further groups the Gaussian spheres and encodes each sphere according to the encoding method corresponding to its group. Similarly, during the decoding stage, the decoding device identifies the group to which each Gaussian sphere belongs and decodes it according to the decoding method corresponding to that group. Thus, by referencing the group to which each Gaussian sphere belongs, different encoding and decoding methods are used for different groups, increasing the amount of information referenced during encoding and decoding, and effectively improving the encoding and decoding performance.

[0234] It should be understood that in the flowchart of the 3DGS model encoding and decoding method shown in Figure 7, the encoding and decoding methods can be independent of each other. The encoding method is executed by the encoding device, and the decoding method is executed by the decoding device. The encoding device and the decoding device can be independent devices or the same device. In addition, the encoded bitstream can be stored in a storage device, such as a computer-readable storage medium, or it can be transmitted to other devices, or distributed to other devices by a distribution device or distribution network.

[0235] The following section provides a detailed explanation of the encoding and decoding process for 3DGS models, based on Figure 8.

[0236] Figure 8 is a flowchart illustrating another 3DGS model encoding and decoding method provided in an embodiment of this application. Referring to Figure 8, taking the interaction flow between the encoding device and the decoding device as an example, the method includes the following steps S801-S807.

[0237] S801, The encoding device acquires the 3DGS model.

[0238] S801 is similar to S701 in Figure 7 above, and will not be described again.

[0239] S802, the encoding device groups multiple Gaussian spheres in the 3DGS model to obtain N Gaussian sphere groups.

[0240] In this embodiment, the Gaussian spheres in different groups have varying degrees of influence on the rendering effect. That is, in some possible implementations, the group to which each Gaussian sphere in the 3DGS model belongs can be determined based on the degree of influence each Gaussian sphere has on the rendering effect.

[0241] For example, taking the division into two groups as an example, Gaussian spheres with a greater impact on the rendering effect can be grouped into one group, and Gaussian spheres with a lesser impact on the rendering effect can be grouped into another group. For example, in the embodiments of this application, the first Gaussian sphere group can be the group with a greater impact on the rendering effect, and the second Gaussian sphere group can be the group with a lesser impact on the rendering effect. It is understood that the Gaussian spheres in the first Gaussian sphere group have a greater impact on the rendering effect than the Gaussian spheres in the second Gaussian sphere group.

[0242] Thus, based on the degree of influence of each Gaussian sphere on the rendering effect, N Gaussian sphere groups are obtained, allowing for the application of different encoding methods to different groups. For example, Gaussian spheres in groups with a higher impact on the rendering effect can use an encoding method with a lower compression ratio, while those in groups with a lower impact can use an encoding method with a higher compression ratio. On the one hand, this allows for more effective compression of the 3DGS model while ensuring rendering quality, improving the compression ratio of the 3DGS model. On the other hand, it reduces the transmission time of the 3DGS model, which in turn reduces the time required to load model data from memory during rendering, thereby improving rendering speed.

[0243] In some embodiments, the encoding device can obtain the influencing factors of each Gaussian sphere on the rendering effect, and based on the influencing factors of each Gaussian sphere on the rendering effect, group multiple Gaussian spheres in the 3DGS model to obtain N Gaussian sphere groups.

[0244] The factors influencing the rendering effect can include at least one of importance, foreground detection results, and semantic recognition results. This means that the encoding device can group based on one, two, or all three of these factors. This provides three factors influencing the rendering effect, increasing the amount of information available for grouping and effectively improving the accuracy of the grouping.

[0245] The following section introduces the factors that affect the rendering effect of the Gaussian sphere and the process of grouping these factors based on their impact on the rendering effect.

[0246] Influencing factor 1: Importance.

[0247] Importance is used to indicate the degree of impact on the rendering effect.

[0248] In some embodiments, the importance of Gaussian spheres is obtained, and multiple Gaussian spheres in the 3DGS model are grouped based on the importance of the Gaussian spheres.

[0249] In one possible implementation, the process of obtaining the importance of the Gaussian sphere may include: obtaining the importance of higher-order sphere covariances in the feature data of the Gaussian sphere, and determining the importance of the Gaussian sphere based on the importance of the higher-order sphere covariances.

[0250] Understandably, the feature data of a Gaussian sphere contains multiple higher-order sphere coefficients. For example, after obtaining the importance of these multiple higher-order sphere coefficients in the feature data of a Gaussian sphere, the higher-order sphere coefficient with the highest importance value can be determined as the importance of the Gaussian sphere.

[0251] Given multiple images of a target scene captured from different perspectives, a 3DGS model can render the corresponding images from each perspective, resulting in a rendered image, such as {R}. i |i∈[0,M)}, where M is the number of images captured of the target scene, and R i This is the rendered image obtained from each image. The pixel colors in the rendered image are summed to obtain the pixel color sum value, such as {C}. i |i∈[0,M)},C i The pixel color and value of each image after rendering. For each higher-order spherical coefficient of the Gaussian sphere, the process of obtaining the importance of the higher-order spherical coefficient can be: based on the higher-order spherical coefficient, the pixel color and value of the rendered image and the following formula (3), determine the importance of the higher-order spherical coefficient.

[0252] In the formula, S i Indicates the importance of higher-order ballistic coefficients; Ci This represents the pixel color and value of the rendered image; h i Represents higher-order ball coefficients; This indicates the degree of change in pixel color and value caused by perturbations of higher-order spherical coefficients, that is, the degree of influence of perturbations of higher-order spherical coefficients on rendered color. Understandably, the greater the influence of perturbations of higher-order spherical coefficients on rendered color, the higher the importance of the higher-order spherical coefficients.

[0253] It is worth noting that, in addition to the method of calculating importance shown in formula (3) above, other methods can also be used to calculate the importance of higher-order ball coefficients, such as the light gaussian method. This application does not limit this method.

[0254] In this embodiment, the influence of Gaussian spheres with importance greater than the target threshold on the rendering effect is greater than the influence of Gaussian spheres with importance less than the target threshold. That is, Gaussian spheres with importance greater than the target threshold have a greater influence on the rendering effect, while Gaussian spheres with importance less than the target threshold have a smaller influence. The target threshold is a pre-set importance threshold.

[0255] For example, taking the determination of the group to which a Gaussian sphere belongs based on its importance by an encoding device, assuming the group is divided into two groups: a first Gaussian sphere group and a second Gaussian sphere group, the group to which the Gaussian sphere belongs can be determined by judging whether the importance of the Gaussian sphere is greater than a target threshold. If the importance of the Gaussian sphere is greater than the target threshold, it indicates that the Gaussian sphere has a greater impact on the rendering effect, and the Gaussian sphere is determined to belong to the first Gaussian sphere group. If the importance of the Gaussian sphere is less than the target threshold, it indicates that the Gaussian sphere has a smaller impact on the rendering effect, and the Gaussian sphere is determined to belong to the second Gaussian sphere group. It should be understood that multiple target thresholds can be used to divide the group, and the group to which the Gaussian sphere belongs can be determined according to the importance of the Gaussian sphere and different target thresholds.

[0256] In the above embodiments, a method is provided to determine the group to which a Gaussian ball belongs based on the importance of the Gaussian ball, which can quickly group multiple Gaussian balls, thereby improving the efficiency of encoding.

[0257] Factor 2: Foreground or background detection results.

[0258] The foreground or background detection results are used to indicate whether the Gaussian sphere belongs to the foreground or background region.

[0259] In some embodiments, the detection results of Gaussian spheres are obtained, and multiple Gaussian spheres in the 3DGS model are grouped based on the detection results of Gaussian spheres.

[0260] In one possible implementation, the process of obtaining the detection results of Gaussian spheres may include: performing foreground or background detection on the rendered image to obtain the foreground and background regions of the rendered image; determining whether the pixel region corresponding to each Gaussian sphere in the rendered image belongs to the foreground region or the background region to obtain the detection results of each Gaussian sphere.

[0261] Here, the pixel region corresponding to each Gaussian sphere refers to the pixel region formed when the Gaussian sphere is projected onto the two-dimensional image plane during the rendering process. Optionally, frame difference, optical flow, or deep learning-based methods can be used to perform foreground or background detection on the rendered image. This application does not limit this approach.

[0262] In this embodiment, the influence of Gaussian spheres belonging to the foreground region on the rendering effect is greater than that of Gaussian spheres belonging to the background region. In other words, Gaussian spheres belonging to the foreground region have a greater influence on the rendering effect, while Gaussian spheres belonging to the background region have a smaller influence.

[0263] For example, taking the determination of the Gaussian sphere's grouping based on the foreground detection results of the Gaussian sphere by the encoding device as an example, assuming the grouping is divided into two groups: a first Gaussian sphere group and a second Gaussian sphere group, the grouping of the Gaussian sphere can be determined by judging whether it belongs to the foreground or background region. If the Gaussian sphere belongs to the foreground region, it indicates that the Gaussian sphere has a greater impact on the rendering effect, and thus the grouping of the Gaussian sphere is determined to be the first Gaussian sphere group. If the Gaussian sphere belongs to the background region, it indicates that the Gaussian sphere has a smaller impact on the rendering effect, and thus the grouping of the Gaussian sphere is determined to be the second Gaussian sphere group.

[0264] In the above embodiments, a method is provided to determine the group to which a Gaussian ball belongs based on the foreground detection result of the Gaussian ball. This method can also quickly group multiple Gaussian balls, thereby improving the efficiency of encoding.

[0265] The third influencing factor is the semantic recognition result.

[0266] The semantic recognition results are used to indicate the semantic type of the Gaussian sphere, such as person, object, or background.

[0267] In some embodiments, the semantic recognition results corresponding to the Gaussian spheres are obtained, and the multiple Gaussian spheres in the 3DGS model are grouped according to the semantic recognition results.

[0268] In one possible implementation, the process of obtaining the semantic recognition results corresponding to Gaussian spheres may include: performing semantic recognition on the rendered image to obtain the semantic recognition results of each entity object in the rendered image; determining the entity objects corresponding to each Gaussian sphere based on the pixel regions corresponding to each Gaussian sphere in the rendered image; and using the semantic recognition results of the entity objects corresponding to each Gaussian sphere as the semantic recognition results of each Gaussian sphere.

[0269] In this rendered image, each entity object can be represented using a rectangular detection box. For example, the entity object corresponding to each Gaussian sphere is determined by identifying the rectangular detection boxes associated with the pixel regions corresponding to each Gaussian sphere. Optionally, a deep learning-based method, such as a semantic recognition model trained on a deep neural network, can be used to perform semantic recognition on the rendered image.

[0270] In this embodiment, the influence of a Gaussian sphere belonging to the first semantic type on the rendering effect is greater than that of a Gaussian sphere belonging to the second semantic type. That is, the Gaussian sphere belonging to the first semantic type has a greater influence on the rendering effect, while the Gaussian sphere belonging to the second semantic type has a smaller influence. For example, the first semantic type can be a character type, and the second semantic type can be an item type.

[0271] For example, taking the determination of the group to which a Gaussian sphere belongs based on the semantic recognition result of the Gaussian sphere by the encoding device, assuming it is divided into two groups, a first Gaussian sphere group and a second Gaussian sphere group, the group to which the Gaussian sphere belongs can be determined by judging whether the semantic type of the Gaussian sphere belongs to the first semantic type or the second semantic type. If the Gaussian sphere belongs to the first semantic type, it means that the Gaussian sphere has a greater impact on the rendering effect, and the group to which the Gaussian sphere belongs is determined to be the first Gaussian sphere group. If the Gaussian sphere belongs to the second semantic type, it means that the Gaussian sphere has a smaller impact on the rendering effect, and the group to which the Gaussian sphere belongs is determined to be the second Gaussian sphere group.

[0272] In the above embodiments, a method is provided to determine the group to which a Gaussian ball belongs based on the semantic recognition result of the Gaussian ball. This method can also quickly group multiple Gaussian balls, thereby improving the efficiency of encoding.

[0273] The above embodiments illustrate the grouping process by using an encoding device to determine the group to which a Gaussian sphere belongs based on one of the following: the importance of the Gaussian sphere, the foreground detection result, and the semantic recognition result. Furthermore, in other embodiments, the encoding device may also determine the group to which the Gaussian sphere belongs based on two or three of the following: the importance of the Gaussian sphere, the foreground detection result, and the semantic recognition result.

[0274] For example, the encoding device can determine the group to which the Gaussian sphere belongs based on the importance of the Gaussian sphere and the foreground detection result. Alternatively, the encoding device can determine the group to which the Gaussian sphere belongs based on the importance of the Gaussian sphere and the semantic recognition result. Or, the encoding device can determine the group to which the Gaussian sphere belongs based on the foreground detection result and the semantic recognition result. Furthermore, the encoding device can determine the group to which the Gaussian sphere belongs based on the importance of the Gaussian sphere, the foreground detection result, and the semantic recognition result. This application embodiment does not limit this approach.

[0275] When determining the group to which a Gaussian sphere belongs based on two or three of the following factors: the importance of the Gaussian sphere, the foreground detection result, and the semantic recognition result, a preset classification rule can be used. For example, taking the determination of the group to which a Gaussian sphere belongs based on the foreground detection result and the semantic recognition result by the encoding device, assuming it is divided into three groups: Group 1, Group 2, and Group 3, the group to which the Gaussian sphere belongs can be determined by judging whether the Gaussian sphere belongs to the foreground region or the background region, and by judging whether the Gaussian sphere's semantic type belongs to the first semantic type or the second semantic type. If the Gaussian sphere belongs to the foreground region and belongs to the first semantic type, then the Gaussian sphere belongs to Group 1. If the Gaussian sphere belongs to the foreground region and belongs to the second semantic type, then the Gaussian sphere belongs to Group 2. If the Gaussian sphere belongs to the background region, then the Gaussian sphere belongs to Group 3. For example, a Gaussian sphere belonging to the foreground region and belonging to a person can be classified as Group 1, a Gaussian sphere belonging to the foreground region and belonging to an object can be classified as Group 2, and a Gaussian sphere belonging to the background can be classified as Group 3.

[0276] Alternatively, when determining the group to which a Gaussian sphere belongs based on two or three of the following factors: importance, foreground detection results, and semantic recognition results, a scoring method can be used to obtain scores for importance, foreground detection results, or semantic recognition results. These scores are then weighted and summed to obtain a comprehensive score, which is used to determine the group to which the Gaussian sphere belongs. A higher score indicates a greater impact on the rendering effect. For example, taking the determination of the Gaussian sphere's grouping based on its importance, foreground detection results, and semantic recognition results by an encoding device as an example, the first score for importance, the second score for foreground detection results, and the third score for semantic recognition results can be obtained separately. These three scores are then weighted and summed to obtain a comprehensive score. Assuming the group is divided into two groups, Group 1 and Group 2, if the comprehensive score is greater than a preset score, it indicates a greater impact of the Gaussian sphere on the rendering effect, and the Gaussian sphere is assigned to Group 1. If the comprehensive score is less than the preset score, it indicates a smaller impact of the Gaussian sphere on the rendering effect, and the Gaussian sphere is assigned to Group 2. Assuming the rendering is divided into three groups: Group 1, Group 2, and Group 3, if the overall score is greater than the first preset score, it indicates that the Gaussian sphere has a significant impact on the rendering effect, and the Gaussian sphere is assigned to Group 1. If the overall score is greater than the second preset score but less than the first preset score, it indicates that the Gaussian sphere has a moderate impact on the rendering effect, and the Gaussian sphere is assigned to Group 2, where the first preset score is greater than the second preset score. If the overall score is less than the second preset score, it indicates that the Gaussian sphere has a minor impact on the rendering effect, and the Gaussian sphere is assigned to Group 3.

[0277] It is worth noting that in other embodiments, the encoding device may also use other methods to group multiple Gaussian spheres in the 3DGS model, and this application embodiment does not limit this.

[0278] S803: The encoding device encodes each Gaussian sphere according to the encoding method corresponding to the group to which each Gaussian sphere belongs, obtains the encoded data of the 3DGS model, and encodes the encoded data into the bitstream.

[0279] In one implementation, among N Gaussian sphere groups, the encoding method corresponding to the Gaussian sphere group that has a greater impact on the rendering effect (such as the first Gaussian sphere group) has a lower compression rate, while the encoding method corresponding to the Gaussian sphere group that has a smaller impact on the rendering effect (such as the second Gaussian sphere group) has a higher compression rate.

[0280] Taking a first Gaussian sphere and a second Gaussian sphere as an example, the encoding process of the encoding device for multiple Gaussian spheres can be as follows: First, the feature data of the first Gaussian sphere is quantized to obtain the encoded data of the first Gaussian sphere. Second, the feature data of the second Gaussian sphere is quantized to obtain the encoded data of the second Gaussian sphere. The compression ratio of the first quantization is lower than that of the second quantization.

[0281] In the above embodiments, a lower compression ratio encoding method is used for the Gaussian spheres in the first Gaussian sphere group. A higher compression ratio encoding method is used for the Gaussian spheres in the second Gaussian sphere group. Thus, by using different encoding methods for the Gaussian spheres in different Gaussian sphere groups, Gaussian spheres with a greater impact on rendering effects (i.e., relatively important Gaussian spheres) are encoded using a lower compression ratio, higher bitrate, and less loss encoding method, while Gaussian spheres with a less significant impact on rendering effects (i.e., relatively unimportant Gaussian spheres) are encoded using a higher compression ratio, lower bitrate, and greater loss encoding method. This achieves two goals: firstly, it allows for more effective compression of the 3DGS model while maintaining rendering quality, improving the compression ratio of the 3DGS model; secondly, it reduces the transmission time of the 3DGS model, which in turn reduces the time spent loading model data from memory during rendering, thereby improving rendering speed.

[0282] In some embodiments, the first quantization refers to an encoding method with a low compression ratio, such as scalar quantization. Accordingly, the process by which the encoding device performs the first quantization on the feature data of the first Gaussian sphere can be: performing scalar quantization on the feature data of the first Gaussian sphere.

[0283] For example, the feature data of the first Gaussian sphere can be scalar quantized according to the bit depth shown in Table 2.

[0284] Table 2

[0285] The bit depth configuration for the center point is (11_10_11), representing the bit depth of the three coordinate values ​​of the center point in three dimensions: the X-coordinate has a bit depth of 11, the Y-coordinate has a bit depth of 10, and the Z-coordinate has a bit depth of 11. The bit depth configuration for scale information is (11_10_11), representing the bit depth of the scale values ​​on the three coordinate axes: the X-coordinate has a bit depth of 11, the Y-coordinate has a bit depth of 10, and the Z-coordinate has a bit depth of 11. The bit depth configuration for spatial orientation information is (2_10_10_10), representing the bit depth of each value in the quaternion of spatial orientation information. The bit depth configuration for opacity is 8, representing the bit depth of the opacity value. The bit depth configuration for diffuse color is (8_8_8), representing the bit depth of the diffuse color values ​​in each color channel. The bit depth configuration of the higher-order ball coefficients is (8_8_8), which means that the bit depth of the color value of each higher-order ball coefficient in each color channel is represented. It should be understood that higher-order ball coefficients can include 15 coefficients, each of which includes color values ​​in three color channels.

[0286] In some embodiments, the feature data of the second Gaussian sphere includes first feature data and second feature data. Accordingly, the process of the encoding device performing a second quantization on the feature data of the second Gaussian sphere may be: performing a third quantization on the first feature data and a fourth quantization on the second feature data.

[0287] The second quantization refers to encoding methods with higher compression ratios, including the third and fourth quantizations mentioned above.

[0288] The first feature data may include the center point, scale information, spatial orientation information, opacity, and diffuse color. The third quantization may be scalar quantization. Accordingly, the encoding device performing the third quantization on the first feature data may be: the encoding device performs scalar quantization on the center point, scale information, spatial orientation information, opacity, and diffuse color.

[0289] The second feature data may include higher-order spherical coefficients. The fourth quantization may include scalar quantization and vector quantization, or it may include scalar quantization and product quantization. Accordingly, the encoding device may perform the fourth quantization on the second feature data using the following two quantization methods.

[0290] Quantization Method 1: The encoding device performs scalar quantization on the second feature data and then performs vector quantization on the scalar quantization result.

[0291] In the above implementation, for the Gaussian spheres in the second Gaussian sphere group, after scalar quantization of the higher-order sphere covariance coefficients, vector quantization is further performed, which can further compress the higher-order sphere covariance coefficients. Since the higher-order sphere covariance coefficients account for the largest proportion of the feature data of the Gaussian spheres, further compression of the higher-order sphere covariance coefficients can achieve further compression of the 3DGS model, thereby improving the compression ratio of the 3DGS model.

[0292] In the encoding process based on quantization method one, scalar quantization is performed on the center point, scale information, spatial orientation information, opacity, and diffuse color of the second Gaussian sphere. The higher-order spherical covariances of the second Gaussian sphere are first scalar quantized, and then the scalar quantization result is vector quantized.

[0293] For example, the center point, scale information, spatial orientation information, opacity, and diffuse color of the second Gaussian sphere can be scalar quantized according to the bit depth shown in Table 3. The higher-order spherical covariances of the second Gaussian sphere are first scalar quantized, such as with a bit depth of (8_8_8), i.e., 8 bits. Then, the higher-order spherical covariances can be treated as vectors and vector quantized. For example, the vector quantization result of the higher-order spherical covariances can be a code table of size 4096 with an index bit depth of 12.

[0294] Table 3

[0295] Quantization Method 2: The encoding device performs scalar quantization on the second feature data and then performs product quantization on the scalar quantization result.

[0296] In the above implementation, for the Gaussian spheres in the second Gaussian sphere group, after scalar quantization of the higher-order sphere covariance coefficients, product quantization is further performed, which can further compress the higher-order sphere covariance coefficients. Since the higher-order sphere covariance coefficients account for the largest proportion of the feature data of the Gaussian spheres, further compression of the higher-order sphere covariance coefficients can achieve further compression of the 3DGS model, thereby improving the compression ratio of the 3DGS model.

[0297] In the encoding process based on quantization method two, scalar quantization is performed on the center point, scale information, spatial orientation information, opacity, and diffuse color of the second Gaussian sphere. The higher-order spherical covariances of the second Gaussian sphere are first scalar quantized, and then the scalar quantization results are multiplicatively quantized.

[0298] For example, the center point, scale information, spatial orientation information, opacity, and diffuse color of the second Gaussian sphere can be scalar quantized according to the bit depth shown in Table 4. The higher-order spherical covariances of the second Gaussian sphere are first scalar quantized, such as with a bit depth of (8_8_8), i.e., 8 bits. Then, the higher-order spherical covariances can be treated as vectors and product quantization can be performed on them.

[0299] Table 4

[0300] In addition to quantization methods one and two mentioned above, some embodiments also include quantization method three, where the bitstream does not contain the second feature data. This means that higher-order spherical coefficients are discarded during the encoding stage. Since higher-order spherical coefficients constitute the largest proportion of the Gaussian sphere's feature data, discarding them allows for further compression of the 3DGS model, thereby improving its compression ratio.

[0301] In the encoding process based on quantization method three, the center point, scale information, spatial orientation information, opacity, and diffuse color of the second Gaussian sphere are scalar quantized. Higher-order spherical covariance coefficients are discarded.

[0302] For example, the characteristic data of the second Gaussian sphere can be scalar quantized according to the bit depth shown in Table 5. The higher-order sphere covariances can be discarded.

[0303] Table 5

[0304] In the exemplary table shown in the above embodiments, the scheme is illustrated using the example of different Gaussian spheres having the same scalar quantization bit depth for their center point, scale information, spatial orientation information, opacity, and diffuse color. In other embodiments, since the encoding and decoding of different Gaussian spheres are completely independent, the scalar quantization bit depth for the center point, scale information, spatial orientation information, opacity, and diffuse color of different Gaussian spheres can also be different. Of course, the scalar quantization bit depth for the higher-order spherical covariances of different Gaussian spheres can also be different. In some embodiments, different scalar quantization bit depths can be used for Gaussian spheres in different Gaussian sphere groups. For example, a lower scalar quantization bit depth can be used for Gaussian spheres in the first Gaussian sphere group, resulting in a higher compression ratio. A higher scalar quantization bit depth can be used for Gaussian spheres in the second Gaussian sphere group to ensure rendering quality.

[0305] S804: The encoding device acquires the indication information of multiple Gaussian spheres in the 3DGS model and encodes the indication information of multiple Gaussian spheres into the bitstream.

[0306] The indication information for each Gaussian sphere is used to indicate the group to which each Gaussian sphere belongs.

[0307] In some embodiments, different character flag bits can be used to represent different groups in the indication information of each Gaussian sphere. Accordingly, the process of obtaining the indication information of multiple Gaussian spheres in a 3DGS model can be: the encoding device adds different flag bits to the Gaussian spheres in different Gaussian sphere groups.

[0308] Taking the first and second Gaussian sphere groups as examples, we can add a flag of 0 to the Gaussian spheres in the first Gaussian sphere group and a flag of 1 to the Gaussian spheres in the second Gaussian sphere group. In this way, we can use 0 to refer to the first Gaussian sphere group and 1 to refer to the second Gaussian sphere group.

[0309] In other embodiments, different numerical indices can be used to represent different groups in the indication information of each Gaussian sphere. Accordingly, the process of obtaining the indication information of multiple Gaussian spheres in a 3DGS model can be as follows: the encoding device sorts the Gaussian spheres in each of the N Gaussian sphere groups to obtain the indices of each Gaussian sphere in its respective group.

[0310] It is worth noting that in the embodiment where groups are identified based on sequence numbers, the bitstream also includes the range of each of the N Gaussian sphere groups. For example, the range of each of the N Gaussian sphere groups can be represented by a sequence number interval. It is important to note that the sequence number intervals of different groups do not overlap.

[0311] In addition, the bitstream also includes grouping information for N Gaussian sphere groups. This grouping information includes one or more of the following: the number of groups, the number of Gaussian spheres in each group, and the encoding information for each group. This allows the subsequent decoding device to decode multiple Gaussian spheres based on the grouping information of the N Gaussian sphere groups, thereby obtaining multiple Gaussian spheres. It is understandable that the grouping information serves as auxiliary information for decoding and rendering.

[0312] In the encoding / decoding method based on the above-described quantization method one, the encoding information for each group may include one or more of the following: characteristics of the Gaussian sphere for each group, quantization method, quantization bit depth, numerical range (e.g., maximum and minimum values), higher-order sphere co-coefficients, code table size, and index bit depth. It is understandable that in quantization method one, since the higher-order sphere co-coefficients are vector quantized, the encoding information needs to carry the code table size and index bit depth to assist in decoding and rendering.

[0313] In the encoding / decoding method based on quantization mode two described above, the encoding information for each group may include one or more of the following: the feature name of the Gaussian sphere for each group, the quantization mode, the quantization bit depth, the numerical range, the higher-order sphere co-coefficient, the method of dividing the feature vector into multiple sub-vectors, the code table size of each sub-vector, and the index bit depth of each sub-vector. It is understandable that in quantization mode two, since the higher-order sphere co-coefficients undergo product quantization, the encoding information needs to carry the method of dividing the feature vector into multiple sub-vectors, the code table size of each sub-vector, and the bit depth of each sub-vector index to assist in decoding and rendering.

[0314] In the encoding and decoding method based on quantization mode three described above, the encoding information for each group may include one or more of the following: the characteristic name of the Gaussian sphere for each group, quantization mode, quantization bit depth, numerical range, and higher-order sphere co-order. It is understood that in quantization mode three, since the higher-order sphere co-coefficients are discarded, the higher-order sphere co-order in the encoding information may be 0.

[0315] The contents of S802 to S804 in the embodiment of Figure 8 can correspond to the encoding process of S702 in the embodiment of Figure 7.

[0316] The above describes the encoding process on the encoding side of the 3DGS model. The following describes the decoding process on the decoding side of the 3DGS model.

[0317] S805, the decoding device receives the bit stream.

[0318] S806. The decoding device determines the group to which each Gaussian sphere belongs based on the indication information of each Gaussian sphere in the bitstream.

[0319] In some embodiments of this application, taking a first Gaussian sphere and a second Gaussian sphere comprising a plurality of Gaussian spheres as an example, the decoding device determines the group to which the first Gaussian sphere belongs based on a first indication information. The decoding device determines the group to which the second Gaussian sphere belongs based on a second indication information.

[0320] For example, the first indication information is used to indicate the group to which the first Gaussian sphere belongs, and the second indication information is used to indicate the group to which the second Gaussian sphere belongs. Taking the use of different character flag bits in the indication information to represent different groups as an example, the decoding device can determine the group to which the Gaussian sphere belongs by judging the characters of the flag bits in the indication information. For example, in response to the first indication information having a flag bit of 0, the decoding device determines that the first Gaussian sphere belongs to the first Gaussian sphere group. In response to the first indication information having a flag bit of 1, the decoding device determines that the first Gaussian sphere belongs to the second Gaussian sphere group.

[0321] In other embodiments of this application, the bitstream includes the range of each of the N Gaussian sphere groups. The decoding device can determine the group to which the first Gaussian sphere belongs based on the first indication information and the range of each of the N Gaussian sphere groups. The decoding device can determine the group to which the second Gaussian sphere belongs based on the second indication information and the range of each of the N Gaussian sphere groups.

[0322] The first indication information indicates the sequence number of the first Gaussian sphere in its respective group, and the second indication information indicates the sequence number of the second Gaussian sphere in its respective group. Accordingly, the decoding device can determine the group to which the first Gaussian sphere belongs based on its sequence number and the range of each of the N Gaussian sphere groups. Similarly, the decoding device can determine the group to which the second Gaussian sphere belongs based on its sequence number and the range of each of the N Gaussian sphere groups.

[0323] For example, the decoding device determines, within the range of each of the N Gaussian sphere groups, the Gaussian sphere group corresponding to the range of the sequence number indicated by the first indication information, and designates it as the group to which the first Gaussian sphere belongs. The decoding device then determines, within the range of each of the N Gaussian sphere groups, the Gaussian sphere group corresponding to the range of the sequence number indicated by the second indication information, and designates it as the group to which the second Gaussian sphere belongs.

[0324] S807 The decoding device decodes the encoded data of each Gaussian sphere according to the decoding method corresponding to the group to which each Gaussian sphere belongs, so as to obtain the 3DGS model.

[0325] In some embodiments, taking a first Gaussian sphere and a second Gaussian sphere as an example, the decoding process of the decoding device for each Gaussian sphere may include: performing a first inverse quantization on the encoded data of the first Gaussian sphere to obtain the first Gaussian sphere; and performing a second inverse quantization on the encoded data of the second Gaussian sphere to obtain the second Gaussian sphere.

[0326] The first inverse quantization is an inverse scalar quantization. The process of the decoding device performing the first inverse quantization on the encoded data of the first Gaussian sphere can be as follows: performing inverse scalar quantization on the encoded data of the first Gaussian sphere.

[0327] For example, the decoding device can obtain the bit depth of each feature in the first Gaussian sphere from the encoded information. For each feature, according to the bit depth of the feature, the corresponding number of bytes are read from the encoded data of the first Gaussian sphere to obtain the scalar quantization result of the feature. Then, based on the bit depth, numerical range (such as the maximum and minimum values ​​of the feature), scalar quantization result, and inverse scalar quantization formula, inverse scalar quantization is calculated, thereby enabling decoding to obtain the original values ​​of each feature.

[0328] The second dequantization is one of the following: inverse vector quantization, inverse product quantization, or a combination of inverse scalar quantization, inverse vector quantization, and inverse product quantization.

[0329] For example, the encoded data of the second Gaussian sphere includes encoded data of the first feature data and encoded data of the second feature data. Accordingly, the process of the decoding device performing a second inverse quantization on the encoded data of the second Gaussian sphere may include: performing a third inverse quantization on the encoded data of the first feature data and a fourth inverse quantization on the encoded data of the second feature data.

[0330] The second dequantization includes the third dequantization of the first feature data and the fourth dequantization of the second feature data.

[0331] The third inverse quantization can be inverse scalar quantization. Accordingly, the process of the decoding device performing the third inverse quantization on the encoded data of the first feature data can be: the decoding device performs inverse scalar quantization on the encoded data of the first feature data.

[0332] The fourth inverse quantization can include inverse vector quantization and inverse scalar quantization, or the fourth quantization can include product quantization and scalar quantization, etc. For example, the decoding device can perform fourth inverse quantization on the encoded data of the second feature data in the following two ways.

[0333] In the first inverse quantization method, the decoding device performs inverse vector quantization on the encoded data of the second feature data, and then performs inverse scalar quantization on the result of the inverse vector quantization.

[0334] In the decoding process based on inverse quantization method one, inverse scalar quantization is performed on the encoded data of one or more features of the second Gaussian sphere, excluding higher-order spherical coefficients (such as one or more features from the center point, scale information, spatial orientation information, opacity, and diffuse color). The higher-order spherical coefficients of the second Gaussian sphere are first inverse vector quantized, and then the result of the inverse vector quantization is inverse scalar quantization.

[0335] For example, for one or more features other than higher-order spherical coefficients, the bit depth of these features is obtained from the encoded information. For each feature, the scalar quantization result of the feature is obtained by reading the corresponding number of bytes from the encoded data of the first Gaussian sphere according to the bit depth of the feature. Inverse scalar quantization is then performed based on the bit depth, numerical range (e.g., the maximum and minimum values ​​of the feature), scalar quantization result, and inverse scalar quantization formula, thereby decoding the original values ​​of each feature. For higher-order spherical coefficients, the index data of the higher-order spherical coefficients is first read according to the index bit depth, and the read index data is converted into index values ​​(e.g., integers). Then, the corresponding code table is determined according to the code table size of the higher-order spherical coefficients, and the codeword corresponding to the index value is retrieved from the corresponding code table as the higher-order spherical coefficient feature. Finally, inverse scalar quantization is performed according to the bit depth, numerical range (e.g., the maximum and minimum values ​​of the feature), scalar quantization result, and inverse scalar quantization formula, thus completing the decoding of the higher-order spherical coefficients.

[0336] In the second inverse quantization method, the decoding device performs inverse product quantization on the encoded data of the second feature data, and then performs inverse scalar quantization on the result of the inverse product quantization.

[0337] In the decoding process based on inverse quantization method two, inverse scalar quantization is performed on the encoded data of one or more features of the second Gaussian sphere, excluding higher-order spherical coefficients (such as one or more features from the center point, scale information, spatial orientation information, opacity, and diffuse color). The higher-order spherical coefficients of the second Gaussian sphere are first subjected to inverse product quantization, and then the result of the inverse product quantization is subjected to inverse scalar quantization.

[0338] For example, the inverse scalar quantization of features other than higher-order spherical coefficients is the same as the inverse scalar quantization process shown in the decoding flow based on inverse quantization method one above, and will not be repeated here. For higher-order spherical coefficients, firstly, the index data of the higher-order spherical coefficients is read according to the index bit depth, and the read index data is converted into index values ​​(such as integers). Then, the corresponding code table is determined according to the code table size of the higher-order spherical coefficients, and the codeword corresponding to the index value is retrieved from the corresponding code table as a sub-vector feature. Next, the sub-vector features are concatenated to synthesize higher-order spherical coefficient features according to the method of dividing the feature vector into multiple groups of sub-vectors carried in the encoding information. Finally, inverse scalar quantization is calculated according to the bit depth, numerical range (such as the maximum and minimum values ​​of the feature), scalar quantization result, and inverse scalar quantization formula of the higher-order spherical coefficients, which completes the decoding of the higher-order spherical coefficients.

[0339] Furthermore, when the bitstream does not contain the second feature data, the order of the higher-order ball coefficients in the encoded information is 0. In this case, there is no need to decode the higher-order ball coefficient data; only the encoded data of the other feature data in the second Gaussian ball, excluding the higher-order ball coefficients, needs to be inverse-scalar quantized. The inverse-scalar quantization process is the same as that shown in the decoding process based on inverse quantization method one above, and will not be repeated here.

[0340] The contents of S805 to S806 in the embodiment of Figure 8 can correspond to the decoding process of S704 in the embodiment of Figure 7.

[0341] In the embodiment shown in Figure 8 above, N Gaussian sphere groups are obtained based on the degree of influence of each Gaussian sphere on the rendering effect, and different encoding methods are used for the Gaussian spheres in different groups. Furthermore, by carrying the indication information of each Gaussian sphere in the bitstream using flag bits or sequence numbers, the group to which each Gaussian sphere belongs can be determined based on the indication information during the decoding stage. Then, the decoding process can be executed according to the decoding method corresponding to the group to which each Gaussian sphere belongs, which can effectively improve the decoding efficiency.

[0342] Regarding the indication information involved in the embodiments of this application, in some embodiments, the indication information may use different character flag bits to represent different groups. An example flow of a flag-based encoding / decoding method is provided below based on Figure 9. Figure 9 is a schematic diagram of an example flow of a flag-based encoding / decoding method provided in an embodiment of this application. Referring to Figure 9, taking the division into a first Gaussian sphere group and a second Gaussian sphere group as an example, the example flow of the flag-based encoding method may include:

[0343] ① Grouping of Gaussian balls.

[0344] ② Encode the data by grouping to obtain the encoded data.

[0345] If the Gaussian sphere is in the first Gaussian sphere group: encode according to the first Gaussian sphere group, such as performing the first quantization on the Gaussian spheres in the first Gaussian sphere group and adding a flag bit: 0 to the Gaussian sphere.

[0346] If the Gaussian sphere is in the second Gaussian sphere group: encode according to the second Gaussian sphere group, such as performing second quantization on the Gaussian spheres in the second Gaussian sphere group, and adding a flag bit: 1 to the Gaussian sphere.

[0347] ③ Obtain grouping information.

[0348] ④ Encode into the bitstream. Encode the data, flags, and grouping information into the bitstream.

[0349] Optionally, the encoded bitstream data can also be stored and / or transmitted.

[0350] An example flow for a flag-based decoding method may include:

[0351] ⑥ Read the bitstream.

[0352] ⑦ Parse the bitstream and read the flag bits.

[0353] The group to which the Gaussian ball belongs is determined by judging the character in the flag.

[0354] If the flag is 0, the Gaussian sphere belongs to the first Gaussian sphere group.

[0355] If the flag is 1, then the Gaussian ball belongs to the second Gaussian ball group.

[0356] ⑧ Decode the data by group to obtain the decoded data.

[0357] Decode by the first Gaussian sphere group, such as performing the first inverse quantization on the encoded data of the Gaussian spheres within the first Gaussian sphere group.

[0358] Decoding is performed by grouping the Gaussian spheres into blocks, such as performing a second inverse quantization on the encoded data of the Gaussian spheres within the second Gaussian sphere group.

[0359] Regarding the indication information involved in the embodiments of this application, in some embodiments, the indication information may use different numerical sequence numbers to represent different groups. An example flow of a sequence number-based encoding / decoding method is provided below based on Figure 10. Figure 10 is a schematic diagram of an example flow of a sequence number-based encoding / decoding method provided in an embodiment of this application. Referring to Figure 10, taking the division into a first Gaussian sphere group and a second Gaussian sphere group as an example, the example flow of the sequence number-based encoding method may include:

[0360] ① Grouping of Gaussian balls.

[0361] ② Encode the data by grouping to obtain the encoded data.

[0362] If the Gaussian spheres are in the first Gaussian sphere group: encode according to the first Gaussian sphere group, such as performing the first quantization on the Gaussian spheres in the first Gaussian sphere group, and sorting the multiple Gaussian spheres in the first Gaussian sphere group to obtain the serial numbers of the multiple Gaussian spheres and the range of the first Gaussian sphere group (such as the serial number interval).

[0363] If the Gaussian spheres are grouped into the second Gaussian sphere group: encode according to the second Gaussian sphere group, such as performing a second quantization on the Gaussian spheres within the second Gaussian sphere group, and sorting the multiple Gaussian spheres within the second Gaussian sphere group to obtain the serial numbers of the multiple Gaussian spheres and the range of the second Gaussian sphere group (such as the serial number interval).

[0364] Among them, the Gaussian spheres in the same group are consecutively numbered, and the ranges of different groups do not overlap.

[0365] ③ Obtain grouping information.

[0366] ④ Encode into the bitstream. Encode the data, sequence number, range, and grouping information into the bitstream.

[0367] Optionally, the encoded bitstream data can also be stored and / or transmitted.

[0368] An example flow for a number-based decoding method may include:

[0369] ⑥ Read the bitstream.

[0370] ⑦ Parse the bitstream and read the sequence number.

[0371] By determining which group the sequence number belongs to, the group to which the Gaussian sphere belongs can be identified.

[0372] If the serial number (e.g., 5) belongs to the range of the first Gaussian sphere group (e.g., 1-10), then the Gaussian sphere belongs to the first Gaussian sphere group.

[0373] If the serial number (e.g., 15) belongs to the range of the second Gaussian sphere group (e.g., 11-20), then the Gaussian sphere belongs to the second Gaussian sphere group.

[0374] ⑧ Decode the data by group to obtain the decoded data.

[0375] Decode by the first Gaussian sphere group, such as performing the first inverse quantization on the encoded data of the Gaussian spheres within the first Gaussian sphere group.

[0376] Decoding is performed by grouping the Gaussian spheres into blocks, such as performing a second inverse quantization on the encoded data of the Gaussian spheres within the second Gaussian sphere group.

[0377] It can be observed that the difference between the process shown in Figure 9 and the process shown in Figure 10 lies in the method used to indicate the group to which the Gaussian ball belongs in the bitstream and the method used to identify the group to which the Gaussian ball belongs during decoding. Both flag bits and sequence numbers can achieve the effect of indicating the group to which the Gaussian ball belongs in the bitstream, thereby helping the decoding stage quickly determine the decoding method and improving decoding efficiency.

[0378] It should be noted that the above description is for the purpose of more clearly explaining the 3DGS model encoding method and 3DGS model decoding method described in the embodiments of this disclosure, and should not be construed as a limitation on the specific implementation of this application.

[0379] The above mainly describes the solutions provided by the embodiments of this application from the perspective of processing flow. Correspondingly, the embodiments of this application also provide an encoding device and a decoding device for a 3DGS model. The encoding device for the 3DGS model is used to implement the various encoding methods described above, and the decoding device for the 3DGS model is used to implement the various decoding methods described above. The encoding device and decoding device for the 3DGS model can be the encoding and decoding devices in the above method embodiments, or a device containing the above encoding and decoding devices, or a component that can be used for encoding and decoding. It is understood that, in order to achieve the above functions, the encoding device and decoding device for the 3DGS model include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0380] This application embodiment can divide the 3DGS model encoding device and 3DGS model decoding device into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be understood that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0381] For example, Figure 11 is a schematic diagram of the structure of a 3DGS model encoding device provided in an embodiment of this application. Referring to Figure 11, the 3DGS model encoding device includes an acquisition module 1101 and an encoding module 1102. Wherein:

[0382] The acquisition module 1101 is used to execute S701 shown in FIG7 or S801 shown in FIG8 above;

[0383] The encoding module 1102 is used to execute S702 shown in FIG7 or S802 to S803 shown in FIG8.

[0384] For example, Figure 12 is a schematic diagram of a 3DGS model decoding device provided in an embodiment of this application. Referring to Figure 12, the 3DGS model decoding device includes a receiving module 1201 and a decoding module 1202. Wherein:

[0385] The receiving module 1201 is used to execute S703 shown in FIG7 or S804 shown in FIG8.

[0386] The decoding module 1202 is used to execute S704 shown in FIG7 or S805 to S806 shown in FIG8.

[0387] For a detailed description of the above-mentioned optional methods, please refer to the foregoing method embodiments, which will not be repeated here. Furthermore, the explanations and descriptions of the beneficial effects of any of the 3DGS model encoding and decoding devices provided above can be found in the corresponding method embodiments above, and will not be repeated here.

[0388] As an example, referring to Figure 6, some or all of the functions implemented in the acquisition module 1101 and encoding module 1102 of the encoding device for the 3DGS model shown in Figure 11 can be implemented by the processor 601 in Figure 6 executing the computer execution instructions in the memory 602 in Figure 6.

[0389] In this embodiment, the encoding device for the 3DGS model is presented as an integrated unit divided into functional modules. Here, "module" can refer to a specific ASIC, circuitry, a processor and memory executing one or more software or firmware programs, integrated logic circuitry, and / or other devices that can provide the aforementioned functions. In a simplified embodiment, those skilled in the art will recognize that the encoding device for the 3DGS model can take the form of a server as shown in Figure 6.

[0390] For example, the processor 601 in the server shown in Figure 6 can call the computer execution instructions stored in the memory 602 to enable the server to execute the encoding method of the 3DGS model in the above method embodiment.

[0391] As an example, referring to Figure 5, some or all of the functions implemented in the receiving module 1201 and decoding module 1202 of the decoding device for the 3DGS model shown in Figure 12 can be implemented by the processor 510 in Figure 5 executing the computer execution instructions in the internal memory 521 in Figure 5.

[0392] In this embodiment, the decoding device for the 3DGS model is presented as an integrated unit divided into functional modules. Here, "module" can refer to a specific ASIC, circuit, processor and memory executing one or more software or firmware programs, integrated logic circuits, and / or other devices that can provide the aforementioned functions. In a simplified embodiment, those skilled in the art will recognize that the encoding device for the 3DGS model can take the form of the terminal shown in Figure 5.

[0393] For example, the processor 510 in the terminal shown in Figure 5 can call the computer execution instructions stored in the internal memory 521 to enable the terminal to execute the 3DGS model decoding method in the above method embodiment.

[0394] Since the encoding device for the 3DGS model provided in this application embodiment can execute the above-described encoding method for the 3DGS model, and the decoding device for the 3DGS model can execute the above-described decoding method, the technical effects that can be obtained can be referred to the above-described method embodiment, and will not be repeated here.

[0395] It should be understood that one or more of the above modules or units can be implemented by software, hardware, or a combination of both. When any of the above modules or units are implemented by software, the software exists as computer program instructions and is stored in memory. The processor can be used to execute the program instructions and implement the above method flow. The processor can be built into a SoC (System-on-a-Chip) or ASIC, or it can be a separate semiconductor chip. In addition to the core that executes software instructions for computation or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (Programmable Logic Devices), or logic circuits that implement dedicated logic operations.

[0396] When the above modules or units are implemented in hardware, the hardware can be any one or any combination of a microprocessor, digital signal processing (DSP) chip, microcontroller unit (MCU), artificial intelligence processor, ASIC, SoC, FPGA, PLD, application-specific digital circuit, hardware accelerator, or non-integrated discrete device, which can run the necessary software or perform the above method flow independently of software.

[0397] This application also provides an encoding device (e.g., the encoding device may be a chip or a chip system), which includes a processor for implementing the method executed by the encoding device in any of the above method embodiments. In one possible design, the encoding device further includes a memory. The memory is used to store necessary program instructions and data, and the processor can call the program code stored in the memory to instruct the encoding device to execute the encoding method in any of the above method embodiments. Of course, the memory may not be included in the encoding device. When the encoding device is a chip system, it may be composed of chips or may include chips and other discrete devices; this application does not specifically limit this.

[0398] This application also provides an encoding device, which includes a processing circuit for implementing the method executed by the encoding device in any of the above method embodiments.

[0399] This application also provides a decoding device (e.g., the decoding device may be a chip or a chip system), which includes a processor for implementing the method executed by the decoding device in any of the above method embodiments. In one possible design, the decoding device further includes a memory. The memory is used to store necessary program instructions and data, and the processor can call the program code stored in the memory to instruct the decoding device to execute the decoding method in any of the above method embodiments. Of course, the memory may not be included in the decoding device. When the decoding device is a chip system, it may be composed of chips or may include chips and other discrete devices; this application does not specifically limit this.

[0400] This application also provides a decoding device, which includes a processing circuit for implementing the method executed by the decoding device in any of the above method embodiments.

[0401] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed on an encoding device, the encoding device performs the method executed by any of the 3DGS model encoding devices provided above. Or, when the computer-executable instructions are executed on a decoding device, the decoding device performs the method executed by any of the 3DGS model decoding devices provided above.

[0402] For explanations of the relevant content and descriptions of the beneficial effects in any of the computer-readable storage media provided above, please refer to the corresponding embodiments described above, which will not be repeated here.

[0403] This application also provides a chip. This chip integrates a control circuit and one or more ports for implementing the functions of the encoding or decoding device for the 3DGS model described above. Optionally, the functions supported by this chip can be referred to above, and will not be repeated here. Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0404] This application also provides a computer program product containing computer-executable instructions. When the computer-executable instructions are executed on an encoding device, the encoding device performs any of the methods described in the above embodiments; when the computer-executable instructions are executed on a decoding device, the decoding device performs any of the methods described in the above embodiments. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on the encoding or decoding device, all or part of the flow or function according to the embodiments of this application is generated. The encoding or decoding device may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0405] Computer-executable instructions can be stored in or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, electronic device, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium accessible to an electronic device or a data storage device that includes one or more electronic devices, data centers, etc., that can be integrated with such media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0406] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of this application, such as but not limited to the memory, computer-readable storage medium and communication chip, are all non-transitory.

[0407] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product.

[0408] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0409] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method for encoding a 3DGS model, characterized in that, The method includes: Obtain a 3DGS model, the 3DGS model including multiple Gaussian spheres, the multiple Gaussian spheres including a first Gaussian sphere and a second Gaussian sphere; The plurality of Gaussian spheres are encoded to obtain the encoded data of the 3DGS model. The encoded data is then encoded into a bitstream. The bitstream further includes first indication information and second indication information. The first indication information is used to indicate the method of encoding or decoding the first Gaussian sphere, and the second indication information is used to indicate the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different.

2. The method according to claim 1, characterized in that, The method further includes: The multiple Gaussian spheres are grouped to obtain N Gaussian sphere groups, where N is a positive integer greater than 1.

3. The method according to claim 2, characterized in that, The encoding methods corresponding to the N Gaussian sphere groups are different.

4. The method according to claim 2 or 3, characterized in that, The first Gaussian sphere and the second Gaussian sphere belong to different groups among the N groups of Gaussian spheres.

5. The method according to any one of claims 2-4, characterized in that, The bitstream also includes a range for each of the N Gaussian sphere groups, the range being used to determine the group to which the first Gaussian sphere belongs based on the first indication information, and to determine the group to which the second Gaussian sphere belongs based on the second indication information.

6. The method according to claim 5, characterized in that, The first indication information is used to indicate the sequence number of the first Gaussian sphere in its group, and the second indication information is used to indicate the sequence number of the second Gaussian sphere in its group, wherein Gaussian spheres belonging to the same group among the plurality of Gaussian spheres have consecutive sequence numbers in the corresponding same group; the method further includes: Sort the Gaussian spheres in each of the N Gaussian sphere groups to obtain the sequence number of each Gaussian sphere in its respective group.

7. The method according to any one of claims 2-4, characterized in that, The first indication information is used to indicate the group to which the first Gaussian sphere belongs, and the second indication information is used to indicate the group to which the second Gaussian sphere belongs.

8. The method according to any one of claims 2-7, characterized in that, The bitstream also includes grouping information for the N Gaussian sphere groups.

9. The method according to claim 8, characterized in that, The grouping information includes the number of groups, the number of Gaussian balls in each group, and the encoding information of each group.

10. The method according to any one of claims 1-9, characterized in that, The encoding of the plurality of Gaussian spheres includes: The feature data of the first Gaussian sphere are subjected to a first quantization. Perform a second quantization on the feature data of the second Gaussian sphere; The compression ratio of the first quantization is lower than that of the second quantization.

11. The method according to claim 10, characterized in that, The feature data of the second Gaussian sphere includes first feature data and second feature data. The second quantization of the feature data of the second Gaussian sphere includes: The first feature data is subjected to a third quantization, and the second feature data is subjected to a fourth quantization.

12. The method according to claim 11, characterized in that, The fourth quantization of the second feature data includes: The second feature data is scalar quantized, and the scalar quantization result is then vector quantized.

13. The method according to claim 11, characterized in that, The fourth quantization of the second feature data includes: The second feature data is scalar quantized, and the scalar quantization result is then multiplied quantized.

14. The method according to claim 10, characterized in that, The feature data of the second Gaussian sphere includes first feature data and second feature data, but the bitstream does not contain the second feature data.

15. A method for decoding a 3DGS model, characterized in that, The method includes: The received bitstream includes encoded data of a 3DGS model, which includes multiple Gaussian spheres, including a first Gaussian sphere and a second Gaussian sphere. The bitstream also includes first indication information and second indication information. The first indication information is used to indicate the method of encoding or decoding the first Gaussian sphere, and the second indication information is used to indicate the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different. The encoded data of the first Gaussian sphere is decoded according to the first instruction information to obtain the first Gaussian sphere, and the encoded data of the second Gaussian sphere is decoded according to the second instruction information to obtain the second Gaussian sphere.

16. The method according to claim 15, characterized in that, The multiple Gaussian spheres belong to N Gaussian sphere groups, and the decoding methods corresponding to the N Gaussian sphere groups are different, where N is a positive integer greater than 1.

17. The method according to claim 16, characterized in that, The first Gaussian sphere and the second Gaussian sphere belong to different groups among the N groups of Gaussian spheres.

18. The method according to claim 16 or 17, characterized in that, The bitstream also includes the range of each Gaussian sphere group in the N Gaussian sphere groups, and the method further includes: Based on the first indication information and the range of each Gaussian sphere group in the N Gaussian sphere groups, determine the group to which the first Gaussian sphere belongs; Based on the second indication information and the range of each Gaussian sphere group in the N Gaussian sphere groups, the group to which the second Gaussian sphere belongs is determined.

19. The method according to claim 18, characterized in that, The first indication information is used to indicate the sequence number of the first Gaussian sphere in its group, and the second indication information is used to indicate the sequence number of the second Gaussian sphere in its group, wherein Gaussian spheres belonging to the same group among the plurality of Gaussian spheres have consecutive sequence numbers in the corresponding same group; the method further includes: The group to which the first Gaussian sphere belongs is determined based on the sequence number of the first Gaussian sphere in its group and the range of each Gaussian sphere group in the N Gaussian sphere groups; The group to which the second Gaussian sphere belongs is determined based on the sequence number of the second Gaussian sphere in its group and the range of each of the N Gaussian sphere groups.

20. The method according to claim 16 or 17, characterized in that, The method further includes: Based on the first indication information, determine the group to which the first Gaussian sphere belongs; Based on the second instruction information, determine the group to which the second Gaussian ball belongs.

21. The method according to any one of claims 16-20, characterized in that, The bitstream also includes grouping information for the N Gaussian sphere groups; the method further includes: Based on the grouping information of the N Gaussian spheres, the plurality of Gaussian spheres are decoded to obtain the plurality of Gaussian spheres.

22. The method according to claim 21, characterized in that, The grouping information includes the number of groups, the number of Gaussian balls in each group, and the encoding information of each group.

23. The method according to any one of claims 15-22, characterized in that, The step of decoding the encoded data of the first Gaussian sphere according to the first indication information to obtain the first Gaussian sphere, and decoding the encoded data of the second Gaussian sphere according to the second indication information to obtain the second Gaussian sphere, includes: Based on the first instruction information, the encoded data of the first Gaussian sphere is subjected to a first inverse quantization to obtain the first Gaussian sphere; According to the second instruction information, the encoded data of the second Gaussian sphere is subjected to a second inverse quantization to obtain the second Gaussian sphere.

24. The method according to claim 23, characterized in that, The encoded data of the second Gaussian sphere includes encoded data of the first feature data and encoded data of the second feature data. The second inverse quantization of the encoded data of the second Gaussian sphere includes: The encoded data of the first feature data is subjected to a third inverse quantization, and the encoded data of the second feature data is subjected to a fourth inverse quantization.

25. The method according to claim 24, characterized in that, The fourth inverse quantization of the encoded data of the second feature data includes: The encoded data of the second feature data is subjected to inverse vector quantization, and the result of inverse vector quantization is subjected to inverse scalar quantization.

26. The method according to claim 24, characterized in that, The fourth inverse quantization of the encoded data of the second feature data includes: The encoded data of the second feature data is subjected to inverse product quantization, and the result of inverse product quantization is subjected to inverse scalar quantization.

27. An encoding device for a 3DGS model, characterized in that, The device includes: An acquisition module is used to acquire a 3DGS model, the 3DGS model including multiple Gaussian spheres, the multiple Gaussian spheres including a first Gaussian sphere and a second Gaussian sphere; An encoding module is used to encode the plurality of Gaussian spheres to obtain the encoded data of the 3DGS model, and to encode the encoded data into a bitstream. The bitstream further includes first indication information and second indication information. The first indication information is used to indicate the method of encoding or decoding the first Gaussian sphere, and the second indication information is used to indicate the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different.

28. A decoding device for a 3DGS model, characterized in that, The device includes: A receiving module is used to receive a bitstream, the bitstream including encoded data of a 3DGS model, the 3DGS model including multiple Gaussian spheres, the multiple Gaussian spheres including a first Gaussian sphere and a second Gaussian sphere, the bitstream also including first indication information and second indication information, the first indication information being used to indicate the method of encoding or decoding the first Gaussian sphere, the second indication information being used to indicate the method of encoding or decoding the second Gaussian sphere, the first indication information and the second indication information being different; The decoding module is used to decode the encoded data of the first Gaussian sphere according to the first indication information to obtain the first Gaussian sphere, and to decode the encoded data of the second Gaussian sphere according to the second indication information to obtain the second Gaussian sphere.

29. An encoding device, characterized in that, The method includes a memory and a processor, the memory and the processor being connected; the memory is used to store computer-executed instructions; the processor is used to invoke the computer-executed instructions to perform the method as described in any one of claims 1-14.

30. A decoding device, characterized in that, The method includes a memory and a processor, the memory and the processor being connected; the memory is used to store computer-executable instructions; the processor is used to invoke the computer-executable instructions to perform the method as described in any one of claims 15-26.

31. A chip, characterized in that, The device includes a processor and an interface circuit, the interface circuit being configured to receive computer execution instructions; the processor being configured to execute the computer execution instructions to perform the method as described in any one of claims 1-14 or 15-26.

32. A computer-readable storage medium, characterized in that, The method includes computer execution instructions that, when executed on an encoding device, cause the encoding device to perform the method as described in any one of claims 1-14; and when executed on a decoding device, cause the decoding device to perform the method as described in any one of claims 15-26.

33. A computer-readable storage medium, characterized in that, The system includes a bitstream, which includes encoded data of a 3DGS model. The 3DGS model includes multiple Gaussian spheres, including a first Gaussian sphere and a second Gaussian sphere. The bitstream also includes first indication information and second indication information. The first indication information is used to indicate the method of encoding or decoding the first Gaussian sphere, and the second indication information is used to indicate the method of encoding or decoding the second Gaussian sphere. The first indication information and the second indication information are different.

34. A computer program product, characterized in that, The method includes computer execution instructions that, when executed on an encoding device, cause the encoding device to perform the method as described in any one of claims 1-14; and when executed on a decoding device, cause the decoding device to perform the method as described in any one of claims 15-26.

Images