A three-dimensional gaussian representation error concealment method, device and medium against packet loss

Abstract

The present application relates to the technical field of data communication, and particularly relates to a three-dimensional Gaussian representation anti-packet loss error concealment method, device and medium, the method comprising: completing three-dimensional Gaussian anchor point attribute extraction and spatial uniform layering random grouping, separating anchor point position information and core attributes and completing global spatial dimension scattering grouping; performing anchor point level atomization packaging and channel transmission, binding and packaging anchor point strong correlation attributes and then sending them into unreliable channel transmission; completing packet loss positioning at the receiving end, generating a missing anchor point attribute recovery baseline by using a local interpolation method; based on a priori perception graph neural network, completing high-frequency residual prediction, combining confidence determination to realize attribute fusion repair and safe rollback, and completing three-dimensional Gaussian scene packet loss error concealment reconstruction. The present application aggregates strong correlation attributes of the same anchor point by using anchor point level atomization packaging, so that the packet loss failure mode is constrained to complete anchor point loss, and strong artifacts caused by attribute mismatch are avoided.

Description

Technical Field

[0001] This invention relates to the field of data communication technology, and in particular to a method, device and medium for error concealment against packet loss using three-dimensional Gaussian representation. Background Technology

[0002] In recent years, 3D Gaussian Splatting (3DGS) has become the mainstream representation paradigm in fields such as 3D scene reconstruction, virtual / augmented reality, remote immersive communication, and real-time 3D streaming media due to its high-fidelity rendering and efficient drawing capabilities. In practical applications, high-precision 3DGS models typically contain massive Gaussian point clouds and multi-dimensional attribute parameters such as features, scale, and offset, resulting in huge data volumes and severe bandwidth and storage overhead during network transmission and edge distribution. To achieve efficient transmission, existing technologies generally employ high-ratio compression schemes. Among them, advanced compression methods such as HAC++ achieve excellent compression performance through attribute-level decoupling, clustering, and entropy coding, maintaining high-quality scene reconstruction results under offline storage and ideal lossless channels.

[0003] Current mainstream 3DGS compression and transmission schemes generally adopt an attribute-level independent packetization method for organizing bitstreams. This involves extracting and independently encoding the same type of attributes (such as feature, scale, and offset) of all anchor points to form independent bitstreams for transmission. While this method can maximize the elimination of redundancy and improve source coding efficiency under ideal channel conditions, it has significant drawbacks in real-world network environments. Mobile and wireless networks commonly experience random and burst packet loss. If some attribute bitstreams are lost, it directly leads to incomplete or mismatched anchor point parameters, resulting in fragmented anchor points. This, in turn, causes severe structural distortions and strong visual artifacts during rendering. Compared to complete but missing anchor points, fragmented anchor points cause more severe damage to reconstruction quality and significantly interfere with subsequent error concealment and scene restoration.

[0004] Meanwhile, existing 3DGS data is typically organized and serialized in a spatially adjacent order to improve compression efficiency by leveraging spatial correlation. However, in scenarios of sudden packet loss, the loss of consecutive data packets can cause large areas of highly clustered anchor points to simultaneously fail, creating large-scale holes in the 3D scene. Because the prior local geometry and texture information around the hole area is synchronously missing, traditional error-hiding methods such as interpolation and neighborhood fitting are insufficient to effectively recover the missing information. This results in noticeable breaks and distortion in the rendered image, failing to meet the stability and fidelity requirements of immersive interaction and real-time transmission.

[0005] In summary, existing 3D Gaussian representations lack robustness in unreliable network transmissions, easily exhibiting broken anchor artifacts and large, unrepairable holes when faced with random or sudden packet loss. They also lack specific packet loss mitigation mechanisms and error concealment methods tailored to the characteristics of 3DGS. To achieve stable transmission and high-quality rendering of 3DGS over real-world networks, it is urgently necessary to propose a targeted, robust error concealment method adapted to the structural characteristics of 3D Gaussians. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a method, device, and medium for error hiding in three-dimensional Gaussian representation to resist packet loss. This enables reliable recovery of scene geometry and texture details even in the event of packet loss, thereby improving the integrity and visual consistency of 3D rendering.

[0007] In a first aspect, the present invention provides a three-dimensional Gaussian representation error concealment method for resisting packet loss, comprising the following steps: S1: Complete the three-dimensional Gaussian anchor point attribute extraction and spatial uniform hierarchical random grouping, separate anchor point position information from core attributes and complete global spatial dimension shuffling and grouping; S2: Perform anchor-level atomic packaging and channel transmission for anti-spoofing, bind and encapsulate the anchor-strongly related attributes and send them into the unreliable channel for transmission. S3: Complete packet loss location at the receiving end and use local interpolation to generate missing anchor point attributes to restore the baseline; S4: Based on the prior perceptual graph neural network, high-frequency residual prediction is completed. Combined with confidence judgment, attribute fusion repair and safe rollback are realized to complete the hidden reconstruction of packet loss error in 3D Gaussian scene.

[0008] Optionally, step S1 specifically includes: S11: Independently read the three-dimensional Gaussian anchor point position information, and separately parse and extract the core attribute tuples of each anchor point without position coordinates. The core attribute tuples include high-dimensional feature attributes, scale attributes, and relative quantization grid offset attributes. S12: Based on the 3D scene bounding box, perform mesh spatial layering on the global 3D space, and divide all Gaussian anchor points into corresponding spatial sub-regions; S13: A hierarchical random sampling method without replacement is used to complete the spatial uniform random group mapping, so that the anchor points in the same transmission group are uniformly and discretely distributed in the three-dimensional global space. S14: By combining Morton order dimensionality reduction sorting with pseudo-random hash distribution, the uniform grouping and arrangement of three-dimensional anchor points in the global space can be achieved with low computing power.

[0009] Optionally, in S1, the Morton order is mapped to the pseudo-random hash shuffle as follows: After quantizing the three-dimensional coordinates of the anchor points, the Morton code is solved to complete the one-dimensional sequence sorting. The sorted sequence is divided into continuous batches with fixed capacity. A pseudo-random hash function with a fixed seed is used to perform indexing operations on the anchor points within the batch. Based on the hash modulo result, the anchor points are assigned to different transmission packets.

[0010] Optionally, step S2 specifically includes: S21: Based on the spatial uniform grouping results, bind and encapsulate the core attributes of all Gaussian anchor points within the same group into an indivisible atomic data packet, and synchronously configure the anchor point activation mask within the data packet; S22: Establish an unreliable network channel transmission model, set data packet reception status variables, and distinguish between two states: successful data packet reception and transmission loss. S23: Based on the atomic packing rules, the packet loss failure mode transformation is completed, so that all core attributes of a single Gaussian anchor point share the same data packet receiving state, realizing the all-or-nothing failure mechanism in the packet loss scenario.

[0011] Optionally, in S2, the all-or-nothing failure mechanism is as follows: If the data packet is received normally, all core attributes of the corresponding anchor point are complete and usable. If the data packet is lost, all anchor points within the data packet are judged to be completely missing, forming a clean missing void without attribute disorder, thus preventing rendering artifacts caused by the loss of a single attribute.

[0012] Optionally, step S3 specifically includes: S31: The decoding end completes packet loss detection through the data packet sequence number, and combines the independently stored global anchor point location information to divide the set of missing anchor points into the set of normal available anchor points; S32: Based on the three-dimensional spatial coordinates of the missing anchor point, filter nearby available anchor points based on the three-dimensional Euclidean distance to construct the local spatial neighborhood of the missing anchor point; S33: Based on the HAC hash grid structure, hash semantic features are extracted by querying the spatial coordinates of anchor points to obtain implicit semantic priors of local region geometry and texture; S34: Construct bilateral interpolation weights by integrating spatial distance constraints and hash semantic difference constraints, and use k-nearest neighbor interpolation to solve for the core attributes of missing anchor points to generate low-frequency recovery baseline data.

[0013] Optionally, the bilateral interpolation weights in S34 include spatial weights and semantic weights. The spatial weights are negatively correlated with the physical spatial distance of the anchor point, and the semantic weights are negatively correlated with the spatial distance of the hash feature. After normalization, the two types of weights are jointly used to complete the weighted interpolation calculation of the anchor point attributes.

[0014] Optionally, step S4 specifically includes: S41: Construct a local graph topology with missing anchor points as the central node and available anchor points in the neighborhood as ordinary nodes. Piece together spatial location, core attributes, and hash prior features to complete the graph node feature initialization. Use interpolation baselines to fill in the missing nodes for feature placement. S42: Introduce a single-layer EdgeConv network to extract edge features between nodes and aggregate neighborhood feature information to predict the high-frequency residual components corresponding to the core attributes of missing anchor points. S43: Calculate the confidence score of local neighborhood feature consistency, complete the weighted fusion of interpolation baseline and high-frequency residual to obtain the final repair attribute, and configure the confidence threshold to achieve safe rollback.

[0015] Optionally, in S4, the safe rollback mechanism is as follows: When the confidence level of the residual prediction is lower than the confidence threshold, the high-frequency residual components are cleared, and the bilateral interpolation baseline is directly used as the missing anchor point to finally repair the attribute.

[0016] In a second aspect, the present invention also provides a three-dimensional Gaussian representation error-hiding device for resisting packet loss, used to implement the three-dimensional Gaussian representation error-hiding method for resisting packet loss as described above. The error-hiding device includes a data extraction and grouping module, an atomized packet transmission module, a receiver baseline repair module, and a graph neural network residual repair module, wherein: The data extraction and grouping module is used to perform anchor point attribute extraction and spatial uniform random grouping processes. The atomic packet transmission module is used to complete the atomic encapsulation of anchor point attributes, unreliable channel transmission, and packet loss failure mode conversion. The receiver baseline repair module is used to realize packet loss identification, neighborhood construction and spatial semantic bilateral weighted interpolation baseline solution; The graph neural network residual repair module is used to construct a local graph structure, predict high-frequency attribute residuals, and execute confidence fusion and safe rollback repair logic.

[0017] In a third aspect, the present invention also provides a computer-readable storage medium storing a computer-executable program, which, when invoked by a processor, implements the error-hiding method for resisting packet loss using three-dimensional Gaussian representation as described above.

[0018] The application of the technical solution of the present invention has at least the following beneficial effects: (1) This invention provides a method, device, and medium for error concealment in 3D Gaussian representation to resist packet loss. It eliminates broken anchor point artifacts from the transmission mechanism and significantly improves the robustness against packet loss. The method of this invention adopts anchor point-level atomic packaging to aggregate the strongly correlated attributes of the same anchor point into the same data packet for joint transmission, so that the packet loss failure mode is strictly constrained to the loss of complete anchor points, thus avoiding strong artifacts caused by attribute mismatch from the root. Compared with existing 3D Gaussian representation (3DGS) compression and transmission technologies (such as HAC++), the method of this invention simplifies the difficult problem of artifact suppression into a controllable point cloud missing completion problem, which greatly improves the transmission robustness and rendering stability of 3DGS under unreliable networks.

[0019] (2) The method of the present invention effectively alleviates the large-area voids caused by sudden packet loss and improves recoverability. Existing technologies organize data in spatially adjacent order, which will generate continuous and concentrated large-area blind spots when sudden packet loss occurs, making error hiding difficult to implement. The method of the present invention uses a hierarchical random spatial uniform grouping strategy to make the anchor points contained in each data packet approximately uniformly distributed in the global space; when continuous packet loss occurs, the lost anchor points are spatially scattered, forming sparse and dispersed micro-holes, preserving sufficient local neighborhood redundancy information, significantly improving the feasibility of error hiding and recovery quality at the decoding end.

[0020] (3) The method of this invention achieves fast error hiding with low overhead and high robustness, meeting the requirements of real-time rendering. By adopting a k-NN interpolation module based on spatial-semantic bilateral weighting at the decoding end, the method of this invention can quickly and robustly recover missing anchor points with low computational overhead. This invention also introduces HAC hash grid features as semantic priors to effectively suppress misinterpolation and excessive blurring across object edges.

[0021] (4) This invention supports high-fidelity detail reconstruction, achieving both steady-state and high-precision results. To address the problem of high-frequency details being easily lost during interpolation recovery, this invention provides a priori-aware multi-level error-hiding optimization pipeline, integrates hash grid / MLP priors with a single-layer EdgeConv graph neural network (GNN) to achieve residual prediction, and combines confidence fusion and a safe rollback mechanism to accurately restore high-fidelity details such as edge sharpness and texture structure without rendering crashes or system failures. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the error hiding method in a preferred embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1:

[0026] like Figure 1 As shown, this embodiment provides a three-dimensional Gaussian representation error concealment method to resist packet loss, including the following steps (S1 to S4): S1: Complete the extraction of 3D Gaussian anchor point attributes and spatial uniform hierarchical random grouping, separate anchor point position information from core attributes, and complete the global spatial dimension shuffling and grouping.

[0027] In this embodiment, the process of extracting the three-dimensional Gaussian anchor point attributes and spatially uniformly layered random grouping is as follows: S11: Independently read the three-dimensional Gaussian anchor point position information, and separately parse and extract the core attribute tuples of each anchor point without position coordinates. The core attribute tuples include high-dimensional feature attributes, scale attributes, and relative quantization grid offset attributes.

[0028] Specifically, the position information of global anchor points in 3D space is stored separately from the texture / geometric attributes of the anchor points. The independent global position file is read to obtain the location information of the anchor points in the scene. The set of basic position coordinates of a Gaussian anchor point in three-dimensional space, denoted as ,in Indicates the first The global base coordinates of each anchor point. This independent set of global coordinates will serve as the physical basis for subsequent spatial layering and uniform grouping. Simultaneously, for existing efficient compression schemes such as HAC++, the strongly correlated core attributes that need to be atomically combined and packaged for each anchor point are extracted separately.

[0029] Since the location information is stored independently, the first [position] is defined here. Anchor points The core attribute tuple contains only the following strongly correlated items: ; in, Indicates anchor point High-dimensional feature properties (e.g., spherical harmonic coefficients); Indicates anchor point The scale property (which controls the shape and size of the Gaussian ellipsoid); Indicates anchor point The offset property relative to the quantized grid.

[0030] This embodiment ensures the subsequent core attribute elements by separating the position coordinates. It consists only of strongly bound variables that are highly susceptible to fragmentation artifacts, thus providing a pure data source for precise atomic packaging.

[0031] S12: Based on the 3D scene bounding box, perform mesh spatial layering on the global 3D space, and divide all Gaussian anchor points into corresponding spatial sub-regions.

[0032] Specifically, to achieve uniform grouping throughout the entire global space, the entire continuous 3D space is first uniformly divided based on the scene's 3D bounding box. A set of non-overlapping local spatial sub-regions (or voxel meshes) is denoted as a spatial layer set. .

[0033] Traverse the global anchor set Based on the position coordinates of each anchor point This is assigned to the corresponding spatial sub-region. The definition falls within the... Sub-regions The subset of anchor points within is The following equation is satisfied: And for any .

[0034] S13: A hierarchical random sampling method without replacement is used to complete the spatial uniform random group mapping, so that the anchor points within the same transmission group are uniformly and discretely distributed in the three-dimensional global space.

[0035] The setting at the encoding end will assign all anchor points to In each independent transmission packet (i.e., the subsequent atomic packet unit), the set of packet indices is denoted as... To make any group The anchor points within the region exhibit an approximately uniform distribution in the global three-dimensional space. This embodiment employs a hierarchical random sampling strategy without replacement for mapping. For each spatial sub-region... Calculate what it should pay to the first Groups Number of anchor points contributed To ensure uniformity, this number is approximately equal to the expected value of the total number of points in the sub-region evenly distributed among each group. ; Subsequently, in each sub-region Randomly selected Anchor points and assign them to groups. Therefore, the first Anchor point index set of the final transmitted packets The mathematical expression is: ; in, This represents a random sampling function without replacement.

[0036] S14: By combining Morton order dimensionality reduction sorting with pseudo-random hash distribution, the uniform grouping and arrangement of three-dimensional anchor points in the global space can be achieved with low computing power.

[0037] In this embodiment, the Morton order and pseudo-random hash shuffling mapping are as follows: After quantizing the three-dimensional coordinates of the anchor points, the Morton code is solved to complete the one-dimensional sequence sorting. The sorted sequence is divided into continuous batches with fixed capacity. A pseudo-random hash function with a fixed seed is used to perform indexing operations on the anchor points within the batch. Based on the hash modulo result, the anchor points are assigned to different transmission packets.

[0038] Specifically, the process of breaking down the mapping includes: One-dimensional reduction of spatial locality (Morton ordering): First, the three-dimensional base coordinates of all extracted anchor points are quantized, and the Morton code (Morton Order) corresponding to each anchor point is calculated. The Morton code is then used to sort the global set of anchor points, so that highly clustered and adjacent anchor points in three-dimensional physical space remain continuously adjacent in the sorted one-dimensional sequence.

[0039] Step-wise processing divides the sorted global one-dimensional anchor sequence into multiple consecutive batches (Steps) of fixed capacity. Due to the Morton sorting, the sequence segments contained in the same Step naturally correspond to one or more highly clustered local spatial blocks in physical three-dimensional space.

[0040] Pseudo-random hash interleaving: This setting configures the system to divide data in parallel into... An independent transmission packet or channel (e.g., set) In each Step's processing loop, instead of traditional sequential packing, a pseudo-random hash function (such as the SplitMix64 algorithm) carrying a set seed is introduced. The system uses this hash function to perform a high-dimensional hash operation on the sequence index of each anchor point within the Step, and pairs the hash values... The modulo operation is performed to accurately map the anchor point and force its distribution to the corresponding transport packet (Lane).

[0041] S2: Performs anchor-level atomic packaging and channel transmission for anti-spoofing purposes, binds and encapsulates anchor-related attributes and sends them to unreliable channel transmission, and unifies packet loss failure logic.

[0042] In this embodiment, anchor-level atomized packetization and channel transmission specifically include: S21: Based on the spatial uniform grouping results, the core attributes of all Gaussian anchor points within the same group are bound and encapsulated into an indivisible atomic data packet. The anchor point activation existence mask is synchronously configured within the data packet. The implementation process is as follows: Based on uniformly grouped index sets We begin to physically package and encapsulate the Gaussian anchor point properties.

[0043] For the Transmission packets All anchor points inside Core attributes (features) ,scale Offset They are bound together as an indivisible data tuple and stored in the same network transport packet (pak).

[0044] No. Atomized transmission packet The mathematical expression for it is defined as: ; in, Indicates the first One generated atomic data packet; For the first The global index set of anchor points within each package; , , They represent the first The high-dimensional features, scale, and spatial offset corresponding to each anchor point; This is the existence mask for the anchor point in the current coding block, used to identify whether the anchor point is active during entropy coding or variable-length coding.

[0045] S22: Establish an unreliable network channel transmission model, set data packet reception state variables, and distinguish between two states: successful data packet reception and transmission loss. The implementation process is as follows: The encapsulated atomic data packet sequence Send to an unreliable physical transport channel (such as a UDP-based wireless communication or streaming media network).

[0046] Define channel state variables Indicates the first The reception status of each packet. When random or sudden packet loss occurs in the network, if the packet successfully arrives at the decoding end, then... If the packet is lost during transmission, then Therefore, the actual set of data packets received by the decoding end. for: .

[0047] S23: Based on the atomic packing rules, the packet loss failure mode transformation is completed, so that all core attributes of a single Gaussian anchor point share the same data packet receiving state, realizing the all-or-nothing failure mechanism in the packet loss scenario.

[0048] Specifically, the all-or-nothing failure mechanism is as follows: If the data packet is received normally, all core attributes of the corresponding anchor point are complete and usable. If the data packet is lost, all anchor points within the data packet are judged to be completely missing, forming a clean missing void without attribute disorder, thus preventing rendering artifacts caused by the loss of a single attribute.

[0049] Under this channel model, due to the atomic packetization mechanism adopted in step S21, the packet loss degradation mode is forced to transform into All-or-Nothing. Specifically, mathematically, for any anchor point... Its characteristics ,scale Offset The receiving state shares the same channel state variable. (in ).

[0050] State 1 (All): If Then the anchor point All attributes are fully preserved, and anchor points can be rendered perfectly.

[0051] State 2 (None): If If the packet is lost, all the anchors it contains are considered to be completely missing (i.e., a complete "clean hole").

[0052] S3: Complete packet loss location at the receiving end and use local interpolation to generate missing anchor point attributes to restore the baseline.

[0053] In this embodiment, step S3 specifically includes: S31: The decoding end performs packet loss detection using the data packet sequence number, and combines this with the independently stored global anchor point location information to divide the set of missing anchor points into a set of normally usable anchor points. The implementation process is as follows: After receiving the bitstream transmitted over the network, the decoder detects lost atomic data packets based on the sequence number of the data packets. Due to the global basic location set Already acquired independently, based on channel status. Locate the missing anchor points and define the set of missing anchor point indices to be repaired. The set of available anchor indexes successfully received Because the aforementioned spatial uniform grouping strategy was adopted, the set The missing anchor points are uniformly dispersed in three-dimensional space.

[0054] S32: Using the three-dimensional spatial coordinates of the missing anchor point as a reference, and filtering nearby available anchor points based on the three-dimensional Euclidean distance, a local spatial neighborhood of the missing anchor point is constructed. The implementation process is as follows: For the set to be repaired Each missing anchor point Its basic coordinates in three-dimensional space It is known that, within the set of available anchor points In the middle, distance is found based on three-dimensional Euclidean distance. Recent A number of valid anchor points are used to construct the local neighborhood of the missing anchor point. The expression is as follows: .

[0055] S33: Based on the HAC hash grid structure, hash semantic features are extracted by querying anchor point spatial coordinates to obtain implicit semantic priors of local region geometry and texture. The implementation process is as follows: For missing anchor points With known anchor points in the neighborhood Using its position coordinates and Query the underlying hash grid and extract the corresponding hash features, denoted as follows: and This hash semantic feature implies implicit semantic priors about the local region's geometry and texture.

[0056] S34: Construct a two-sided interpolation weight by fusing spatial distance constraints and hash semantic difference constraints, and adopt... The nearest neighbor interpolation method is used to solve for the core attributes of missing anchor points and generate low-frequency baseline data. The implementation process is as follows: After obtaining the neighborhood set and hash features, the bilateral interpolation weights are calculated. The bilateral interpolation weights are jointly determined by the spatial distance penalty (spatial weight) and the semantic difference penalty (semantic weight). Spatial weights ( Spatial weight is inversely proportional to the physical distance between two points; the closer the distance, the greater the weight. The expression for spatial weight is as follows: ; in, For spatial scale hyperparameters; Semantic weight The semantic weight is inversely proportional to the distance between two points in the hash feature space. The more similar the semantic features, the greater the weight. The expression for semantic weight is as follows: ; in, For semantic feature scale hyperparameters; Calculate all known anchor points in the neighborhood. For missing anchor points Joint unnormalized weights The final bilateral interpolation weights are obtained by normalizing the weights. : ; ; Finally, using this bilateral weighting For missing anchor points Core attributes (features) ,scale Offset A weighted summation is performed to calculate a high-quality interpolation baseline. : ; ; ; Due to the uniform dispersion of the missing points, within this neighborhood The known anchor points are typically physically close to the missing point. This ensures the reliability of subsequent local prior information.

[0057] S4: Based on the prior perceptual graph neural network, high-frequency residual prediction is completed. Combined with confidence judgment, attribute fusion repair and safe rollback are realized to complete the hidden reconstruction of packet loss error in 3D Gaussian scene.

[0058] In this embodiment, step S4 specifically includes: S41: Construct a local graph topology using missing anchor points as the central node and available neighboring anchor points as ordinary nodes. Initialize graph node features by concatenating spatial locations, core attributes, and hashed prior features. For missing nodes, use interpolation baselines to fill feature positions. The implementation process is as follows: For the set to be repaired Missing anchors in With its physical coordinates Centered on the neighborhood constructed in S3 Establish graph topology in local space .

[0059] Node set : Contains a central missing node and known anchor nodes in the neighborhood. .

[0060] edge set : Includes neighboring nodes Pointing to the central node The directed edge.

[0061] Initialize the input feature vector for each node For known nodes Its features are concatenated from location, core attributes (features, scale, offset) and hash / MLP prior features. For missing nodes Its core properties are derived from the low-frequency interpolation baseline calculated in main step three. Perform placeholder initialization: ; ; in, This represents the concatenation operation of the feature channel dimensions.

[0062] S42: A single-layer EdgeConv network is introduced to extract edge features between nodes, and neighborhood feature information is aggregated to predict the high-frequency residual components corresponding to the core attributes of missing anchor points. The implementation process is as follows: First, define the edge feature extraction function. (This embodiment is implemented using a multilayer perceptron (MLP), with the following parameters:) ), used to calculate neighboring nodes For the central node Residual information representation The expression is as follows: ; Subsequently, a symmetric aggregation function (such as max or mean pooling) is used to aggregate all edge features within the local neighborhood, outputting a corrected feature representation of the missing anchor points. : ; Finally, the corrected features are passed through a regression prediction head (composed of fully connected layers). High-frequency residual prediction values ​​mapped to target core attributes (Including characteristic residuals) Scale residuals and offset residual The expression for the high-frequency residual prediction value is as follows: .

[0063] S43: Calculate the confidence score of local neighborhood feature consistency, complete the weighted fusion of interpolation baseline and high-frequency residual to obtain the final repair attribute, and configure the confidence threshold to achieve safe rollback. The implementation process is as follows: Calculate the confidence score of the residual prediction based on the geometric connectivity or characteristic variance of the local neighborhood. .

[0064] When local neighborhood information is sufficient and highly consistent Approaching 1; when the neighborhood is extremely sparse or the hash feature differences are too large (indicating that it is at the boundary of a complex object and is severely missing), Approaching 0.

[0065] Final fix to anchor point properties The interpolation baseline is obtained by fusing it with the weighted high-frequency residuals. ; ; ; In this embodiment, the safety rollback mechanism is as follows: Set confidence threshold When the prediction confidence level of the current region is assessed At that time, the residual term is forcibly set to zero (i.e., This allows the final repair result to safely revert to the bilateral interpolation baseline based on hash priors in S3. .

[0066] In addition, this embodiment also provides a three-dimensional Gaussian representation error-hiding device for packet loss resistance, used to implement the three-dimensional Gaussian representation error-hiding method for packet loss resistance as described above. The error-hiding device includes a data extraction and grouping module, an atomized packet transmission module, a receiver baseline repair module, and a graph neural network residual repair module, wherein: The data extraction and grouping module is used to perform anchor point attribute extraction and spatial uniform random grouping processes. The atomic packet transmission module is used to complete the atomic encapsulation of anchor point attributes, unreliable channel transmission, and packet loss failure mode conversion. The receiver baseline repair module is used to realize packet loss identification, neighborhood construction and spatial semantic bilateral weighted interpolation baseline solution; The graph neural network residual repair module is used to construct a local graph structure, predict high-frequency attribute residuals, and execute confidence fusion and safe rollback repair logic.

[0067] It should be noted that computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0068] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0069] In addition, this embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the error hiding method described above.

[0070] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for performing the above-described error hiding method. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the error hiding method provided in the above embodiments, and will not be repeated here.

[0071] The above description is only a preferred embodiment of the present invention and does not limit the scope of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of the present invention.

Claims

1. A three-dimensional Gaussian representation-based error concealment method for resisting packet loss, characterized in that, Includes the following steps: S1: Complete the three-dimensional Gaussian anchor point attribute extraction and spatial uniform hierarchical random grouping, separate anchor point position information from core attributes and complete global spatial dimension shuffling and grouping; S11: Independently read the three-dimensional Gaussian anchor point position information, and separately parse and extract the core attribute tuples of each anchor point without position coordinates. The core attribute tuples include high-dimensional feature attributes, scale attributes, and relative quantization grid offset attributes. S12: Based on the 3D scene bounding box, perform mesh spatial layering on the global 3D space, and divide all Gaussian anchor points into corresponding spatial sub-regions; S13: A hierarchical random sampling method without replacement is used to complete the spatial uniform random group mapping, so that the anchor points in the same transmission group are uniformly and discretely distributed in the three-dimensional global space. S14: By combining Morton order dimensionality reduction sorting with pseudo-random hash distribution, the uniform grouping and arrangement of three-dimensional anchor points in the global space is achieved with low computing power. The mapping between Morton's order and pseudo-random hash shuffling is as follows: After quantizing the three-dimensional coordinates of the anchor points, the Morton code is solved to complete the one-dimensional sequence sorting. The sorted sequence is divided into continuous batches with fixed capacity. A pseudo-random hash function with a fixed seed is used to perform indexing operations on the anchor points within the batch. The anchor points are assigned to different transmission packets based on the hash modulo result. S2: Perform anchor-level atomic packaging and channel transmission for anti-spoofing, bind and encapsulate the anchor-strongly related attributes and send them into the unreliable channel for transmission. S21: Based on the spatial uniform grouping results, bind and encapsulate the core attributes of all Gaussian anchor points within the same group into an indivisible atomic data packet, and synchronously configure the anchor point activation mask within the data packet; S22: Establish an unreliable network channel transmission model, set data packet reception status variables, and distinguish between two states: successful data packet reception and transmission loss. S23: Based on the atomic packing rules, the packet loss failure mode transformation is completed, so that all core attributes of a single Gaussian anchor point share the same data packet reception state, realizing the all-or-nothing failure mechanism in the packet loss scenario. S3: Complete packet loss location at the receiving end and use local interpolation to generate missing anchor point attributes to restore the baseline; S31: The decoding end completes packet loss detection through the data packet sequence number, and combines the independently stored global anchor point location information to divide the set of missing anchor points into the set of normal available anchor points; S32: Based on the three-dimensional spatial coordinates of the missing anchor point, filter nearby available anchor points based on the three-dimensional Euclidean distance to construct the local spatial neighborhood of the missing anchor point; S33: Based on the HAC hash grid structure, hash semantic features are extracted by querying the spatial coordinates of anchor points to obtain implicit semantic priors of local region geometry and texture; S34: Construct bilateral interpolation weights by integrating spatial distance constraints and hash semantic difference constraints, and use k-nearest neighbor interpolation to solve for the core attributes of missing anchor points to generate low-frequency restored baseline data; S4: Based on the prior perceptual graph neural network, high-frequency residual prediction is completed. Combined with confidence judgment, attribute fusion repair and safe rollback are realized to complete the hidden reconstruction of packet loss error in 3D Gaussian scene. S41: Construct a local graph topology with missing anchor points as the central node and available anchor points in the neighborhood as ordinary nodes. Piece together spatial location, core attributes, and hash prior features to complete the graph node feature initialization. Use interpolation baselines to fill in the missing nodes for feature placement. S42: Introduce a single-layer EdgeConv network to extract edge features between nodes and aggregate neighborhood feature information to predict the high-frequency residual components corresponding to the core attributes of missing anchor points. S43: Calculate the confidence score of local neighborhood feature consistency, complete the weighted fusion of interpolation baseline and high-frequency residual to obtain the final repair attribute, and configure the confidence threshold to achieve safe rollback.

2. The error concealment method for three-dimensional Gaussian representation against packet loss according to claim 1, characterized in that, In S2, the all-or-nothing failure mechanism is as follows: If the data packet is received normally, all core attributes of the corresponding anchor point are complete and usable. If the data packet is lost, all anchor points within the data packet are judged to be completely missing, forming a clean missing void without attribute disorder, thus preventing rendering artifacts caused by the loss of a single attribute.

3. The error concealment method for three-dimensional Gaussian representation against packet loss according to claim 1, characterized in that, The bilateral interpolation weights in S34 include spatial weights and semantic weights. The spatial weights are negatively correlated with the physical spatial distance of the anchor point, and the semantic weights are negatively correlated with the spatial distance of the hash feature. After normalization, the two types of weights are jointly used to complete the weighted interpolation calculation of the anchor point attributes.

4. The error concealment method for three-dimensional Gaussian representation against packet loss according to claim 1, characterized in that, In S4, the safe rollback mechanism is as follows: When the confidence level of the residual prediction is lower than the confidence threshold, the high-frequency residual components are cleared, and the bilateral interpolation baseline is directly used as the missing anchor point to finally repair the attribute.

5. A three-dimensional Gaussian representation error-hiding device for resisting packet loss, characterized in that, For implementing the three-dimensional Gaussian representation error-hiding method for resisting packet loss as described in any one of claims 1 to 4, the error-hiding device includes a data extraction and grouping module, an atomized packet transmission module, a receiver baseline repair module, and a graph neural network residual repair module, wherein: The data extraction and grouping module is used to perform anchor point attribute extraction and spatial uniform random grouping processes. The atomic packet transmission module is used to complete the atomic encapsulation of anchor point attributes, unreliable channel transmission, and packet loss failure mode conversion. The receiver baseline repair module is used to realize packet loss identification, neighborhood construction and spatial semantic bilateral weighted interpolation baseline solution; The graph neural network residual repair module is used to construct a local graph structure, predict high-frequency attribute residuals, and execute confidence fusion and safe rollback repair logic.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer-executable program, which, when invoked by a processor, implements the error-hiding method for resisting packet loss using three-dimensional Gaussian representation as described in any one of claims 1 to 4.

Images