WebSocket-based real-time data interaction method for three-dimensional scene

Abstract

The application discloses a WebSocket-based real-time data interaction method for a three-dimensional scene and belongs to the technical field of three-dimensional scene data interaction. The method comprises the following steps: establishing a WebSocket bidirectional session between a three-dimensional client and a server; extracting an object space identifier, a parent node chain, a geometric bounding volume and a material change amount from three-dimensional scene changes, and generating an operation frame; the operation frame carries a dependent operation identifier and a session sequence mark; the server constructs a partial order queue according to the dependent operation identifier, and pushes the operation frame through the WebSocket bidirectional session; the three-dimensional client submits the operation frame to a scene graph according to the partial order queue after receiving the operation frame, and if a previous operation frame corresponding to the dependent operation identifier is not reached, the operation frame is stored in a temporary area, and a missing request is sent through the WebSocket bidirectional session; and the server retransmits the operation frame according to the missing request. Through the dependent operation identifier, the partial order queue and the missing retransmission mechanism, the method reduces the transmission data amount in real-time data interaction of the three-dimensional scene, and improves scene submission consistency.

Description

Technical Field

[0001] This invention relates to the field of 3D scene data interaction technology, specifically a 3D scene real-time data interaction method based on WebSocket. Background Technology

[0002] Real-time data interaction in 3D scenes requires timely synchronization of scene changes occurring on the server to the 3D client in scenarios such as digital twins, online collaboration, and cloud rendering. Existing technologies often use HTTP polling or ordinary WebSocket push to achieve scene updates. HTTP polling requests scene data at fixed time intervals, and its real-time performance is limited by the polling cycle, and it may repeatedly transmit unchanged scene content. Although ordinary WebSocket push establishes a persistent connection between the client and the server, the operation frames generated by the server usually lack descriptions of dependencies between scene objects and global order information. When the client receives operation frames that are out of order or missing, it cannot determine the submission conditions of the operation frames, and may directly discard frames that cannot be applied immediately or trigger a full scene retransmission, resulting in inconsistent scene graph node attributes, incorrect spatial bounding volume updates, or material state rollback. The server lacks precise retransmission criteria for lost frames and can only perform coarse-grained scene data retransmission.

[0003] Existing operation frames also have a relatively coarse granularity in expressing changes to 3D scenes. The server often uses a complete scene snapshot or all object attributes as the update unit, without combining object space identifiers, parent node chains, geometric bounding volumes, and material channel increments to generate operation frames. Clients cannot perform local positioning and updates based on the hierarchical paths of scene graph nodes and material channels, increasing invalid data transmission and client processing burden. In WebSocket bidirectional sessions, the design needs to address the dependencies and order recovery between operation frames, enabling the client to temporarily store the current operation frame and resend it via session request when a dependent frame is missing. Furthermore, 3D scene changes need to be refined into object space information and material channel changes, allowing operation frames to describe local changes in scene nodes to meet the consistency and transmission efficiency requirements of real-time 3D scene interaction. Summary of the Invention

[0004] To address the issues of insufficient dependency order control and inadequate granularity in scene change expression during real-time data interaction in existing 3D scenes, this invention provides a WebSocket-based method for real-time data interaction in 3D scenes. By establishing a two-way WebSocket session, the server generates operation frames carrying dependency operation identifiers and session sequence numbers. A partially ordered queue is constructed based on the dependency operation identifiers. The client temporarily stores missing dependency frames and requests retransmission, achieving ordered submission of 3D scene changes on the client side. The server extracts object space identifiers, parent node chains, geometric bounding volumes, and material change amounts from the 3D scene changes, ensuring that operation frames contain object space hierarchy information and material channel increments. This method reduces the impact of network out-of-order processing and frame loss on the scene. Figure 1 This reduces the amount of data transmitted during 3D scene data interaction, thus minimizing the impact on consistency.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a real-time data interaction method for 3D scenes based on WebSocket, comprising: Establish a two-way WebSocket session between the 3D client and server; The server extracts object space identifiers, parent node chains, geometric bounding volumes, and material change amounts from 3D scene changes, and generates operation frames. The operation frame carries a dependent operation identifier and a session sequence number; The server constructs a partial-order queue according to the dependent operation identifier and pushes the operation frame through the WebSocket bidirectional session. After receiving the operation frame, the 3D client submits it to the scene graph according to the partial order queue. If the earlier operation frame corresponding to the dependent operation identifier has not arrived, the operation frame is stored in the temporary storage area and a missing request is sent through the WebSocket bidirectional session. The server resends the operation frame based on the missing request.

[0006] Preferably, the server extracts object space identifiers, parent node chains, geometric bounding volumes, and material change amounts from the 3D scene changes, and generates an operation frame, including: When the 3D scene changes, the server reads the object space identifier and parent node chain of the object being changed in the scene graph; Calculate the hierarchy depth of the modified object based on the parent node chain, and generate the dependent operation identifier of the operation frame according to the hierarchy depth; Write the minimum and maximum corner points of the geometric bounding volume into the spatial field of the operation frame; The material change amount is split into channel change records according to the material channel, and the channel change records are bound to the object space identifier and written into the operation frame.

[0007] Preferably, the server constructs a partial-order queue according to the dependent operation identifier and pushes the operation frame through the WebSocket bidirectional session, including: The server assigns a session sequence number to each operation frame and looks up the earlier operation frame according to the dependent operation identifier; If the preceding operation frame has not entered the sending queue, the current operation frame is placed in the waiting queue, and the session sequence number is recorded in the waiting queue. Once the preceding operation frame enters the sending queue, the current operation frame is moved from the waiting queue into the sending queue and sorted according to the session sequence number; The operation frames are pushed one by one according to the sorted sending queue through the WebSocket bidirectional session.

[0008] Preferably, storing the operation frame in a temporary storage area and sending a missing request via the WebSocket bidirectional session includes: The 3D client reads the object space identifier, the dependent operation identifier, and the session sequence number from the operation frame; The 3D client searches for the target node in the scene graph index according to the object space identifier; If the prior operation frame has been submitted, then the operation frame is applied to the target node according to the session sequence number; If the preceding operation frame has not been submitted, the operation frame is written to the temporary storage area, and a missing request is generated according to the dependent operation identifier. The missing request is sent to the server via the WebSocket bidirectional session.

[0009] Preferably, the step of splitting the material change amount into channel change records according to material channels, and binding the channel change records with the object space identifier and writing them into the operation frame includes: Read the base color change value, roughness change value, and luminance change value from the material change amount according to the material channel; The base color change value, the roughness change value, and the luminance change value are each encapsulated into a channel change record; Assign a channel identifier to each of the aforementioned channel change records; The channel identifier and the object space identifier are combined to form a binding key, and the binding key is written into the material field of the operation frame.

[0010] Preferably, the step of pushing the operation frame frame by frame through the sorted sending queue via the WebSocket bidirectional session includes: The server reads the round-trip latency of the WebSocket bidirectional session and the rendering frame budget of the 3D client; The sending queue is divided into push windows according to the rendering frame budget, and the operation frame is selected in each push window according to the session index. The selected operation frame is encapsulated into a WebSocket data frame, and the window identifier of the push window and the session sequence number are written into the extended field of the WebSocket data frame.

[0011] Preferably, establishing a two-way WebSocket session between the 3D client and the server includes: The 3D client sends a WebSocket handshake request carrying the scene identifier and the client capability identifier to the server; The server looks up the scene session table based on the scene identifier; If the scene identifier exists in the scene session table, the server writes the client capability identifier into the scene session table. The server returns a WebSocket handshake response to the 3D client, carrying the session identifier and heartbeat cycle; The 3D client sends heartbeat frames to the server according to the heartbeat cycle, and the server updates the scene session table according to the session identifier.

[0012] Preferably, the 3D client searches for the target node in the scene graph index according to the object space identifier, including: The 3D client reads the root node index table from the scene graph index; The parent node is located level by level in the root node index table according to the hierarchical path in the object space identifier; The target node is located in the child node index table of the parent node according to the last-level identifier in the object space identifier; If the target node does not exist, the 3D client creates a placeholder node in the scene graph index and writes the object space identifier into the placeholder node; Return the placeholder node as the target node.

[0013] Preferably, the server reads the round-trip latency of the WebSocket two-way session and the rendering frame budget of the 3D client, including: The server sends timestamp probe frames and records the sending time in the WebSocket bidirectional session; The server receives the response frame of the timestamp detection frame returned by the 3D client and records the receiving time; The round-trip time of the WebSocket bidirectional session is obtained based on the sending time and the receiving time; During the registration phase, the 3D client reports the rendering frame interval and the available duration of a single frame to the server. The server combines the rendering frame interval with the available duration of a single frame into the rendering frame budget.

[0014] Preferably, if the target node does not exist, the 3D client creates a placeholder node in the scene graph index and writes the object space identifier into the placeholder node, including: The 3D client generates placeholder node identifiers based on the object space identifiers; Create the placeholder node in the scene graph index according to the hierarchical path; Write the last-level identifier into the node name field of the placeholder node; Write the object space identifier into the index key field of the placeholder node; Add the placeholder node identifier to the queue to be bound, and record the session sequence number in the queue to be bound.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: The operation frame contains a dependency operation identifier and a session sequence number. The server constructs a partially ordered queue based on the dependency operation identifier. The dependency operation identifier is calculated based on the parent node chain and hierarchy depth, ensuring that operation frames retain the parent-child or attribute dependency relationships in the scene graph. The session sequence number identifies the order of operation frames within the same dependency set. The server assigns a session sequence number to each operation frame and looks up the preceding operation frame corresponding to the dependency operation identifier. If the preceding operation frame is not in the sending queue, the current operation frame enters the waiting queue. Once the preceding operation frame enters the sending queue, the current operation frame is moved into the sending queue and sorted by the session sequence number. Operation frames without dependencies can skip the waiting queue and participate in the push directly. After receiving the operation frame, the 3D client reads the dependency operation identifier and session sequence number and looks up the target node in the scene graph index. If the preceding operation frame corresponding to the dependency operation identifier has been committed, the operation frame is applied to the target node according to the session sequence number. If the preceding operation frame has not been committed, the operation frame is written to the temporary storage area, and a missing request is sent via a WebSocket bidirectional session. The server resends the corresponding operation frame based on the missing request. This method avoids the loss of all subsequent operation frames or triggers a full retransmission due to the missing of a single frame when WebSocket messages are out of order or packets are lost. It ensures that operation frames with dependencies are submitted in order, while operation frames without dependencies are not affected by waiting, reducing recovery waiting time and the amount of retransmitted data, and maintaining the consistency of the state of the 3D scene graph.

[0016] The server extracts object space identifiers, parent node chains, geometric bounding volumes, and material change values ​​from 3D scene changes to generate an operation frame. Object space identifiers and parent node chains are used to locate target nodes in the client's scene graph index and serve as the basis for dependency calculations. The geometric bounding volume is written into the space field of the operation frame using the minimum and maximum corner points, describing changes in the object space range. Material change values ​​are split by material channel into base color change values, roughness change values, and luminance change values, each encapsulated as a channel change record, assigned a channel identifier, and bound to the object space identifier as a binding key, which is then written into the material field of the operation frame. The client locates the parent node level by level in the root node index table based on the hierarchical path in the object space identifier and searches for the target node in the parent node's child node index table. If the target node does not exist, a placeholder node is created, the object space identifier is written to the placeholder node, and it is returned as the target node. The client only updates the changed material channel and geometric bounding volume fields; it does not need to receive all object attributes or a full scene snapshot. This representation method enables 3D scene changes to be transmitted incrementally at the object and channel levels, reducing invalid data transmission, lowering network load and client scene graph update latency, while avoiding repeated synchronization of unchanged material channels and spatial fields. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of a real-time data interaction method for 3D scenes based on WebSocket; Figure 2 This is a flowchart of the partial-order queue and push window operation frame push process based on the dependency operation identifier. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1This invention provides a method for real-time data interaction in a 3D scene based on WebSocket. A two-way WebSocket session is established between the 3D client and the server, through which both parties send and receive data. When changes occur in the 3D scene, the server extracts the object space identifier, parent node chain, geometric bounding volume, and material change amount from the changed object, generating an operation frame. The operation frame carries a dependent operation identifier and a session sequence number. The server constructs a partial-order queue based on the dependent operation identifier and pushes the operation frame through the WebSocket two-way session. After receiving the operation frame, the 3D client submits it to the scene graph according to the partial-order queue. When a prior operation frame corresponding to a dependent operation identifier has not arrived, the 3D client stores the operation frame in a temporary storage area and sends a missing request through the WebSocket two-way session. The server resends the operation frame based on the missing request.

[0021] In practice, the server determines the object being changed when the 3D scene changes. The server reads the object's object space identifier in the scene graph. The server also reads the parent node chain of the object in the scene graph. The object space identifier is used to uniquely identify the object in the scene graph index. The parent node chain contains the sequence of parent nodes of the object in the scene graph tree.

[0022] The server calculates the hierarchy depth of the modified object based on the parent node chain. The server counts the number of parent nodes in the parent node chain and increments this count by one to obtain the hierarchy depth. The server generates dependency operation identifiers for operation frames based on the hierarchy depth. Each dependency operation identifier corresponds to a hierarchy depth. The operation frame carries both the dependency operation identifier and the session sequence number.

[0023] In practice, the server constructs an operation frame structure. This structure includes a dependent operation identifier field, a session sequence number field, a space field, and a material field. The server writes the dependent operation identifier into the dependent operation identifier field. The server also writes the session sequence number into the session sequence number field.

[0024] The server writes the minimum corner point of the bounding volume to the spatial field of the operation frame. The server also writes the maximum corner point of the bounding volume to the spatial field of the operation frame. The bounding volume is extracted from the 3D scene modification. The spatial field records the bounding area of ​​the modified object. The spatial field contains a minimum corner point field and a maximum corner point field. The minimum corner point field writes the minimum corner point of the bounding volume. The maximum corner point field writes the maximum corner point of the bounding volume.

[0025] In practice, the server breaks down material changes into channel change records based on material channels. The server then binds each channel change record to an object space identifier and writes it into the operation frame.

[0026] The server reads the base color change value, roughness change value, and luminance change value from the material change values ​​according to the material channel. The server encapsulates the base color change value into a base color channel change record. The server encapsulates the roughness change value into a roughness channel change record. The server encapsulates the luminance change value into a luminance channel change record.

[0027] The server assigns a base color channel identifier to the base color channel change record. The server assigns a roughness channel identifier to the roughness channel change record. The server assigns a light emission channel identifier to the light emission channel change record.

[0028] The server binds the base color channel identifier to the object space identifier to form a base color binding key. The server binds the roughness channel identifier to the object space identifier to form a roughness binding key. The server binds the emissivity channel identifier to the object space identifier to form an emissivity binding key.

[0029] The server writes the base color binding key, roughness binding key, and luminance binding key into the material field of the operation frame. The material field also contains records of changes to the base color channel, roughness channel, and luminance channel.

[0030] As can be understood, the material field of the operation frame contains a binding key and a channel change record. The binding key is used to associate the object space identifier with the material channel. The channel change record is used to describe the changed value of the corresponding material channel.

[0031] In specific implementation, please refer to Figure 2 The server constructs a partial-order queue based on dependent operation identifiers. The server assigns a session sequence number to each operation frame. Operation frames are generated by the server when the 3D scene changes. The session sequence number is used to identify the order of operation frames in the partial-order queue. The server reads the dependent operation identifiers carried in the operation frames. The server finds the earlier operation frame based on the dependent operation identifier. The earlier operation frame is the operation frame identified by the dependent operation identifier.

[0032] If a preceding operation frame has not entered the sending queue, the server places the current operation frame into the waiting queue. The server records the session sequence number corresponding to the current operation frame in the waiting queue. The waiting queue is maintained on the server side and is used to temporarily store the current operation frame.

[0033] Once an earlier operation frame enters the send queue, the server moves the current operation frame from the waiting queue to the send queue. The server sorts the operation frames in the send queue according to the session sequence number. Operation frames are then pushed frame by frame through the sorted send queue via a WebSocket bidirectional session.

[0034] In practice, the server reads the round-trip latency of the WebSocket bidirectional session and the rendering frame budget of the 3D client. The server uses the rendering frame budget as the basis for dividing the push window. The server divides the sending queue into push windows according to the rendering frame budget. Each push window contains a set of operation frames from the sending queue. Within each push window, the server selects an operation frame according to the session sequence number. The server encapsulates the selected operation frame into a WebSocket data frame. The server writes the window identifier and session sequence number of the push window into the extended fields of the WebSocket data frame.

[0035] As can be understood, push windows are used to control the pace of pushing operation frames. The server determines the number of operation frames that can be pushed within each push window based on the rendering frame budget. The server then distributes the operation frames in the sending queue into the corresponding push windows according to the session sequence number.

[0036] Optionally, the server writes the window identifier and session sequence number of the push window into the extended field of the WebSocket data frame, so that the 3D client can identify the push window to which the operation frame belongs and the order of the operation frames within the push window.

[0037] In practice, the 3D client receives operation frames pushed by the server via a two-way WebSocket session. The 3D client reads the object space identifier, dependent operation identifier, and session sequence number from the operation frame. The object space identifier uniquely identifies the modified object in the scene graph index. The dependent operation identifier identifies the dependencies between operation frames. The session sequence number indicates the order of the operation frames in the partial order queue.

[0038] The 3D client searches for the target node in the scene graph index by its object space identifier. The 3D client reads the root node index table from the scene graph index. The 3D client then locates the parent node level by level in the root node index table according to the hierarchical path in the object space identifier. The hierarchical path consists of multiple levels of identifiers in the object space identifier. The 3D client searches for the parent node level by level in the root node index table according to the hierarchical path. Finally, the 3D client searches for the target node in the child node index table of the parent node by the lowest-level identifier in the object space identifier.

[0039] If the target node exists, the 3D client uses it as the search result. If the target node does not exist, the 3D client creates a placeholder node in the scene graph index and writes the object space identifier into the placeholder node. The 3D client then returns the placeholder node as the target node.

[0040] In practice, the 3D client generates placeholder node identifiers based on the object space identifier. The 3D client uses a pre-defined hash function to calculate the placeholder node identifiers. The calculation formula is as follows:

[0041] in, This indicates a placeholder node identifier. This indicates a preset hash function. The input of the preset hash function is the object space identifier and the session sequence number, and the output is a fixed-length identifier value. This indicates the object space identifier, which is read from the operation frame. This indicates the session sequence number, which is read from the operation frame.

[0042] The 3D client creates placeholder nodes in the scene graph index according to the hierarchical path. The 3D client writes the last-level identifier to the node name field of the placeholder node. The 3D client writes the object space identifier to the index key field of the placeholder node. The 3D client adds the placeholder node identifier to the binding queue. The 3D client records the session sequence number in the binding queue.

[0043] In practice, the 3D client checks whether the prior operation frame corresponding to the dependent operation identifier has been committed. If the prior operation frame has been committed, the 3D client applies the operation frame to the target node according to the session sequence number. If the prior operation frame has not been committed, the 3D client writes the operation frame to the temporary storage area. The 3D client generates a missing request according to the dependent operation identifier. The missing request carries the dependent operation identifier and the scene identifier. The 3D client sends the missing request to the server via a WebSocket bidirectional session.

[0044] It is understandable that placeholder nodes are used to provide temporary node positions when there is no target node in the scene graph index. When the 3D client subsequently receives the prior operation frame corresponding to the dependent operation identifier, it locates the placeholder node in the scene graph index based on the placeholder node identifier and applies the operation frame to the placeholder node when submitting the operation frame in the temporary area.

[0045] In practice, the 3D client sends a WebSocket handshake request carrying a scene identifier and a client capability identifier to the server. The 3D client reads the scene identifier of the current 3D scene. The scene identifier identifies the 3D scene. The 3D client reads the client capability identifier. The client capability identifier describes the rendering and interactive capabilities supported by the 3D client. The 3D client constructs a WebSocket handshake request. The message header of the WebSocket handshake request includes a scene identifier field and a client capability identifier field. The 3D client writes the scene identifier into the scene identifier field of the WebSocket handshake request. The 3D client writes the client capability identifier into the client capability identifier field of the WebSocket handshake request. The 3D client sends the WebSocket handshake request to the server via the WebSocket protocol.

[0046] In practice, the server parses the scenario identifier from the WebSocket handshake request. The server then searches its maintained scenario session table for a matching entry. The scenario session table records information about the scenario sessions established with the server. It contains multiple entries, each with at least a scenario identifier field, a session identifier field, a client capability identifier field, a heartbeat cycle field, and a last heartbeat time field. The server uses the scenario identifier as the lookup key. The server iterates through the scenario session table to determine if an entry with the same scenario identifier already exists.

[0047] If a matching entry exists in the scenario session table, the server parses the client capability identifier from the WebSocket handshake request. The server then writes the client capability identifier into the client capability identifier field of the matching entry in the scenario session table. Finally, the server overwrites the original client capability identifier with the new one.

[0048] If no matching entry exists in the scenario session table, the server creates a new entry. The server then writes the scenario identifier into the scenario identifier field of the new entry. Finally, the server writes the client capability identifier into the client capability identifier field of the new entry.

[0049] In practice, the server generates a session identifier based on the scenario identifier and the client's capability identifier. The session identifier calculation formula is as follows:

[0050] in, This represents the session identifier, which is used to identify the two-way WebSocket session between the 3D client and the server; This represents the scene identifier, which is carried by the 3D client in the WebSocket handshake request to identify the 3D scene; This represents the client capability identifier, which is carried by the 3D client in the WebSocket handshake request and is used to describe the rendering and interaction capabilities supported by the 3D client. This represents the string concatenation operator, which concatenates the string of the scene identifier and the string of the client capability identifier in sequence to generate the session identifier.

[0051] The server reads the heartbeat cycle from the scene session table. The heartbeat cycle is pre-configured by the server based on the client's capability identifier. The server constructs a WebSocket handshake response. The WebSocket handshake response includes a session identifier field and a heartbeat cycle field. The server writes the session identifier to the session identifier field of the WebSocket handshake response. The server writes the heartbeat cycle to the heartbeat cycle field of the WebSocket handshake response. The server sends the WebSocket handshake response to the 3D client.

[0052] It is understandable that the session identifier and heartbeat cycle are written into the WebSocket handshake response, and the three-dimensional client obtains the session identifier and heartbeat cycle by parsing the WebSocket handshake response.

[0053] In practice, the 3D client parses the session identifier and heartbeat period from the WebSocket handshake response. The 3D client starts a heartbeat timer. The period of the heartbeat timer is the heartbeat period. When the heartbeat timer expires, the 3D client constructs a heartbeat frame. The heartbeat frame contains a session identifier field and a timestamp field. The 3D client writes the session identifier into the session identifier field of the heartbeat frame. The 3D client writes the current time into the timestamp field of the heartbeat frame. The 3D client then sends the heartbeat frame to the server via a two-way WebSocket session.

[0054] The server receives heartbeat frames sent by the 3D client via a WebSocket bidirectional session. The server parses the session identifier from the heartbeat frame. The server searches the scene session table for an entry that matches the session identifier in the heartbeat frame. The server writes the time the heartbeat frame was received into the last heartbeat time field of the matching entry. The server updates the scene session table according to the session identifier.

[0055] In practice, the server reads the round-trip latency of the WebSocket bidirectional session and the rendering frame budget of the 3D client. The server sends a timestamp probe frame within the WebSocket bidirectional session. The timestamp probe frame includes a probe identifier field and a sending time field. The server generates the probe identifier. The server reads its local clock and uses the local clock value as the sending time. The server writes the probe identifier into the probe identifier field of the timestamp probe frame. The server writes the sending time into the sending time field of the timestamp probe frame. The server then sends the timestamp probe frame to the 3D client via the WebSocket bidirectional session.

[0056] After receiving a timestamp probe frame, the 3D client constructs a response frame. The 3D client reads the probe identifier and sending time from the timestamp probe frame. The 3D client writes the probe identifier to the probe identifier field of the response frame. The 3D client writes the sending time to the sending time field of the response frame. The 3D client then returns the response frame to the server via a WebSocket bidirectional session.

[0057] The server receives a response frame from the 3D client, which contains a timestamp probe frame. The server reads its local clock and uses this value as the received time. Based on the sending and receiving times, the server calculates the round-trip latency of the WebSocket session. The calculation formula is as follows:

[0058] in, This represents the round-trip latency of a WebSocket two-way session. The round-trip latency is the time elapsed from when the server sends a timestamp probe frame to when the server receives a response frame, in milliseconds. This indicates the receiving time, which is the server's local clock value read when the server receives the response frame; This indicates the sending time, which is the server's local clock value read when the server sends the timestamp probe frame.

[0059] server according to The round-trip time is calculated. The server stores the round-trip time in the session state corresponding to the WebSocket bidirectional session.

[0060] Optionally, the server periodically sends timestamp probe frames. Each time the server sends a timestamp probe frame, it records the new sending time. Each time the server receives a response frame, it records the new receiving time. The server updates the round-trip time based on the new sending and receiving times.

[0061] In practice, during the registration phase, the 3D client reports the rendering frame interval and the available duration of a single frame to the server. The 3D client reads the rendering frame interval, which is the time interval between two consecutive rendering frames, in milliseconds. The 3D client also reads the available duration of a single frame, which is the maximum time available for the 3D client to process a single rendering frame, in milliseconds. The 3D client constructs a registration message, which includes a rendering frame interval field and a single frame available duration field. The 3D client writes the rendering frame interval to the rendering frame interval field of the registration message. The 3D client also writes the single frame available duration to the single frame available duration field of the registration message. Finally, the 3D client sends the registration message to the server via a WebSocket bidirectional session.

[0062] The server receives the registration message. It then parses the rendering frame interval and the available duration per frame from the message. The server combines these components into a rendering frame budget. This budget includes both a rendering frame interval component and a single-frame available duration component. The rendering frame interval component represents the rendering frame interval itself, and the single-frame available duration component represents the single-frame available duration. The server stores the rendering frame budget in the session state corresponding to the WebSocket bidirectional session.

[0063] It's understandable that after the server reads the rendering frame budget, it divides the sending queue into push windows based on the rendering frame budget. The rendering frame interval component is used to determine the time span of the push window. The single-frame available duration component is used to limit the upper limit of the time for the 3D client to process operation frames.

[0064] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for real-time data interaction in a 3D scene based on WebSocket, characterized in that, include: Establish a two-way WebSocket session between the 3D client and server; The server extracts object space identifiers, parent node chains, geometric bounding volumes, and material change amounts from 3D scene changes, and generates operation frames. The operation frame carries a dependent operation identifier and a session sequence number; The server constructs a partial-order queue according to the dependent operation identifier and pushes the operation frame through the WebSocket bidirectional session. After receiving the operation frame, the 3D client submits it to the scene graph according to the partial order queue. If the earlier operation frame corresponding to the dependent operation identifier has not arrived, the operation frame is stored in the temporary storage area and a missing request is sent through the WebSocket bidirectional session. The server resends the operation frame based on the missing request.

2. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 1, characterized in that, The server extracts object space identifiers, parent node chains, geometric bounding volumes, and material change amounts from 3D scene changes, and generates an operation frame, including: When the 3D scene changes, the server reads the object space identifier and parent node chain of the object being changed in the scene graph; Calculate the hierarchy depth of the modified object based on the parent node chain, and generate the dependent operation identifier of the operation frame according to the hierarchy depth; Write the minimum and maximum corner points of the geometric bounding volume into the spatial field of the operation frame; The material change amount is split into channel change records according to the material channel, and the channel change records are bound to the object space identifier and written into the operation frame.

3. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 1, characterized in that, The server constructs a partial-order queue according to the dependent operation identifier and pushes the operation frame through the WebSocket bidirectional session, including: The server assigns a session sequence number to each operation frame and looks up the earlier operation frame according to the dependent operation identifier; If the preceding operation frame has not entered the sending queue, the current operation frame is placed in the waiting queue, and the session sequence number is recorded in the waiting queue. Once the preceding operation frame enters the sending queue, the current operation frame is moved from the waiting queue into the sending queue and sorted according to the session sequence number; The operation frames are pushed one by one according to the sorted sending queue through the WebSocket bidirectional session.

4. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 1, characterized in that, The step of storing the operation frame in a temporary storage area and sending a missing request via the WebSocket bidirectional session includes: The 3D client reads the object space identifier, the dependent operation identifier, and the session sequence number from the operation frame; The 3D client searches for the target node in the scene graph index according to the object space identifier; If the prior operation frame has been submitted, then the operation frame is applied to the target node according to the session sequence number; If the preceding operation frame has not been submitted, the operation frame is written to the temporary storage area, and a missing request is generated according to the dependent operation identifier. The missing request is sent to the server via the WebSocket bidirectional session.

5. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 2, characterized in that, The step of splitting the material change amount into channel change records according to the material channel, and binding the channel change records with the object space identifier and writing them into the operation frame includes: Read the base color change value, roughness change value, and luminance change value from the material change amount according to the material channel; The base color change value, the roughness change value, and the luminance change value are each encapsulated into a channel change record; Assign a channel identifier to each of the aforementioned channel change records; The channel identifier and the object space identifier are combined to form a binding key, and the binding key is written into the material field of the operation frame.

6. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 3, characterized in that, The step of pushing the operation frame frame by frame through the WebSocket bidirectional session according to the sorted sending queue includes: The server reads the round-trip latency of the WebSocket bidirectional session and the rendering frame budget of the 3D client; The sending queue is divided into push windows according to the rendering frame budget, and the operation frame is selected in each push window according to the session index. The selected operation frame is encapsulated into a WebSocket data frame, and the window identifier of the push window and the session sequence number are written into the extended field of the WebSocket data frame.

7. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 1, characterized in that, The establishment of a two-way WebSocket session between the 3D client and server includes: The 3D client sends a WebSocket handshake request carrying the scene identifier and the client capability identifier to the server; The server looks up the scene session table based on the scene identifier; If the scene identifier exists in the scene session table, the server writes the client capability identifier into the scene session table. The server returns a WebSocket handshake response to the 3D client, carrying the session identifier and heartbeat cycle; The 3D client sends heartbeat frames to the server according to the heartbeat cycle, and the server updates the scene session table according to the session identifier.

8. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 4, characterized in that, The 3D client searches for the target node in the scene graph index according to the object space identifier, including: The 3D client reads the root node index table from the scene graph index; The parent node is located level by level in the root node index table according to the hierarchical path in the object space identifier; The target node is located in the child node index table of the parent node according to the last-level identifier in the object space identifier; If the target node does not exist, the 3D client creates a placeholder node in the scene graph index and writes the object space identifier into the placeholder node; Return the placeholder node as the target node.

9. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 6, characterized in that, The server reads the round-trip latency of the WebSocket two-way session and the rendering frame budget of the 3D client, including: The server sends timestamp probe frames and records the sending time in the WebSocket bidirectional session; The server receives the response frame of the timestamp detection frame returned by the 3D client and records the receiving time; The round-trip time of the WebSocket bidirectional session is obtained based on the sending time and the receiving time; During the registration phase, the 3D client reports the rendering frame interval and the available duration of a single frame to the server. The server combines the rendering frame interval with the available duration of a single frame into the rendering frame budget.

10. The method for real-time data interaction in a 3D scene based on WebSocket according to claim 8, characterized in that, If the target node does not exist, the 3D client creates a placeholder node in the scene graph index and writes the object space identifier into the placeholder node, including: The 3D client generates placeholder node identifiers based on the object space identifiers; Create the placeholder node in the scene graph index according to the hierarchical path; Write the last-level identifier into the node name field of the placeholder node; Write the object space identifier into the index key field of the placeholder node; Add the placeholder node identifier to the queue to be bound, and record the session sequence number in the queue to be bound.

Images