A virtual reality scene position synchronization method and device

Abstract

The application provides a virtual reality scene position synchronization method and device, the method comprises the following steps: defining a field of a message sent between a client and a server; determining a time interval T1 of sending the message between the client and the server, and setting a client state sampling time interval T2; dynamically adjusting the time interval T1 of sending the message, and each client sends a message comprising client state information to the server based on the respective time interval T1 of sending the message; and other clients update the state information of the client corresponding to the received message based on the update step length set by the other clients and the received message, and simulate a virtual object of a user corresponding to the client corresponding to the received message. The method makes the movement of the client more synchronized on different device terminals of different clients, and can greatly reduce the visual difference when starting to move.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, and in particular to a method and apparatus for synchronizing virtual reality scene positions. Background Technology

[0002] Currently, with the rapid development of network technology, graphics, and hardware, AR / VR and online games are gradually becoming part of people's daily lives. More and more people are enriching their leisure time through computers, smartphones, AR / VR glasses, and other terminal devices, interacting and experiencing virtual reality. To achieve multi-user interaction in virtual reality, developers typically use a client / server (C / S) architecture, deploying one (or multiple) server and connecting to multiple clients via persistent TCP connections. Based on this C / S architecture, synchronization is a crucial concept, ensuring that the content seen by multiple client devices is consistent.

[0003] Once a user enters virtual reality (VR), a unique object is created within the VR environment, with corresponding coordinates. The quality of coordinate synchronization in VR significantly impacts the user experience during multi-user interaction. In apps with high real-time requirements, the client needs to frequently communicate with the server to calculate the user's coordinates in the virtual world. However, with numerous clients interacting with the server, the amount of data transmitted over the network becomes substantial, placing performance pressure on the server. Furthermore, the unpredictable nature of network transmission times can easily lead to asynchronous positioning among different clients.

[0004] Existing technology application number CN201811230734.3 provides a method for location synchronization in online games, suitable for virtual worlds with multiple users interacting simultaneously and requiring strong real-time performance. It offers a method for synchronizing game character positions with largely consistent screen display across multiple clients. However, existing technologies suffer from drawbacks: clients need to communicate frequently with the server, resulting in large amounts of data transmission; simultaneously, servers experience performance pressure with a large number of users; due to the uncertainty of network transmission, the time it takes for the client to receive messages about other users' current positions forwarded by the server is also uncertain (caused by network fluctuations). This leads to an unsmooth simulation of other users' positions in the virtual world, with inconsistent speeds and even instantaneous movement, impacting the user experience. Furthermore, current common technical solutions result in time differences in movement performance across different devices, which can affect user judgment and thus the experience in apps with high real-time requirements. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a virtual reality scene location synchronization method and apparatus to solve the technical problems of untimely location message synchronization and excessive data transmission between the client and server in the prior art.

[0006] According to a first aspect of the present invention, a method for synchronizing the location of a virtual reality scene is provided, the method comprising the following steps:

[0007] Step S1: Define the fields for sending messages between the client and the server to establish a communication connection between the client and the server; the client is installed on a virtual device, and each virtual device is configured with the client;

[0008] Step S2: Determine the time interval T1 for sending messages between the client and the server, and set the client status sampling time interval T2; each client records the client status information corresponding to each virtual device at each status sampling time point;

[0009] Step S3: For any client, if the change in client status information at the current sampling time point exceeds a preset first threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is reduced; if the change in client status information at the current sampling time point is less than a preset second threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is increased; each client sends a message including client status information to the server based on its corresponding time interval T1 for sending messages.

[0010] Step S4: After receiving the message sent by the client, the server sends the message to other clients; the other clients update the status information of the client corresponding to the received message stored in their own storage based on their own set update step size and the received message, and simulate the virtual object of the user corresponding to the client of the received message.

[0011] Preferably, in step S4, the other clients update their stored client status information corresponding to the received message based on their own set update step size and the received message, including:

[0012] Step S41: The other client has its own set update step size, which is obtained based on the other client's own current frame rate; after receiving the message, the other client parses the message to obtain the message content, which includes the current position POS, current orientation DIR, current speed SPEED, and current movement status flag of the client that sent the message.

[0013] Step S42: If the time interval between the other client and the last update of the client's status information corresponding to the message is less than or equal to the update step size, then update the status information of the client corresponding to the received message based on the message content.

[0014] Preferably, if the time interval between the last update of the status information of the client corresponding to the message and the other client is greater than the update step size, then the update of the status information of the client corresponding to the message is stopped.

[0015] Preferably, when the other client parses the current movement status identifier FLAG from the message sent by the server and finds that the field value is "stop", it stops its own movement within a third preset time period.

[0016] Preferably, updating the client status information corresponding to the received message includes: updating the client status information corresponding to the received message based on a correction factor and the message content; the correction factor is used to simulate the user's acceleration, and the client status information corresponding to the received message is updated by superimposing the correction factor and the message content.

[0017] Preferably, a fourth preset time threshold is set, and the status update of the client corresponding to the received message is completed within the time limit of the fourth preset time threshold.

[0018] Preferably, a fifth preset time threshold is set, wherein the fourth preset time threshold is less than the fifth preset time threshold; the fifth preset time threshold is the upper limit of the time interval T1 for sending messages.

[0019] According to a second aspect of the present invention, a virtual reality scene location synchronization device is provided, the device comprising:

[0020] Connection module: Configured to define fields for sending messages between the client and the server to establish a communication connection between the client and the server; the client is installed on a virtual device, and each virtual device is configured with the client;

[0021] Period determination module: configured to determine the time interval T1 for sending messages between the client and the server, and set the client status sampling time interval T2; each client records the client status information corresponding to each virtual device at each status sampling time point;

[0022] Determining Module: Configured to, for any client, if the change in client status information at the current sampling time point exceeds a preset first threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is reduced; if the change in client status information at the current sampling time point is less than a preset second threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is increased; each client sends a message including client status information to the server based on its corresponding time interval T1 for sending messages.

[0023] Simulation module: After the server receives a message sent by the client, it forwards the message to other clients; the other clients update the state information of the client corresponding to the received message stored in their own storage based on their own set update step size and the received message, simulating the virtual object of the user corresponding to the client of the received message.

[0024] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0025] Beneficial effects

[0026] This invention can minimize unnecessary location message synchronization between the client and the server, reduce the amount of data transmitted over the network, and alleviate the performance pressure on the server. The client simulates the user's movement in virtual reality through the movement parameters broadcast by the server, making the entire movement smoother and more fluid, without issues such as inconsistent speed or instantaneous movement, thus improving the user experience. It also reduces the time difference in location synchronization between different client devices, further enhancing the user experience.

[0027] This invention makes the movement of clients more synchronized across different client devices, significantly reducing visual discrepancies at the start of movement. Existing common location synchronization methods require other clients to begin simulating movement only after one client has moved for a period of time (hundreds of milliseconds). This invention makes the client's simulation of other users' movement smoother and more seamless, eliminating issues such as inconsistent movement speed or instantaneous teleportation, thus improving the user experience. This invention uses error elimination and other methods to keep the current client location synchronized with the server location. This invention also reduces the frequency of mobile protocol transmissions to some extent, decreasing the amount of data transmitted over the network and alleviating the performance pressure on the server. Attached Figure Description

[0028] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention, and the invention is illustrated by the following drawings. In the drawings:

[0029] Figure 1 This is a flowchart of a virtual reality scene position synchronization method according to one embodiment of the present invention;

[0030] Figure 2 This is a detailed flowchart illustrating a virtual reality scene position synchronization method according to one embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of a virtual reality scene position synchronization device according to one embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0033] First, combine Figure 1-2 This invention describes a virtual reality scene location synchronization method. The method includes the following steps:

[0034] Step S1: Define the fields for sending messages between the client and the server to establish a communication connection between the client and the server; the client is installed on a virtual device, and each virtual device is configured with the client;

[0035] Step S2: Determine the time interval T1 for sending messages between the client and the server, and set the client status sampling time interval T2; each client records the client status information corresponding to each virtual device at each status sampling time point;

[0036] Step S3: For any client, if the change in client status information at the current sampling time point exceeds a preset first threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is reduced; if the change in client status information at the current sampling time point is less than a preset second threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is increased; each client sends a message including client status information to the server based on its corresponding time interval T1 for sending messages.

[0037] Step S4: After receiving the message sent by the client, the server sends the message to other clients; the other clients update the status information of the client corresponding to the received message stored in their own storage based on their own set update step size and the received message, and simulate the virtual object of the user corresponding to the client of the received message.

[0038] The message is generated based on the communication protocol. The communication protocol is TCP, which uses a long connection. The message includes fields representing the client's status information, including current position (POS), current orientation (DIR), current speed (SPEED), and current movement status flag (FLAG). The FLAG indicates that the user corresponding to the client is in a walking, running, jumping, or stopped state.

[0039] In step S2, T2 is obtained based on the current frame rate of the client. For example, if the user is detected to be running and the frame rate corresponding to the client is 100, then T2 is 10ms. If the frame rate corresponding to the client is 20, then T2 is 50ms.

[0040] When performing the first sampling on the client, the client status information corresponding to the previous sampling time point is set to 0.

[0041] In this embodiment, the client status information at the current sampling time point is compared with the client status information at the previous sampling time point. If the deviation between the current movement orientation and the movement orientation at the previous sampling time point exceeds a first preset angle, then the change in status information exceeds a preset first threshold. The preset angle is, for example, 15 degrees. If the change rate of the current speed exceeds a first preset ratio, then the change in status information exceeds a preset first threshold. The current speed change ratio is calculated as: |current speed - movement speed at the previous sampling time point| / movement speed at the previous sampling time point. The preset ratio is, for example, 20%.

[0042] In this embodiment, the client status information at the current sampling time point is compared with the client status information at the previous sampling time point. If the deviation between the current movement orientation and the movement orientation at the previous sampling time point is less than a second preset angle, then the change in status information is less than a preset second threshold. The client status information at the current sampling time point is compared with the client status information at the previous sampling time point. If the current speed change ratio is less than a second preset ratio, then the change in status information is less than a preset second threshold. Here, the current speed change ratio = |current speed - movement speed at the previous sampling time point| / movement speed at the previous sampling time point.

[0043] In step S4, the other clients update their stored client status information corresponding to the received message based on their own set update step size and the received message, including:

[0044] Step S41: The other client has its own set update step size, which is obtained based on the other client's own current frame rate; after receiving the message, the other client parses the message to obtain the message content, which includes the current position POS, current orientation DIR, current speed SPEED, and current movement status flag of the client that sent the message.

[0045] Step S42: If the time interval between the other client and the last update of the client's status information corresponding to the message is less than or equal to the update step size, then update the status information of the client corresponding to the received message based on the message content.

[0046] In this embodiment, when other clients obtain a message for the first time, they set the time of the last update of the client's status information corresponding to the message to 0.

[0047] Furthermore, if the time interval between the last update of the status information of the client corresponding to the message and the other client is greater than the update step size, then the update of the status information of the client corresponding to the message is stopped.

[0048] Furthermore, when the other client parses the current movement status identifier FLAG from the message sent by the server and finds that the field value is "stop", it stops its own movement within a third preset time period.

[0049] Further, updating the client status information corresponding to the received message includes:

[0050] The client status information corresponding to the received message is updated based on the correction factor and the message content; the correction factor is used to simulate the user's acceleration, and the client status information corresponding to the received message is updated by superimposing the correction factor and the message content.

[0051] In this embodiment, a correction factor is added to simulate acceleration (acc, which can be positive or negative). For example, there is an acceleration process from standing to walking and from walking to running, and a deceleration process from running to walking and then to standing, v = v0 + at.

[0052] Further, updating the client's status information corresponding to the received message based on the message content includes:

[0053] Set a fourth preset time threshold, and complete the status update of the client corresponding to the message within the time limit of the fourth preset time threshold.

[0054] Furthermore, a fifth preset time threshold is set, and the fourth preset time threshold is less than the fifth preset time threshold; the fifth preset time threshold is the upper limit of the time interval T1 of the sent message.

[0055] For example, the maximum time interval for sending messages is the fifth preset time threshold, which is 500ms. If the movement parameters of virtual object A do not change (speed and orientation remain unchanged), then simulating the movement of virtual object A on other clients is relatively accurate. There is no need to report the current movement status to the server every T1 interval (e.g., 100ms).

[0056] Upon receiving the broadcast message from the server, other terminal devices immediately begin simulating movement. At this point, the time difference between the sending client A and the other client B simulating it is only a network delay. When other clients receive subsequent movement messages, they need to eliminate the current simulation error within a certain time using vector superposition. After receiving the movement stop message, other clients need to smoothly move to the destination position within a certain time to maintain synchronization with the server's position. Furthermore, based on user operating habits, the movement parameters do not change significantly in most cases, thereby reducing the frequency of movement protocol transmissions and lowering server performance pressure. Simultaneously, the movement simulation and error elimination by other terminal devices ensure the synchronization of client positions, improving the user experience.

[0057] This invention determines the time interval for sending network protocols based on the user's current input. If the user's movement in the virtual world does not involve significant changes in speed or direction, the maximum sending interval for the network protocol can be set to once every 500ms (this value is configurable; a common approach is to send the protocol to the server once every 100 to 200 milliseconds). This significantly reduces the frequency of network protocol transmission, lowers the amount of data transmitted over the network, and alleviates the pressure on the server.

[0058] This invention allows the client to simulate movement based on the current location, orientation, and speed of other users broadcast by the server. Even if there are network fluctuations or the client has not yet received subsequent movement messages, the client will not stop simulating, making location synchronization smoother and significantly reducing the problem of unsmooth movement caused by network fluctuations, thus improving the user experience.

[0059] The client uses its current position, orientation, and speed to simulate the movement of other users in virtual reality, which will inevitably introduce some error. To reduce this error, when the client receives the current position of other users forwarded by the server, it needs to eliminate the positional error through vector superposition within a certain time interval (e.g., 200ms) to ensure that the client's position is synchronized with the server's position.

[0060] Due to the uncertainty of network transmission, the maximum time for a single simulated movement after the client receives a message needs to be limited (e.g., 500ms). In other words, a single movement simulation will not exceed 500ms to avoid error problems caused by long-term simulation.

[0061] An embodiment is provided to illustrate a method for synchronizing the location of a virtual reality scene.

[0062] 1. The time interval for sending network protocols is determined based on the user's current input. If the user's movement in the virtual world does not involve significant changes in speed or direction, the maximum sending interval for the network protocol can be set to once every 500ms (this value is configurable; a common approach is to send the protocol to the server once every 100 to 200 milliseconds). This significantly reduces the frequency of network protocol transmission, lowers the amount of data transmitted over the network, and alleviates the pressure on the server.

[0063] 2. The client simulates movement based on the current location, orientation, and speed of other users broadcast by the server. Even if there are network fluctuations or the client has not yet received subsequent movement messages, the client will not stop simulating, making location synchronization smoother and greatly reducing the problem of unsmooth movement caused by network fluctuations, thus improving the user experience.

[0064] 3. The client uses its current location, orientation, and speed to simulate the movement of other users in the virtual world, which will introduce some error. To reduce this error, when the client receives the current location of other users forwarded by the server, it needs to eliminate the position error through vector superposition within a certain time interval (e.g., 200ms) to keep the client's position synchronized with the server's position.

[0065] 4. At the same time, due to the uncertainty of network transmission, the maximum time for a single simulated movement after the client receives the mobile protocol each time also needs to be limited (e.g., 500ms). That is to say, a single movement simulation will not exceed 500ms to avoid error problems caused by long-term simulation.

[0066] like Figure 3 As shown, the present invention provides a virtual reality scene position synchronization device. The device includes:

[0067] Connection module: Configured to define fields for sending messages between the client and the server to establish a communication connection between the client and the server; the client is installed on a virtual device, and each virtual device is configured with the client;

[0068] Period determination module: configured to determine the time interval T1 for sending messages between the client and the server, and set the client status sampling time interval T2; each client records the client status information corresponding to each virtual device at each status sampling time point;

[0069] Determining Module: Configured to, for any client, if the change in client status information at the current sampling time point exceeds a preset first threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is reduced; if the change in client status information at the current sampling time point is less than a preset second threshold compared to the change at the previous sampling time point, then the time interval T1 for sending messages is increased; each client sends a message including client status information to the server based on its corresponding time interval T1 for sending messages.

[0070] Simulation module: After the server receives a message sent by the client, it forwards the message to other clients; the other clients update the state information of the client corresponding to the received message stored in their own storage based on their own set update step size and the received message, simulating the virtual object of the user corresponding to the client of the received message.

[0071] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0072] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0073] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0075] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, a physical server, or a cloud server, etc., requiring the installation of a Windows or Windows Server operating system) to execute some steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for position synchronization of virtual reality scenes, characterized in that, The method comprises: Step S1: defining fields of messages sent between a client and a server to establish a communication connection between the client and the server; the client is installed on a virtual device, and each virtual device is configured with the client; Step S2: determining a time interval T1 of sending messages between the client and the server, and setting a client state sampling time interval T2; each client records the client state information corresponding to each state sampling time point of the virtual device where the client is located; Step S3: for any client, if the client state information corresponding to the current sampling time point is compared with the client state information corresponding to the last sampling time point, the state information changes more than a preset first threshold, the time interval T1 of sending messages is reduced; if the client state information corresponding to the current sampling time point is compared with the client state information corresponding to the last sampling time point, the state information changes less than a preset second threshold, the time interval T1 of sending messages is increased; each client sends a message including the client state information to the server based on the corresponding time interval T1 of sending messages; Step S4: after the server receives the message sent by the client, the message is sent to other clients; based on the update step length set by the other clients and the received message, the other clients update the state information of the client corresponding to the received message stored by the other clients, and simulate the virtual object corresponding to the user of the client corresponding to the received message; In the step S4, based on the update step length set by the other clients and the received message, the other clients update the state information of the client corresponding to the received message stored by the other clients, comprising: Step S41: the other clients have the update step length set by the other clients, and the update step length is obtained according to the current frame rate of the other clients; after the other clients receive the message, the other clients analyze the message to obtain the message content, and the message content includes the current position POS, the current motion direction DIR, the current speed SPEED and the current movement state identifier FLAG of the client sending the message; Step S42: if the time interval from the last time when the other clients update the state information of the client corresponding to the message is less than or equal to the update step length, the state information of the client corresponding to the received message is updated based on the message content; if the time interval from the last time when the other clients update the state information of the client corresponding to the message is greater than the update step length, the update of the state information of the client corresponding to the message is stopped; The update of the state information of the client corresponding to the received message comprises: updating the state information of the client corresponding to the received message based on a correction factor and the message content; the correction factor is used to simulate the acceleration of the user, and the state information of the client corresponding to the received message is updated by superimposing the correction factor and the message content.

2. The method of claim 1, wherein, When the other client parses the field value corresponding to the current mobile state identifier FLAG from the message sent by the server as stop, the other client stops its own movement within a third preset time length.

3. The method of claim 1, wherein, A fourth preset time threshold is set, and the state update of the client corresponding to the received message is completed within the time limit of the fourth preset time threshold.

4. The method of claim 3, wherein, A fifth preset time threshold is set, and the fourth preset time threshold is less than the fifth preset time threshold; the fifth preset time threshold is an upper limit of the time interval T1 of the sent message.

5. A virtual reality scene position synchronization apparatus, characterized by The device comprises: A connection module configured to define the fields of the messages sent between the client and the server to establish the communication connection between the client and the server; the client is installed on a virtual device, and each virtual device is configured with the client; A period determination module configured to determine the time interval T1 of the messages sent between the client and the server, and set a client state sampling time interval T2; each client records the client state information corresponding to each state sampling time point of the virtual device where the client is located; A determination module configured to, for any client, if the client state information corresponding to the current sampling time point is compared with the client state information corresponding to the last sampling time point, the state information changes more than a preset first threshold, the time interval T1 of the sent message is reduced; if the client state information corresponding to the current sampling time point is compared with the client state information corresponding to the last sampling time point, the state information changes less than a preset second threshold, the time interval T1 of the sent message is increased; each client sends a message including the client state information to the server based on the time interval T1 of the sent message corresponding to the client; An analog module configured to, after the server receives the message sent by the client, send the message to other clients; the other clients update the state information of the client corresponding to the received message stored by the other clients based on the update step length set by the other clients and the received message, and simulate the virtual object of the user corresponding to the client corresponding to the received message; The other clients update the state information of the client corresponding to the received message stored by the other clients based on the update step length set by the other clients and the received message, comprising: The other clients have an update step length set by the other clients, and the update step length is obtained according to the current frame rate of the other clients; after the other clients receive the message, the other clients parse the message to obtain the message content, and the message content includes the current position POS, the current movement direction DIR, the current speed SPEED and the current mobile state identifier FLAG of the client sending the message; If the time interval of the other clients from the last update of the state information of the client corresponding to the message is less than or equal to the update step length, the state information of the client corresponding to the received message is updated based on the message content; if the time interval of the other clients from the last update of the state information of the client corresponding to the message is greater than the update step length, the update of the state information of the client corresponding to the message is stopped. The updating of the state information of the client corresponding to the received message comprises: updating the state information of the client corresponding to the received message based on a correction factor and the message content; and the correction factor is used to simulate the acceleration of a user, and the state information of the client corresponding to the received message is updated by superimposing the correction factor and the message content.

6. An electronic device, comprising: The electronic device comprises: a processor configured to execute a plurality of instructions; a memory configured to store the plurality of instructions; wherein the plurality of instructions are configured to be stored in the memory, loaded and executed by the processor, and implement the method according to any one of claims 1-4.

7. A computer readable storage medium characterized by The storage medium stores a plurality of instructions; and the plurality of instructions are configured to be loaded and executed by the processor, and implement the method according to any one of claims 1-4.

Images