Virtual reality model-based method and cooperation system for multi-terminal communication, and terminal device

Abstract

A virtual reality model-based method for multi-terminal communication based on a, relating to the technical field of extended reality. In the method, a terminal device searches for a first communication address in a local area network; when the first communication address is not found, a second communication address is broadcasted by means of the local area network; a plurality of requests are received from a plurality of terminal devices; and a plurality of virtual reality models are sent to the plurality of terminal devices on the basis of the plurality of requests, wherein the plurality of terminal devices have one-to-one correspondence to the plurality of virtual reality models. By implementing the technical scheme provided by the present application, the problem of poor maintainability caused by binding the terminal devices in the local area network with one server can be solved, and the user experience is improved.

Description

Multi-terminal communication method, collaborative system and terminal device based on virtual reality model

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 25, 2023, with application number 202311246398.2 and invention name “A multi-terminal communication method and collaborative system based on virtual reality model”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of extended reality technology, and in particular to a multi-terminal communication method, collaborative system, and terminal device based on a virtual reality model. Background Art

[0003] Extended Reality (XR) combines the real and virtual through computers and wearable devices to create a virtual environment that allows for human-computer interaction. It is a general term for technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR). In terms of industry applications, AR driving, VR cycling, and VR education are among the most widely used in the current market.

[0004] Existing technologies for achieving multi-terminal collaborative parallel control rely on setting up a server to manage the virtual characters running on multiple terminals. However, if a server crashes, the virtual characters on any terminal connected to the server will be unable to run.

[0005] Application Contents

[0006] The present application provides a multi-terminal communication method, a collaborative system and a terminal device based on a virtual reality model.

[0007] In a first aspect, the present application provides a multi-terminal communication method based on a virtual reality model, which is applied to a terminal device, including: searching for a first communication address in a local area network; the first communication address is a communication address sent by other terminal devices in the local area network; when the first communication address is not searched and obtained, broadcasting a second communication address through the local area network; the second communication address is the communication address of the terminal device itself; receiving multiple virtual reality model requests from multiple terminal devices, each of the multiple requests is used to indicate the acquisition of a virtual reality model; based on the multiple virtual reality model requests, sending multiple virtual reality models to multiple terminal devices, and the multiple terminal devices and the multiple virtual reality models have a one-to-one correspondence.

[0008] By adopting the above technical solution, the terminal device broadcasts the second communication address through the local area network, so that other terminal devices in the local area network can establish communication with it based on the second communication address. That is, the terminal device that broadcasts the second communication address is equivalent to a server, which distributes virtual reality models to other terminal devices communicating with it, so that multiple terminal devices are in the same XR scene. Since the terminal device acting as a server can be any terminal device in the local area network, when the terminal device acting as a server goes down or fails, any terminal device in the local area network that has not failed can continue to act as a server to achieve communication between other terminal devices in the local area network, thereby achieving collaborative parallel control of multiple terminal devices, avoiding the problem of poor maintainability caused by all terminal devices in the local area network being bound to one server, and thus ensuring the user experience.

[0009] Optionally, the terminal device broadcasts the second communication address in the local area network, so that other terminal devices in the local area network establish communication with the terminal device through the second communication address after searching for the second communication address.

[0010] Optionally, after sending multiple virtual reality models to multiple terminal devices based on the multiple requests, the method also includes: receiving a first parameter from a first terminal device among the multiple terminal devices, the first parameter being used to indicate a state change of a first virtual reality model running in the first terminal device, the first virtual reality model being one of the multiple virtual reality models; updating the multiple virtual reality models based on the first parameter; and sending the updated multiple first virtual reality models to the corresponding multiple terminal devices respectively.

[0011] By adopting the above technical solution, the terminal device updates the first virtual reality model based on the first parameter, and updates the remaining virtual reality models at the same time, thereby achieving synchronous update of the virtual reality models corresponding to multiple terminal devices, thereby achieving the purpose of user experience of multi-terminal communication collaboration.

[0012] Optionally, after searching for the first communication address in the local area network, the method further includes: when the first communication address is searched, sending a request to obtain a virtual reality model to the first communication address; obtaining the first virtual reality model from the first communication address; sending model parameters to the first communication address, where the model parameters are used to indicate a change in the state of the first virtual reality model; and obtaining an updated first virtual reality model from the first communication address.

[0013] By adopting the above technical solution, after searching for the first communication address, the terminal device obtains the first virtual reality model through the first communication address, and sends model parameters and receives the updated first virtual reality model through the first communication address to achieve communication with other terminal devices, avoiding real-time communication with other terminal devices through a fixed server, and improving the practicality of the terminal device.

[0014] Optionally, when the first communication address is not obtained, the method further includes: receiving instruction information from a user to obtain a virtual reality model; generating a second virtual reality model other than the multiple virtual reality models based on the instruction information; detecting whether parameters of the second virtual reality model have changed; and updating the second virtual reality model when the parameters of the second virtual reality model have changed.

[0015] By adopting the above technical solution, in addition to receiving multiple requests, the terminal device also responds to the corresponding user to obtain instruction information of the virtual reality model, which can improve the utilization rate of the terminal device.

[0016] Optionally, after generating a second virtual reality model outside the multiple virtual reality models, the method further includes: transmitting the second virtual reality model to the connected peripheral device through the peripheral device interface to present the second virtual reality model in the peripheral device.

[0017] By adopting the above technical solution, the terminal device can support more types of peripheral devices through the peripheral device interface, allowing users to experience more types of virtual reality models, thereby greatly improving the user experience.

[0018] Optionally, receiving instruction information from a user to obtain a virtual reality model includes: displaying optional virtual reality models based on the type of the peripheral device to instruct the user to select a virtual reality model.

[0019] By adopting the above technical solution, the terminal device displays an optional virtual reality model based on the peripheral device and instructs the user to send a virtual reality model request, which can avoid the user selecting a virtual reality model that cannot be displayed by the peripheral device corresponding to the terminal device, resulting in the user being unable to smoothly experience the XR project.

[0020] Optionally, the second communication address is sent via a local area network based on a broadcast communication protocol, and the multiple requests are received based on a point-to-point communication transmission protocol. By adopting the above technical solution, the terminal device broadcasts the second communication address based on the broadcast communication protocol, allowing multiple terminal devices to establish communication with the terminal device respectively, thereby improving the speed of establishing communication with multiple terminal devices. Furthermore, multiple requests sent by multiple terminal devices are received via the point-to-point communication transmission protocol to ensure the stability of receiving multiple requests, thereby improving the speed of establishing communication with multiple terminal devices and the stability of information transmission.

[0021] In a second aspect of the present application, a multi-terminal communication collaboration system is provided, including: a search module for searching for a first communication address in a local area network; the first communication address is a communication address sent by other terminal devices in the local area network; a broadcast module for broadcasting a second communication address through the local area network when the first communication address is not found; the second communication address is the communication address of the terminal device itself; a receiving module for receiving multiple requests from multiple terminal devices, each of the multiple requests is used to indicate the acquisition of a virtual reality model; a sending module for sending multiple virtual reality models to multiple terminal devices based on the multiple requests, and the multiple terminal devices and the multiple virtual reality models have a one-to-one correspondence.

[0022] In the third aspect of the present application, a terminal device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the terminal device executes the method as described in any one of the first aspects.

[0023] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, the method steps described in any one of the first aspects are executed.

[0024] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1. Since the communication address can be sent through any of the multiple terminal devices to connect to other terminal devices in the local area network to achieve collaborative parallel control of multiple terminal devices, it avoids the problem of poor maintainability caused by the terminal devices in the local area network being bound to one server, thereby ensuring the user experience.

[0026] 2. Since the terminal device also includes a peripheral device interface, it can identify multiple types of peripheral devices, which can facilitate users to experience more virtual reality models.

[0027] 3. When customers corresponding to multiple terminal devices choose to experience the same virtual reality model, the terminal devices can independently update the virtual reality models corresponding to the multiple terminal devices, so that multiple terminal devices can experience the same virtual reality model at the same time, thereby further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1 is a schematic architecture diagram of a collaborative system for multi-terminal communication based on a virtual reality model provided in an embodiment of the present application.

[0029] FIG2 is a flow chart of a multi-terminal communication method based on a virtual reality model disclosed in an embodiment of the present application.

[0030] FIG3 is a schematic diagram of the structure of a multi-terminal communication collaboration system disclosed in an embodiment of the present application.

[0031] FIG4 is a schematic structural diagram of a terminal device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0033] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0034] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0035] During the implementation of this application, the inventors of the present application discovered that in order to enable multiple users to interact in the same space at the same time in the application of XR technology, multiple computers can establish communication with a fixed server, and then the selected virtual character can be locked, and the locked virtual character can be stored in the server. When a user needs to operate a virtual object, a virtual character operation request is first sent to the server through a computer. After receiving the virtual character operation request, the server first checks whether the virtual object requested to be operated is locked in the locked virtual characters stored in the server; if it is not locked, the server will first add a lock record of the object to the object lock table, and then send an "operable" response to the computer; if the virtual character requested to be operated is locked, a "not operable" response is sent to the computer to prevent multiple computers participating in the collaboration from operating the same virtual object at the same time and causing operation conflicts, thereby realizing the collaborative parallel control function of multiple computers.

[0036] The implementation of this solution is highly dependent on the server. In actual application, if the server fails, users will be unable to operate the virtual character, affecting the user experience.

[0037] In order to ensure the user experience, this application proposes a collaborative system for multi-terminal communication based on a virtual reality model, which can solve the problem that the implementation of XR projects has a high degree of dependence on servers, resulting in low maintainability of XR projects and poor user experience.

[0038] It should be understood that the "peripheral device" in the embodiment of the present application is a device that has the ability to display a virtual reality model to the user. For example, it can be but is not limited to VR fitness equipment, VR e-sports equipment, AR racing equipment or handle control equipment. The peripheral device can be wearable or non-wearable equipment, and the embodiment of the present application does not impose any restrictions on this.

[0039] It should also be understood that the above-mentioned computer for connecting peripheral devices can be any terminal device with a connection to peripheral devices, such as a mobile phone, tablet computer, laptop computer, PDA, computing device or other device, and the embodiments of the present application do not impose any limitations on this.

[0040] Figure 1 shows a schematic diagram of the architecture of a collaborative system for multi-terminal communication based on a virtual reality model applicable to an embodiment of the present application. Figure 1 schematically shows three terminal devices and three peripheral devices, and the terminal devices and the peripheral devices have a one-to-one correspondence. It can be understood that the collaborative system 100 shown in the embodiment of the present application may include more or fewer terminal devices and peripheral devices. The embodiment of the present application does not specifically limit the number of terminal devices and peripheral devices, and is set based on the needs of the application scenario. The following description takes the collaborative system 100 including three terminal devices and three peripheral devices as an example.

[0041] The terminal device 110 can be connected to the peripheral device 101 in a wired manner or a wireless manner, and the terminal device 110 can detect the device type of the peripheral device 101. Optionally, the terminal device 110 communicates with the terminal device 120 and the terminal device 130 in a wireless manner.

[0042] Terminal device 110: The terminal device 110 is used to display optional virtual reality models to the user according to the type of the peripheral device 101, and generate a corresponding virtual reality model according to the virtual reality model selected by the user, so as to send the virtual reality model to the peripheral device 101.

[0043] Peripheral device 101: The peripheral device 101 is used to display the virtual reality model generated by the terminal device 110 to the user.

[0044] The purpose and connection method of the terminal device 120 and the peripheral device 102 are as described above. In addition, the terminal device 120 is connected to the terminal device 110 in a wireless manner.

[0045] The purpose and connection method of the terminal device 130 and the peripheral device 103 are as described above. In addition, the terminal device 130 is connected to the terminal device 110 in a wireless manner.

[0046] In the collaborative system for multi-terminal communication based on virtual reality models provided in an embodiment of the present application, terminal device 110 establishes a wired connection with peripheral device 101. Terminal device 110 searches for a communication address within the local area network. If neither terminal device 120 nor terminal device 130 broadcasts a communication address, terminal device 110 cannot communicate with terminal devices 120 and 130. In this case, terminal device 110 broadcasts its own communication address within the local area network, enabling terminal devices 120 and 130 to establish wireless communication with itself based on the broadcasted communication address. Terminal device 110 then receives multiple requests from terminal devices 120 and 130 in real time and then sends virtual reality models to terminal devices 120 and 130, respectively, based on the multiple requests. Furthermore, terminal device 110 can also display selectable virtual reality models to the user based on the type of connected peripheral device 101, instructing the user to select a virtual reality model.

[0047] In other scenarios, if any one of the terminal device 120 and the terminal device 130 has sent a communication address, the terminal device 110 establishes wireless communication with the terminal device 120 or the terminal device 130 corresponding to the communication address based on the searched communication address. The terminal device 110 displays the selectable virtual reality model to the user by determining the type of the connected peripheral device 101, and sends a request to the terminal device 120 or the terminal device 130 to obtain the virtual reality model according to the virtual reality model selected by the user, and receives the generated virtual reality model.

[0048] Below, a multi-terminal communication method based on a virtual reality model according to an embodiment of the present application is described in detail with reference to FIG1 . The multi-terminal communication method based on a virtual reality model provided in an embodiment of the present application may be performed by terminal device 101, terminal device 102, or terminal device 103 as shown in FIG1 .

[0049] 2 , a multi-terminal communication method 200 based on a virtual reality model provided in an embodiment of the present application is described in detail.

[0050] Step S201: Searching for a first communication address in a local area network.

[0051] The first communication address can be understood as a communication address sent by any terminal device other than the terminal device itself within the local area network.

[0052] It should be understood that the first communication address may be an IP address or other address that can establish communication, and this application does not impose any limitation on this.

[0053] Step S202: When the first communication address is not found, broadcast the second communication address via the local area network.

[0054] The second communication address can be understood as the communication address of the terminal device itself.

[0055] For example, when the terminal device fails to retrieve the first communication address, it can broadcast the second communication address in the local area network so that other terminal devices in the local area network can search for the second communication address sent by the terminal device, and then establish communication with the terminal device through the second communication address.

[0056] Step S203: receiving multiple requests from multiple terminal devices, where each of the multiple requests is used to instruct to obtain a virtual reality model.

[0057] A virtual reality model can be understood as a model that includes a virtual character and a virtual scene corresponding to the virtual character.

[0058] In the embodiments of the present application, a user selects a virtual reality model on a terminal device based on the virtual character they wish to experience, and the terminal device generates the aforementioned request in response to the user's selected virtual reality model. Multiple requests can be understood as multiple users selecting a virtual reality model on their corresponding terminal devices based on the virtual character they wish to experience, and the multiple terminal devices generating multiple requests in response to the user's selection.

[0059] In the embodiment of the present application, any number of requests may be instructing to obtain the same virtual reality model. For example, there are 10 requests for model A, model B, and model C, of ​​which 3 requests are instructing to obtain model A, 6 requests are instructing to obtain model B, and 1 request is instructing to obtain model C.

[0060] It's easy to understand that when the requests are the same, the corresponding avatar and the virtual scene corresponding to the avatar are also the same. However, when the terminal device generates the VR model, it does so based on the user's state (e.g., standing position) and the request sent by the user. If the VR model request sent by the corresponding user of the terminal device is the same but the state is different, the VR model generated by the terminal device will be different.

[0061] It should be understood that in the embodiments of the present application, there is a one-to-one correspondence between multiple requests, multiple terminal devices, and multiple virtual reality models. For example, the 10 requests mentioned above can be sent by 10 terminals. If there are 10 requests in total, the terminal device generates 10 virtual reality models based on the 10 requests, and any number of the 10 requests can be the same. These 10 virtual reality models are then sent to the 10 terminal devices mentioned above. In this embodiment, users can select the same virtual reality model, avoiding selection conflicts caused by multiple users selecting the same virtual reality model, thereby ensuring that the user experience of multiple users can be satisfied at the same time.

[0062] In actual applications, users can experience virtual reality models through peripheral devices.

[0063] In an optional implementation of the embodiment of the present application, the terminal device transmits the second virtual reality model to the connected peripheral device through the peripheral device interface, so as to present the second virtual reality model in the peripheral device.

[0064] It is easy to understand that the peripheral device interface is an interface used to identify the type of peripheral device connected to the terminal device. For example, the peripheral device interface can be an OpenXR interface, an NWH interface, or a peripheral device interface including both OpenXR and NWH interfaces.

[0065] In the implementation, the terminal device includes two interfaces, an OpenXR interface and an NWH interface. Depending on the type of the connected peripheral device, the second virtual reality model is transmitted to the peripheral device through one of the OpenXR interface and the NWH interface, so that the peripheral device can display the second virtual reality model to the user.

[0066] In this example, the terminal device includes two interfaces: OpenXR interface and NWH interface, which can support more types of peripheral devices, so that users can experience more types of virtual reality models, greatly improving the user experience.

[0067] It's easy to understand that different types of peripheral devices can display different virtual reality models. For example, the virtual models that can be selected when the peripheral device is a VR fitness device are different from the virtual models that can be selected when the peripheral device is an AR racing device.

[0068] In one example, the terminal device receives instruction information from the user to obtain a virtual reality model, and may display optional virtual reality models based on the type of the peripheral device to instruct the user to select a virtual reality model.

[0069] The terminal device can determine, based on the type of the peripheral device, that the peripheral device can display a virtual reality model, and then display options corresponding to the virtual reality models that the peripheral device can display to the user, instructing the user to select a virtual reality model that needs to be experienced from multiple options.

[0070] In this example, the terminal device displays optional virtual reality models based on the peripheral device and instructs the user to send a virtual reality model request. This can prevent the user from selecting a virtual reality model that cannot be displayed by the peripheral device corresponding to the terminal device, resulting in the user being unable to smoothly experience the XR project.

[0071] It should be understood that the terminal device may also include more types of interfaces to support more types of peripheral devices, and this application does not impose any limitation on this.

[0072] In another example, the terminal device may display options corresponding to all virtual reality models. When the user selects a virtual reality model that cannot be displayed by the connected peripheral device, the terminal device displays "This model is not selectable" to instruct the user to complete the selection of the virtual reality model.

[0073] In an embodiment of the present application, before generating multiple requests, multiple terminal devices display optional virtual reality models to the user and instruct the user to select the virtual reality model that needs to be experienced. The method is the same as the method in which the above-mentioned terminal device instructs the user to select the virtual reality model that needs to be experienced, and will not be repeated here.

[0074] Step S204: Based on the multiple requests, multiple virtual reality models are sent to multiple terminal devices, and the multiple terminal devices and the multiple virtual reality models have a one-to-one correspondence.

[0075] Exemplarily, after receiving multiple requests from multiple terminal devices, the terminal device generates multiple virtual reality models based on the virtual reality models obtained by each request indication in the multiple requests, and then sends the virtual reality models to the multiple terminal devices respectively based on the correspondence between the multiple requests and the multiple terminal devices, so that the multiple terminal devices obtain the virtual reality models corresponding to them.

[0076] An embodiment of the present application provides a multi-terminal communication method based on a virtual reality model, in which a terminal device searches for a first communication address in a local area network. When the first communication address is not found, the terminal device sends its corresponding second communication address through the local area network, so that multiple terminal devices can establish communication with it based on the second communication address, thereby receiving multiple requests from multiple terminal devices, and sending multiple virtual reality models to multiple terminal devices based on the multiple requests, so that the terminal device and multiple terminal devices are in the same XR scene. Since the terminal device can be any terminal device in the local area network that has not searched for the first communication address and establishes communication with multiple terminal devices by sending the second communication address, when the terminal device is down or fails, any one of the multiple terminal devices can be used to connect to other terminal devices in the local area network to achieve collaborative parallel control of multiple terminal devices, thereby avoiding the problem of poor maintainability caused by the terminal devices in the local area network being bound to one server, thereby ensuring the user experience.

[0077] In some embodiments, after sending multiple virtual reality models to multiple terminal devices based on multiple requests, the terminal device receives a first parameter from a first terminal device among the multiple terminal devices, where the first parameter is used to indicate a state change of a first virtual reality model running in the first terminal device, where the first virtual reality model is one of the multiple virtual reality models; based on the first parameter, the multiple virtual reality models are updated; and the updated multiple virtual reality models are respectively sent to the corresponding terminal devices.

[0078] The first terminal device is used to represent a terminal device whose virtual reality model state has changed among the multiple terminal devices. It should be understood that any number of the multiple terminal devices may correspond to virtual reality model state changes.

[0079] It is easy to understand that the multiple terminal devices include a first terminal device and other terminal devices except the first terminal device. The first virtual reality model belongs to the virtual scene in the other virtual reality models corresponding to the other terminal devices. When the first virtual reality model is updated, the terminal device will simultaneously update the other virtual reality models to achieve synchronization of the virtual reality models of multiple terminal devices.

[0080] Taking the case where only the first terminal device among multiple terminal devices changes its state as an example, the first virtual reality model sent by the terminal device to the first terminal device is a standing person. The virtual character in the first virtual reality model is in a stationary standing state, and the virtual scene corresponding to the character is the scenery seen in the stationary standing state. After the first terminal device receives the first virtual reality model, the user walks to the right, and a first parameter is generated at this time. The first parameter is the virtual character walking to the right. The terminal device updates the first virtual reality model based on the first parameter, updating the virtual character from a stationary state to a virtual character walking to the right. The virtual scene corresponding to the character is also the scenery seen in the walking state. At the same time, the virtual scenes in the remaining virtual reality models change, and the terminal device simultaneously updates the remaining virtual reality models. Finally, the terminal device sends the updated first virtual reality model to the first terminal device, and the updated remaining virtual reality models are sent to the remaining terminal devices.

[0081] In this embodiment, the terminal device updates the first virtual reality model based on the first parameter, and updates the remaining virtual reality models at the same time, thereby achieving synchronous update of virtual reality models corresponding to multiple terminal devices, thereby achieving the purpose of user experience of multi-terminal communication collaboration.

[0082] In other embodiments, before the terminal device updates the first virtual reality model and the remaining virtual reality models according to the first parameter, any number of the remaining terminal devices can send an instruction to the terminal device to "not update synchronously with the first terminal device". After receiving the instruction, the terminal device does not update the terminal device that sent the information according to the first parameter.

[0083] The above example uses the terminal device receiving multiple requests from multiple terminal devices. The following describes how the terminal device generates a request in response to a virtual reality model selected by a user corresponding to the terminal device while receiving multiple requests.

[0084] Exemplarily, the terminal device is further configured to receive instruction information from a user to obtain a virtual reality model; generate a second virtual reality model other than the multiple virtual reality models based on the instruction information; detect whether parameters of the second virtual reality model have changed; and update the second virtual reality model when parameters of the second virtual reality model have changed.

[0085] The instruction information is used to instruct the terminal device to respond to the user's selection and generate a virtual reality model to represent the virtual reality model that needs to be obtained.

[0086] It is easy to understand that the second virtual reality model is the virtual reality model selected and experienced by the user corresponding to the terminal device.

[0087] In this embodiment, in addition to receiving multiple requests, the terminal device also responds to the corresponding user's instruction information to obtain the virtual reality model, which can improve the utilization rate of the terminal device.

[0088] As described above, when the corresponding users in multiple terminal devices have the same virtual reality model as that selected by the user corresponding to the terminal device, the terminal device will independently update its corresponding virtual reality model and synchronously update its own virtual reality model and the virtual reality models corresponding to other terminal devices.

[0089] Taking the terminal device 101, terminal device 102 and terminal device 103 shown in Figure 1 as an example, a multi-terminal communication method based on a virtual reality model provided in an embodiment of the present application is described in more detail through specific scenarios.

[0090] In this example, the XR project provides a street scene, which includes a virtual scene (a building), character 1 (a pedestrian standing on the left side of the building), and character 2 (a driver driving a car). Terminal device 101 establishes communication with terminal device 102 and terminal device 103 by broadcasting a second communication address. The corresponding user selects the first virtual reality model, which includes a virtual character (character 1) and a virtual scene (building, character 2); the user corresponding to terminal device 102 selects the second virtual reality model, which includes a virtual character (character 2) and a virtual scene (building, character 1); the user corresponding to terminal device 103 selects the third virtual model, which includes a virtual character (character 1) and a virtual scene (building, character 2). As can be seen from the above, the virtual reality models obtained by terminal device 101 and terminal device 103 are the same.

[0091] The terminal device 101 generates a first virtual reality model, a second virtual reality model, and a third virtual reality model in response to the selections of the users corresponding to the terminal devices 101, 102, and 103, and sends the second virtual reality model and the third virtual reality model to the terminal devices 102 and 103 respectively. The terminal devices 101, 102, and 103 respectively display the virtual reality models selected by the users through the peripheral devices. The users corresponding to the terminal devices 101 and 103 see themselves standing on the left side of the building, and there is a driver driving on the road. The user corresponding to device 102 sees a scene in which he or she is driving a car, and a pedestrian is standing on the left side of the building. Each user performs an action, among which character 1 in the first virtual reality model changes from standing on the left side of the building to walking to the right side of the building. The user state corresponding to the first virtual reality model changes, generating a first parameter. Character 2 in the second virtual reality model changes from driving to parking and getting out of the car. The user state corresponding to the second virtual reality model changes, generating a second parameter. Character 1 in the third virtual reality model changes from standing on the left side of the building to walking away from the left side of the building. The user state corresponding to the third virtual reality model changes, generating a third parameter. At this time, the terminal device updates the first, second, and third virtual reality models by acquiring the above first, second, and third parameters, and sends the updated first virtual reality model to terminal device 102 and the updated third virtual reality model to terminal device 103. The updated first virtual reality model displayed to the user by terminal device 101 is: character 1 walking from the left side of the building to the right side. The virtual scene from character 1's perspective is a scene of walking from the left side of the building to the right side of the building. The state of character 2 included in the virtual scene changes from driving to parking. and gets off the car; the updated second virtual reality model displayed to the user by the terminal device 102 is as follows: the character 2 changes from the driving state to the parking state and gets off the car, the virtual scene from the perspective of the character 2 changes from the driving state to the parking state and gets off the car, and the character 1 included in the virtual scene changes from the standing state on the left side of the building to the state of walking to the right side of the building; the updated third virtual reality model displayed to the user by the terminal device 103 is as follows: the character 1 changes from standing on the left side of the building to walking away from the building to the left side, the virtual scene from the perspective of the character 1 is a scene of walking away from the building to the left side, and the state of the character 2 included in the virtual scene remains driving.

[0092] In implementation, after the terminal device and multiple terminal devices generate a request, they can also display "Complete Selection" and "Reselect" to the user to instruct the user to click "Complete Selection" when not changing the selection, indicating that they have entered the preparation state, and click "Reselect" when changes are needed, indicating that the virtual reality model needs to be reselected. After the terminal device obtains the "Complete Selection" information sent by multiple terminal devices and the corresponding user selects "Complete Selection", it starts to generate the virtual reality model according to the generated request, so as to avoid the user selecting the wrong virtual reality model and being unable to modify it in time, affecting the user's usage experience.

[0093] It is easy to understand that after receiving the request, the terminal device can also send the virtual reality model to the terminal device that sent the request at an interval of 10 seconds to avoid the user being unable to reselect the virtual reality model to experience.

[0094] The above are the steps performed by the terminal device when the terminal device fails to search for the first communication address. The following are the steps performed when the terminal device searches for the first communication address.

[0095] In some embodiments, after the terminal device obtains the first communication address based on the local area network, it searches for the first communication address and sends a request to the first communication address to obtain the virtual reality model; obtains the first virtual reality model from the first communication address; sends model parameters to the first communication address, where the model parameters are used to indicate a change in the state of the first virtual reality model; and obtains the updated first virtual reality model from the first communication address.

[0096] It should be understood that when the terminal device searches for the first communication address, the first virtual reality model is used to represent the virtual reality model generated according to the request corresponding to the terminal device; the model parameters are used to represent the model parameters generated due to the change in the status of the user corresponding to the terminal device; the updated first virtual reality model is used to represent the first virtual reality model generated by the terminal device corresponding to the first communication address according to the model parameters, and the first virtual reality model generated according to other model parameters sent by other terminals in the local area network.

[0097] Similar to the above-mentioned method of synchronizing virtual reality models between a terminal device and multiple terminal devices, the terminal device sends a request to obtain the virtual reality model to the first communication address in response to the virtual reality model selected by the user, obtains the first virtual reality model from the first communication address, and then sends model parameters generated according to the user's state change to the first communication address to obtain an updated first virtual reality model generated according to the model parameters; when the first virtual reality model has not changed, the terminal device receives the first virtual reality model updated according to the model parameters and other model parameters from the first communication address.

[0098] In this embodiment, after searching for the first communication address, the terminal device obtains the first virtual reality model through the first communication address, and sends model parameters and receives the updated first virtual reality model through the first communication address to achieve communication with other terminal devices, avoiding real-time communication with other terminal devices through a fixed server, and improving the practicality of the terminal device.

[0099] In other embodiments, after the terminal device obtains the first communication address based on the local area network, if the first communication address is not found, the terminal device instructs the staff or user to enter the communication address of other terminal devices in the local area network to interact with other terminal devices through the communication address.

[0100] Optionally, the terminal device sends the second communication address in the local area network based on a broadcast communication protocol, and receives multiple requests based on a point-to-point communication transmission protocol.

[0101] Exemplarily, the second communication address is based on the UDP protocol and is sent through the local area network, and the multiple requests are received based on the KCP protocol.

[0102] It should be understood that the terminal device can receive multiple requests based on the KCP protocol, and can also send virtual reality models or other information based on the KCP protocol.

[0103] In an embodiment of the present application, the terminal device broadcasts a second communication address in the local area network based on the UDP protocol, so that multiple terminal devices in the local area network can respectively receive the second communication address sent by themselves, thereby establishing communication with multiple terminal devices. Furthermore, the terminal device receives multiple requests sent by multiple terminal devices based on the KCP protocol.

[0104] In this embodiment, the terminal device broadcasts a second communication address based on a broadcast communication protocol, so that multiple terminal devices can establish communication with the terminal device respectively, so as to improve the speed of establishing communication with multiple terminal devices. Furthermore, multiple requests sent by multiple terminal devices are received through a point-to-point communication transmission protocol to ensure the stability of receiving multiple requests, thereby improving the speed of establishing communication with multiple terminal devices and the stability of information transmission.

[0105] It should be understood that when a terminal device establishes communication with multiple terminal devices, the communication can also be established based on the TCP protocol, and this application does not impose any restrictions on this.

[0106] It is understandable that, in order to implement the functions described in FIG2 , the terminal device includes hardware and / or software modules corresponding to the execution of each function. In combination with the steps of the various examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0107] In this embodiment, the terminal device can be divided into functional modules according to the above-mentioned method example. For example, different functional modules can be divided according to different functions, or two or more functions can be integrated into a single processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division. In actual implementation, other division methods may be used.

[0108] In the case of dividing each functional module according to each function, Figure 3 shows a possible schematic diagram of the multi-terminal communication device 300 involved in the above embodiment, and the multi-terminal communication device 300 includes: a search module 301, which is used to search for a first communication address in a local area network; a broadcast module 302, which is used to broadcast a second communication address through the local area network when the first communication address is not found; a receiving module 303, which is used to receive multiple requests from multiple terminal devices, each of the multiple requests is used to indicate the acquisition of a virtual reality model; and a sending module 304, which is used to send multiple virtual reality models to multiple terminal devices based on the multiple requests, and the multiple terminal devices and the multiple virtual reality models have a one-to-one correspondence.

[0109] In an optional implementation of the embodiment of the present application, after sending multiple virtual reality models to multiple terminal devices based on multiple requests, the sending module 304 is further used to: receive a first parameter from a first terminal device among the multiple terminal devices, the first parameter being used to indicate a state change of a first virtual reality model running in the first terminal device, the first virtual reality model being one of the multiple virtual reality models; update the multiple virtual reality models based on the first parameter; and send the updated multiple virtual reality models to the corresponding multiple terminal devices respectively.

[0110] In an optional implementation of the embodiment of the present application, after searching the local area network for the first communication address, the search module 301 is further configured to: when the first communication address is found, send a request to the first communication address to obtain a virtual reality model; obtain the first virtual reality model from the first communication address; send model parameters to the first communication address, where the model parameters are used to indicate a change in the state of the first virtual reality model; and obtain the updated first virtual reality model from the first communication address.

[0111] In an optional implementation of the embodiment of the present application, when the first communication address is not obtained, the receiving module 303 is further configured to: receive instruction information from a user instructing to obtain a virtual reality model; generate a second virtual reality model other than the multiple virtual reality models based on the instruction information; detect whether parameters of the second virtual reality model have changed; and update the second virtual reality model when the parameters of the second virtual reality model have changed.

[0112] In an optional implementation of the embodiment of the present application, after generating a second virtual reality model other than the multiple virtual reality models, the sending module 304 is further configured to transmit the second virtual reality model to the connected peripheral device through the peripheral device interface, so as to present the second virtual reality model in the peripheral device.

[0113] In an optional implementation of the embodiment of the present application, the receiving module 303 receives instruction information from the user to obtain a virtual reality model, and is specifically configured to: display optional virtual reality models based on the type of the peripheral device to instruct the user to select a virtual reality model.

[0114] In an optional implementation of the embodiment of the present application, the second communication address is sent through the local area network based on a broadcast communication protocol; and the multiple requests are received based on a point-to-point communication transmission protocol.

[0115] This application also discloses a terminal device. Referring to FIG4 , FIG4 is a schematic diagram of the structure of a terminal device disclosed in an embodiment of this application. The terminal device 400 may include: at least one processor 401 , at least one network interface 404 , a user interface 403 , a memory 405 , and at least one communication bus 402 .

[0116] The communication bus 402 is used to implement the connection and communication between these components.

[0117] The user interface 403 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0118] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0119] The processor 401 may include one or more processing cores. The processor 401 utilizes various interfaces and circuits to connect various components within the server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 405, and by accessing data stored in the memory 405, the processor 401 performs various server functions and processes data. Optionally, the processor 401 may be implemented using at least one hardware form selected from the group consisting of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 401 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is responsible for handling wireless communications. It is understood that the modem may not be integrated into the processor 401 and may be implemented separately on a single chip.

[0120] Among them, the memory 405 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 405 may also be at least one storage device located away from the aforementioned processor 401. Referring to Figure 4, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application for generating a station area topology structure.

[0121] In the terminal device 400 shown in Figure 4, the user interface 403 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 401 can be used to call an application program for generating a station area topology structure stored in the memory 405. When executed by one or more processors 401, the terminal device 400 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the described order of actions, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0122] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0124] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0125] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0127] The foregoing is merely an exemplary embodiment of the present disclosure and is not intended to limit the scope of the present disclosure. In other words, any equivalent variations and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the disclosure and the practical implications thereof.

[0128] This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not described herein. The description and examples are to be considered merely as exemplary, and the scope and spirit of the present disclosure are to be determined by the claims.

Claims

1. A multi-terminal communication method based on a virtual reality model, characterized in that: Applied to terminal equipment, including: Searching for a first communication address in the local area network; the first communication address is a communication address sent by other terminal devices in the local area network; When the first communication address is not found, broadcasting a second communication address through the local area network; the second communication address is the communication address of the terminal device itself; receiving a plurality of requests from a plurality of terminal devices, each of the plurality of requests being used to instruct obtaining a virtual reality model; Based on the multiple requests, multiple virtual reality models are sent to the multiple terminal devices, and the multiple terminal devices have a one-to-one correspondence with the multiple virtual reality models.

2. The method according to claim 1, characterized in that The terminal device broadcasts the second communication address in the local area network, so that other terminal devices in the local area network establish communication with the terminal device through the second communication address after searching for the second communication address.

3. The method according to claim 1, characterized in that After sending the multiple virtual reality models to the multiple terminal devices based on the multiple requests, the method further includes: receiving a first parameter from a first terminal device among the plurality of terminal devices, where the first parameter is used to indicate a state change of a first virtual reality model running in the first terminal device, where the first virtual reality model is one of the plurality of virtual reality models; Based on the first parameter, updating the multiple virtual reality models; The updated multiple virtual reality models are respectively sent to the corresponding multiple terminal devices.

4. The method according to claim 1, characterized in that: After searching the local area network for the first communication address, the method further includes: When the first communication address is searched, sending a request for obtaining the virtual reality model to the first communication address; Obtaining a first virtual reality model from the first communication address; Sending a model parameter to the first communication address, where the model parameter is used to indicate a change in the state of the first virtual reality model; An updated first virtual reality model is obtained from the first communication address.

5. The method according to claim 1 or 3, characterized in that: When the first communication address is not obtained, the method further includes: Receiving instruction information from a user to obtain a virtual reality model; Based on the indication information, generate a second virtual reality model other than the multiple virtual reality models; detecting whether a parameter of the second virtual reality model changes; When the parameters of the second virtual reality model change, the second virtual reality model is updated.

6. The method according to claim 5, characterized in that After generating a second virtual reality model other than the plurality of virtual reality models, the method further includes: The second virtual reality model is transmitted to the connected peripheral device through the peripheral device interface to present the second virtual reality model in the peripheral device.

7. The method according to claim 6, characterized in that The receiving instruction information of the user for obtaining the virtual reality model includes: The indication information is received from the peripheral device, wherein the indication information is generated by the user's selection of multiple virtual reality models presented in the peripheral device.

8. The method according to claim 1, characterized in that: The second communication address is sent through the local area network based on a broadcast communication protocol; The multiple requests are received based on a point-to-point communication transmission protocol.

9. A multi-terminal communication collaboration system, characterized in that: include: A search module, the search module is used to search for a first communication address in the local area network; The first communication address is a communication address sent by other terminal devices in the local area network; A broadcast module, wherein the broadcast module is used to broadcast a second communication address through the local area network when the first communication address is not searched; the second communication address is the communication address of the terminal device itself; A receiving module, the receiving module is used to receive multiple requests from multiple terminal devices, each of the multiple requests is used to instruct to obtain a virtual reality model; A sending module, wherein the sending module is used to send a plurality of virtual reality models to the plurality of terminal devices based on the plurality of requests, and the plurality of terminal devices and the plurality of virtual reality models have a one-to-one correspondence.

10. A terminal device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the terminal device executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method steps according to any one of claims 1 to 8 are performed.

Images