Space sharing method and system

Abstract

The application provides a space sharing method and system, the space sharing method comprises the following steps: a mixed reality head-mounted display terminal identifies a picture displayed by a terminal with AR simulation function, to obtain object space information of the picture on the mixed reality head-mounted display terminal, and forwards the object space information; after the terminal with AR simulation function receives the object space information of the picture on the mixed reality head-mounted display terminal, if it is determined that the object space information is inconsistent with object space information of the picture on the terminal with AR simulation function, the object space information of the picture on the terminal with AR simulation function is adjusted; the mixed reality head-mounted display terminal captures change information of the object space information of the picture on the mixed reality head-mounted display terminal, and synchronizes the change information to the terminal with AR simulation function in real time. The application can realize third-view observation and shooting functions without adding HoloLens devices, and reduces the cost of space sharing.

Description

Technical Field

[0001] This invention belongs to the field of virtual reality technology, and relates to a method and system, particularly a spatial sharing method and system. Background Technology

[0002] Mixed Reality (MR) technology is a further development of Virtual Reality (VR) technology. It enhances the realism of the user experience by presenting virtual scene information within a real-world environment, creating an interactive feedback loop between the real world, the virtual world, and the user. HoloLens, developed by Microsoft, is an MR headset, a mixed reality head-mounted display terminal, but a third party cannot observe the scene seen by the wearer. Current technology requires another HoloLens device for third-person observation, but the high price of HoloLens itself increases costs.

[0003] Therefore, how to provide a spatial sharing method and system to solve the shortcomings of existing technologies, such as the inability to achieve third-person observation and shooting functions without adding HoloLens devices, has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a space sharing method and system to solve the problem that the prior art cannot realize third-person observation and shooting functions when excluding the use of HoloLens devices.

[0005] To achieve the above and other related objectives, the present invention provides a spatial sharing method applied to a spatial sharing system comprising a mixed reality head-mounted display terminal, at least one terminal with AR simulation functionality, and a server communicatively connected to the mixed reality head-mounted display terminal and the terminal with AR simulation functionality. The spatial sharing method includes: the mixed reality head-mounted display terminal identifying an image displayed by the terminal with AR simulation functionality to obtain object spatial information of the image on the mixed reality head-mounted display terminal; and forwarding the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation functionality via the server. The terminal with AR simulation function receives the object spatial information of the image on the mixed reality head-mounted display terminal. If it determines that the object spatial information is inconsistent with the object spatial information of the image on the terminal with AR simulation function, it adjusts the object spatial information of the image on the terminal with AR simulation function. The mixed reality head-mounted display terminal captures the change information of the object spatial information of the image on the mixed reality head-mounted display terminal and synchronizes the change information to the terminal with AR simulation function in real time, and displays the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function.

[0006] In one embodiment of the present invention, prior to the step of the mixed reality head-mounted display terminal recognizing an image displayed by a terminal with AR simulation function, the space sharing method further includes: the server establishing a room for a session between the mixed reality head-mounted display terminal and the terminal with AR simulation function, and forming a queue of the mixed reality head-mounted display terminal and the terminal with AR simulation function that first has a session with the mixed reality head-mounted display terminal.

[0007] In one embodiment of the present invention, if the space sharing system includes multiple terminals with AR simulation function, after the space sharing system performs space sharing on the terminal with AR simulation function that first interacts with the mixed reality head-mounted display terminal, another terminal with AR simulation function is added to the queue to perform the space sharing method.

[0008] In one embodiment of the present invention, the step of forwarding the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server includes: the mixed reality head-mounted display terminal sends the object spatial information of the image on the mixed reality head-mounted display terminal to the room, and after the server identifies and adds the terminal with AR simulation function to the queue, it forwards the object spatial information of the image on the mixed reality head-mounted display terminal to the corresponding terminal with AR simulation function.

[0009] In one embodiment of the present invention, when the room receives object space information sent by any client to the server, it will not forward the object space information back to the client that sent the object space information.

[0010] In one embodiment of the present invention, the spatial information of the object in the image on the mixed reality head-mounted display terminal includes the spatial coordinates and rotation information of the object in the image.

[0011] In one embodiment of the present invention, the step of adjusting the spatial information of the object in the image on the terminal with AR simulation function includes: hiding the image; adjusting the virtual camera set on the terminal with AR simulation function; comparing the spatial information of the object in the image on the mixed reality head-mounted display terminal with the spatial information of the object in the image on the terminal with AR simulation function, and calculating the offset of spatial coordinates and rotation information; adjusting the spatial information of the object in the image received by the terminal with AR simulation function on the mixed reality head-mounted display terminal according to the offset of spatial coordinates and rotation information, so as to obtain the spatial coordinates and rotation information of the object in the adjusted image, and assigning the spatial coordinates and rotation information of the object in the adjusted image to the virtual object displayed on the terminal with AR simulation function.

[0012] In one embodiment of the present invention, when a change in spatial coordinates and rotation of the image is sensed, the mixed reality head-mounted display terminal captures the change information of the spatial information of the object in the image on the mixed reality head-mounted display terminal.

[0013] In one embodiment of the present invention, the mixed reality head-mounted display terminal and the terminal with AR simulation function contain the same three-dimensional model resources.

[0014] Another aspect of the present invention provides a spatial sharing system, including a mixed reality head-mounted display terminal, at least one terminal with AR simulation function, and a server communicatively connected to the mixed reality head-mounted display terminal and the terminal with AR simulation function; wherein, the mixed reality head-mounted display terminal is used to identify an image displayed by the terminal with AR simulation function to obtain object spatial information of the image on the mixed reality head-mounted display terminal, and forward the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server, and capture change information of the object spatial information of the image on the mixed reality head-mounted display terminal, and synchronize the change information to the terminal with AR simulation function in real time, and display the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function; the terminal with AR simulation function is used to, after receiving the object spatial information of the image on the mixed reality head-mounted display terminal, if it is determined that the object spatial information is inconsistent with the object spatial information of the image on the terminal with AR simulation function, adjust the object spatial information of the image on the terminal with AR simulation function.

[0015] As described above, the space sharing method and system of the present invention have the following beneficial effects:

[0016] The space sharing method and system described in this invention can achieve third-person perspective observation and shooting functions without adding HoloLens devices; and reduces the cost of space sharing. Attached Figure Description

[0017] Figure 1 The diagram shown is a schematic representation of the principle structure of the space sharing system of the present invention in one embodiment.

[0018] Figure 2 The diagram shown is a flowchart of one embodiment of the space sharing method of the present invention.

[0019] Component designation explanation

[0020] 1 Space sharing system 11 Mixed Reality Head-Mounted Display Terminal 12 Terminal with AR simulation function 13 server S21~S25 step S231'~S233' step Detailed Implementation

[0021] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0022] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0023] Example 1

[0024] This embodiment provides a space sharing method applied to a space sharing system comprising a mixed reality head-mounted display terminal, at least one terminal with AR simulation function, and a server communicatively connected to the mixed reality head-mounted display terminal and the terminal with AR simulation function; the space sharing method includes:

[0025] The mixed reality head-mounted display terminal identifies an image displayed by a terminal with AR simulation function, obtains the object spatial information of the image on the mixed reality head-mounted display terminal, and forwards the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server.

[0026] After receiving the object space information of the image on the mixed reality head-mounted display terminal, if the terminal with AR simulation function determines that the object space information is inconsistent with the object space information of the image on the terminal with AR simulation function, then adjust the object space information of the image on the terminal with AR simulation function.

[0027] The mixed reality head-mounted display terminal captures the changes in the spatial information of objects in the image on the mixed reality head-mounted display terminal, and synchronizes the changes in the image to the terminal with AR simulation function in real time, and displays the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function.

[0028] The space sharing method provided in this embodiment will be described in detail below with reference to the illustrations. Please refer to... Figure 1 The diagram shows a schematic representation of the principle structure of a space-sharing system in one embodiment. Figure 1 As shown, the space sharing system 1 includes a mixed reality head-mounted display terminal 11, at least one terminal 12 with AR simulation function that is communicatively connected to the mixed reality head-mounted display terminal 11, and a server 13 that is communicatively connected to the mixed reality head-mounted display terminal 11 and the terminal 12 with AR simulation function.

[0029] In this embodiment, the mixed reality head-mounted display terminal 11 is a HoloLens device; in practical applications, the terminal 12 with AR simulation function can be a smart terminal with AR simulation function, for example, in this embodiment, an ARCore device is used.

[0030] Please see Figure 2 The diagram shows a flow chart of a space-sharing method in one embodiment. Figure 2 As shown, the space sharing method specifically includes the following steps:

[0031] S21, the server establishes a room for sessions between the mixed reality head-mounted display terminal and the terminal with AR simulation function, and forms a queue for the mixed reality head-mounted display terminal and the terminal with AR simulation function that first sessions with the mixed reality head-mounted display terminal.

[0032] S22, determine whether the space sharing system includes only one terminal with AR simulation function; if yes, then execute S23. If no, then execute S23'.

[0033] S23, the mixed reality head-mounted display terminal identifies an image displayed by a terminal with AR simulation function, obtains the object spatial information of the image on the mixed reality head-mounted display terminal, and forwards the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server.

[0034] In this embodiment, the spatial information of the object in the image on the mixed reality head-mounted display terminal includes the spatial coordinates and rotation information of the object in the image.

[0035] Specifically, S23 includes: the mixed reality head-mounted display terminal sending the spatial information of the object on the mixed reality head-mounted display terminal to the room, and then, after the server identifies and adds the terminal with AR simulation function to the queue, forwarding the spatial information of the object on the mixed reality head-mounted display terminal to the corresponding terminal with AR simulation function.

[0036] In this embodiment, when the room receives object space information sent by any client to the server, it will not forward the object space information back to the client that sent the object space information.

[0037] S24, after the terminal with AR simulation function receives the object space information of the image on the mixed reality head-mounted display terminal, if it determines that the object space information is inconsistent with the object space information of the image on the terminal with AR simulation function, then adjust the object space information of the image on the terminal with AR simulation function.

[0038] Specifically, S24 includes:

[0039] Hide the image, adjust the virtual camera set on the terminal with AR simulation function, compare the spatial information of the image on the mixed reality head-mounted display terminal with the spatial information of the image on the terminal with AR simulation function, and calculate the offset of spatial coordinates and rotation information.

[0040] Based on the offset of the spatial coordinates and rotation information, the spatial information of the object in the image received by the terminal with AR simulation function on the mixed reality head-mounted display terminal is adjusted to obtain the spatial coordinates and rotation information of the object in the adjusted image, and the spatial coordinates and rotation information of the object in the adjusted image are assigned to the virtual object displayed on the terminal with AR simulation function.

[0041] S25, the mixed reality head-mounted display terminal captures the change information of the object spatial information of the image on the mixed reality head-mounted display terminal, and synchronizes the change information to the terminal with AR simulation function in real time, and displays the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function.

[0042] In this embodiment, when a change in spatial coordinates and rotation of the image is sensed, the mixed reality head-mounted display terminal captures the change information of the spatial information of the object in the image on the mixed reality head-mounted display terminal, and sends the change information to the server in real time.

[0043] S23', if the space sharing system includes multiple terminals with AR simulation function, after the terminal with AR simulation function that first communicates with the mixed reality head-mounted display terminal executes the space sharing method, that is, after S23 to S25 are completed, another terminal with AR simulation function joins the queue to execute the space sharing method.

[0044] Specifically, the queue is added to another terminal with AR simulation capabilities, and the space sharing method further includes:

[0045] S231', the mixed reality head-mounted display terminal identifies an image displayed by another terminal with AR simulation function, obtains the object spatial information of the image on the mixed reality head-mounted display terminal, and forwards the object spatial information of the image on the mixed reality head-mounted display terminal to the other terminal with AR simulation function through the server. In this embodiment, the object spatial information of the image on the mixed reality head-mounted display terminal includes the spatial coordinates and rotation information of the objects in the image.

[0046] Specifically, S231' includes: the mixed reality head-mounted display terminal sending the spatial information of the object on the mixed reality head-mounted display terminal to the room, and then, after the server identifies and adds the terminal with AR simulation function to the queue, forwarding the spatial information of the object on the mixed reality head-mounted display terminal to the corresponding terminal with AR simulation function.

[0047] In this embodiment, when the room receives object space information sent by any client to the server, it will not forward the object space information back to the client that sent the object space information.

[0048] S232', after the other terminal with AR simulation function receives the object space information of the image on the mixed reality head-mounted display terminal, if it is determined that the object space information is inconsistent with the object space information of the image on the other terminal with AR simulation function, then the object space information of the image on the other terminal with AR simulation function is adjusted.

[0049] Specifically, S232' includes:

[0050] Hide the image, adjust the virtual camera set on another terminal with AR simulation function, compare the object spatial information of the image on the mixed reality head-mounted display terminal with the object spatial information of the image on the other terminal with AR simulation function, and calculate the offset of spatial coordinates and rotation information;

[0051] Based on the offset of the spatial coordinates and rotation information, the spatial information of the object in the image received by the other terminal with AR simulation function on the mixed reality head-mounted display terminal is adjusted to obtain the spatial coordinates and rotation information of the object in the adjusted image, and the spatial coordinates and rotation information of the object in the adjusted image are assigned to the virtual object displayed on the terminal with AR simulation function.

[0052] S233', the mixed reality head-mounted display terminal captures the change information of the object spatial information of the image on the mixed reality head-mounted display terminal, and synchronizes the change information to another terminal with AR simulation function in real time, and displays the image on the mixed reality head-mounted display terminal through the other terminal with AR simulation function.

[0053] The space sharing method described in this embodiment can achieve third-person perspective observation and shooting functions without adding HoloLens devices.

[0054] Example 2

[0055] This embodiment provides a space sharing system, characterized in that it includes a mixed reality head-mounted display terminal, at least one terminal with AR simulation function, and a server that is communicatively connected to the mixed reality head-mounted display terminal and the terminal with AR simulation function;

[0056] The mixed reality head-mounted display terminal is used to identify an image displayed by a terminal with AR simulation function, to obtain object spatial information of the image on the mixed reality head-mounted display terminal, and forward the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server, and capture the change information of the object spatial information of the image on the mixed reality head-mounted display terminal, and synchronize the change information to the terminal with AR simulation function in real time, and display the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function;

[0057] The terminal with AR simulation function is used to receive the object spatial information of the image on the mixed reality head-mounted display terminal. If it is determined that the object spatial information is inconsistent with the object spatial information of the image on the terminal with AR simulation function, the terminal adjusts the object spatial information of the image on the terminal with AR simulation function.

[0058] The space sharing system provided in this embodiment will be described in detail below with reference to the accompanying drawings. Please refer to... Figure 1 The diagram shows a schematic representation of the principle structure of a space-sharing system in one embodiment. Figure 1 As shown, the space sharing system 1 includes a mixed reality head-mounted display terminal 11, at least one terminal 12 with AR simulation function that is communicatively connected to the mixed reality head-mounted display terminal 11, and a server 13 that is communicatively connected to the mixed reality head-mounted display terminal 11 and the terminal 12 with AR simulation function.

[0059] In this embodiment, the mixed reality head-mounted display terminal 11 is a HoloLens device; in practical applications, the terminal 12 with AR simulation function can be a smart terminal with AR simulation function, for example, in this embodiment, an ARCore device is used.

[0060] The server 1 is used to establish a room for sessions between mixed reality head-mounted display terminals and terminals with AR simulation capabilities, and to form a queue of mixed reality head-mounted display terminals and terminals with AR simulation capabilities that first have sessions with the mixed reality head-mounted display terminals.

[0061] The server 1 is also used to determine whether the space sharing system includes only one terminal 12 with AR simulation function;

[0062] If the space sharing system includes only one terminal 12 with AR simulation function, the mixed reality head-mounted display terminal 11 is used to identify the image displayed by the terminal 12 with AR simulation function, to obtain the object space information of the image on the mixed reality head-mounted display terminal, and to forward the object space information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server.

[0063] The terminal 12 with AR simulation function is used to receive the object space information of the image on the mixed reality head-mounted display terminal. If it is determined that the object space information is inconsistent with the object space information of the image on the terminal with AR simulation function, the terminal 12 adjusts the object space information of the image on the terminal with AR simulation function.

[0064] Specifically, the terminal 12 with AR simulation function is also used to hide the image, adjust the virtual camera set on the terminal with AR simulation function, compare the spatial information of the image on the mixed reality head-mounted display terminal with the spatial information of the image on the terminal with AR simulation function, calculate the offset of spatial coordinates and rotation information; adjust the spatial information of the image on the mixed reality head-mounted display terminal received by the terminal with AR simulation function to obtain the spatial coordinates and rotation information of the object in the adjusted image, and assign the spatial coordinates and rotation information of the object in the adjusted image to the virtual object displayed on the terminal with AR simulation function.

[0065] The mixed reality head-mounted display terminal 11 is also used to capture the change information of the object spatial information of the image on the mixed reality head-mounted display terminal, and synchronize the change information to the terminal with AR simulation function in real time, and display the image on the mixed reality head-mounted display terminal 11 through the terminal with AR simulation function 12.

[0066] If the space sharing system includes multiple terminals with AR simulation capabilities, after the space sharing system performs space sharing on the terminal with AR simulation capability that first interacts with the mixed reality head-mounted display terminal, another terminal with AR simulation capability can then join the queue so that the space sharing system can perform space sharing.

[0067] The scope of protection of the space sharing method described in this invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0068] The present invention also provides a space sharing system, which can implement the space sharing method described in the present invention. However, the implementation device of the space sharing method described in the present invention includes, but is not limited to, the structure of the space sharing system listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of the present invention are included within the protection scope of the present invention.

[0069] In summary, the space sharing method and system described in this invention can achieve third-person perspective observation and shooting functions without adding HoloLens devices, and reduces the cost of space sharing. This invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0070] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A space sharing method, characterized in that, A space-sharing system is applied to a mixed reality head-mounted display terminal, at least one terminal with AR simulation function, and a server that is communicatively connected to the mixed reality head-mounted display terminal and the terminal with AR simulation function; The space sharing method includes: The mixed reality head-mounted display terminal identifies an image displayed by a terminal with AR simulation function, obtains the object spatial information of the image on the mixed reality head-mounted display terminal, and forwards the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server. After receiving the object space information of the image on the mixed reality head-mounted display terminal, if the terminal with AR simulation function determines that the object space information is inconsistent with the object space information of the image on the terminal with AR simulation function, then adjust the object space information of the image on the terminal with AR simulation function. The mixed reality head-mounted display terminal captures the changes in the spatial information of objects in the image on the mixed reality head-mounted display terminal, and synchronizes the changes in the image to the terminal with AR simulation function in real time, and displays the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function.

2. The space sharing method according to claim 1, characterized in that, Prior to the step of the mixed reality head-mounted display terminal recognizing an image displayed by a terminal with AR simulation functionality, the spatial sharing method further includes: The server establishes a room for sessions between mixed reality head-mounted display terminals and terminals with AR simulation capabilities, and forms a queue for the mixed reality head-mounted display terminals and the terminals with AR simulation capabilities that first have sessions with the mixed reality head-mounted display terminals.

3. The space sharing method according to claim 2, characterized in that, If the space sharing system includes multiple terminals with AR simulation capabilities, after the space sharing system performs space sharing on the terminal with AR simulation capability that first interacts with the mixed reality head-mounted display terminal, another terminal with AR simulation capability can then join the queue to perform the space sharing method.

4. The space sharing method according to claim 2, characterized in that, The step of forwarding the spatial information of the object in the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function via the server includes: The mixed reality head-mounted display terminal sends the spatial information of the object on the mixed reality head-mounted display terminal to the room. After the server identifies and adds the terminal with AR simulation function to the queue, it forwards the spatial information of the object on the mixed reality head-mounted display terminal to the corresponding terminal with AR simulation function.

5. The space sharing method according to claim 4, characterized in that, When the room receives object space information sent to the server by any client, it will not forward the object space information back to the client that sent the object space information.

6. The space sharing method according to claim 1, characterized in that, The spatial information of the objects in the image on the mixed reality head-mounted display terminal includes the spatial coordinates and rotation information of the objects in the image.

7. The space sharing method according to claim 6, characterized in that, The step of adjusting the spatial information of the object in the image on the terminal with AR simulation function includes: Hide the image, adjust the virtual camera set on the terminal with AR simulation function, compare the spatial information of the image on the mixed reality head-mounted display terminal with the spatial information of the image on the terminal with AR simulation function, and calculate the offset of spatial coordinates and rotation information. Based on the offset of the spatial coordinates and rotation information, the spatial information of the object in the image received by the terminal with AR simulation function on the mixed reality head-mounted display terminal is adjusted to obtain the spatial coordinates and rotation information of the object in the adjusted image, and the spatial coordinates and rotation information of the object in the adjusted image are assigned to the virtual object displayed on the terminal with AR simulation function.

8. The space sharing method according to claim 6, characterized in that, When a change in spatial coordinates and rotation of the image is sensed, the mixed reality head-mounted display terminal captures the change information of the spatial information of the object in the image on the mixed reality head-mounted display terminal.

9. The space sharing method according to claim 1, characterized in that, The mixed reality head-mounted display terminal and the terminal with AR simulation function contain the same three-dimensional model resources.

10. A space sharing system, characterized in that, It includes a mixed reality head-mounted display terminal, at least one terminal with AR simulation function, and a server that is communicatively connected to the mixed reality head-mounted display terminal and the terminal with AR simulation function; The mixed reality head-mounted display terminal is used to identify an image displayed by a terminal with AR simulation function, to obtain object spatial information of the image on the mixed reality head-mounted display terminal, and forward the object spatial information of the image on the mixed reality head-mounted display terminal to the terminal with AR simulation function through the server, and capture the change information of the object spatial information of the image on the mixed reality head-mounted display terminal, and synchronize the change information to the terminal with AR simulation function in real time, and display the image on the mixed reality head-mounted display terminal through the terminal with AR simulation function; The terminal with AR simulation function is used to receive the object spatial information of the image on the mixed reality head-mounted display terminal. If it is determined that the object spatial information is inconsistent with the object spatial information of the image on the terminal with AR simulation function, the terminal adjusts the object spatial information of the image on the terminal with AR simulation function.

Images