Synchronization system and synchronization method of coordinate system

Abstract

The invention provides a synchronization system and a synchronization method of a coordinate system. The synchronization method includes: communicatively connecting a first head-mounted display device to a first mobile device, the first mobile device including a first light source, and the first mobile device and the first head-mounted display device corresponding to a first coordinate system; outputting, by the first light source, first light based on a first mode; identifying, by a second head-mounted display device, the first mobile device according to the first light, wherein the second head-mounted display device corresponds to a second coordinate system; and the second head-mounted display device takes the first mobile device as a reference point to synchronize the second coordinate system and the first coordinate system.

Description

Technical Field

[0001] This invention relates to extended reality (XR) technology, and more particularly to a synchronization system and method for a coordinate system. Background Technology

[0002] In traditional XR systems, if multiple XR devices need to share map information, a reliable communication channel must be established between them. When the application environment lacks a reliable communication method, the XR devices cannot share map information. On the other hand, XR systems can use markers in the field for positioning. Taking ArUco markers as an example, different ArUco markers can have different shapes or sizes. Users need to consider ambient light sources to place the ArUco markers in appropriate locations. If positioning is to be achieved by detecting ArUco markers, the user must first input the relevant information about the ArUco marker locations into each XR device. Therefore, using ArUco markers for positioning is very inconvenient for users. Summary of the Invention

[0003] This invention provides a coordinate system synchronization system and synchronization method that can automatically synchronize the coordinate system of a head-mounted display (HMD).

[0004] The present invention discloses a coordinate system synchronization system, comprising a first mobile device, a first head-mounted display device, and a second head-mounted display device. The first mobile device includes a first light source. The first head-mounted display device is communicatively connected to the first mobile device, wherein the first mobile device and the first head-mounted display device correspond to a first coordinate system. The second head-mounted display device corresponds to a second coordinate system, wherein the first light source outputs first light based on a first mode, and the second head-mounted display device identifies the first mobile device based on the first light and uses the first mobile device as a reference point to synchronize the second coordinate system with the first coordinate system.

[0005] In one embodiment of the present invention, the above-described synchronization system further includes a second mobile device. The second mobile device includes a second light source, wherein the second light source outputs second light based on a second mode different from the first mode, wherein the second head-mounted display device identifies the first mobile device and the second mobile device respectively based on the first light and the second light to generate an identification result, wherein the second head-mounted display device selects the first mobile device from the first mobile device and the second mobile device as a reference point based on the identification result.

[0006] In one embodiment of the present invention, in response to the first light source stopping outputting the first light based on the first mode, the second head-mounted display device stops synchronizing the second coordinate system with the first coordinate system.

[0007] In one embodiment of the present invention, the second head-mounted display device described above periodically synchronizes with the first coordinate system according to the first optical synchronization second coordinate system and the first coordinate system.

[0008] In one embodiment of the present invention, the second head-mounted display device is communicatively connected to at least one of the first head-mounted display device and the first mobile device, and obtains the first coordinate system from at least one of the first head-mounted display device and the first mobile device.

[0009] In one embodiment of the present invention, during the period when the first light source outputs the first light based on the first mode, the second head-mounted display device and the first head-mounted display device share the same content.

[0010] In one embodiment of the present invention, the second head-mounted display device and the first head-mounted display device share content through a cloud server.

[0011] In one embodiment of the present invention, the first light source described above includes an infrared light-emitting diode.

[0012] A method for synchronizing a coordinate system according to the present invention includes: communicating a first head-mounted display device to a first mobile device, wherein the first mobile device includes a first light source, and the first mobile device and the first head-mounted display device correspond to a first coordinate system; outputting first light from the first light source based on a first mode; identifying the first mobile device based on the first light from a second head-mounted display device, wherein the second head-mounted display device corresponds to a second coordinate system; and using the first mobile device as a reference point to synchronize the second coordinate system with the first coordinate system.

[0013] In one embodiment of the present invention, the above-described synchronization method further includes: a second light source of the second mobile device outputting a second light based on a second mode different from the first mode; a second head-mounted display device identifying the first mobile device and the second mobile device respectively based on the first light and the second light to generate an identification result; and the second head-mounted display device selecting the first mobile device from the first mobile device and the second mobile device as a reference point based on the identification result.

[0014] In one embodiment of the present invention, the above-described synchronization method further includes: in response to the first light source stopping outputting first light based on a first mode, the second head-mounted display device stopping the synchronization of the second coordinate system with the first coordinate system.

[0015] In one embodiment of the present invention, the above-described synchronization method further includes: periodically synchronizing the second coordinate system with the first coordinate system according to the first optical synchronization method of the second head-mounted display device.

[0016] In one embodiment of the present invention, the above-described synchronization method further includes: being communicatively connected by a second head-mounted display device to at least one of a first head-mounted display device and a first mobile device, and obtaining a first coordinate system from at least one of the first head-mounted display device and the first mobile device.

[0017] In one embodiment of the present invention, the above-described synchronization method further includes: during the period when the first light source outputs first light based on a first mode, the second head-mounted display device and the first head-mounted display device perform content sharing.

[0018] In one embodiment of the present invention, the second head-mounted display device and the first head-mounted display device share content through a cloud server.

[0019] In one embodiment of the present invention, the first light source described above includes an infrared light-emitting diode.

[0020] Based on the above, the head-mounted display device of the present invention can identify a specific mobile device by the light output by the mobile device, and then synchronize the coordinate system according to the target mobile device. Therefore, the head-mounted display device can automatically complete the coordinate system synchronization between the head-mounted display device and another head-mounted display device corresponding to the target mobile device. Attached Figure Description

[0021] Figure 1 A schematic diagram of a synchronization system for a coordinate system is shown according to an embodiment of the present invention.

[0022] Figure 2 A flowchart illustrating coordinate system synchronization is shown according to an embodiment of the present invention.

[0023] Figure 3 A flowchart illustrating a synchronization method for a coordinate system is shown in one embodiment of the present invention. Detailed Implementation

[0024] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element references are used in the drawings and description to denote the same or similar parts.

[0025] Figure 1A schematic diagram of a coordinate system synchronization system 100 is illustrated according to an embodiment of the present invention. The synchronization system 100 may include one or more head-mounted display devices (HMDs) and one or more mobile devices. Specifically, the synchronization system 100 may include HMD 110, HMD 120, and mobile device 210. In one embodiment, the synchronization system 100 may further include mobile device 230. One or more HMDs and one or more mobile devices can be communicatively connected to each other. Matched HMDs and mobile devices may have the same coordinate system. For example, assuming HMD 110 is matched with mobile device 210, HMD 110 and mobile device 210 can communicate with each other to achieve coordinate system synchronization.

[0026] HMDs (e.g., HMD 110 or 120) can be worn on a user's head and can be used to provide the user with an XR environment (or XR scene), such as a virtual reality (VR) environment, an augmented reality (AR) environment, or a mixed reality (MR) environment. Mobile devices (e.g., mobile devices 210 or 230) can be held by the user or worn on the user's body. The user can operate the mobile device to interact with the XR scene provided by the HMD paired with the mobile device.

[0027] HMD 110 may include processor 111, storage medium 112, transceiver 113, image extraction device 114, and display 115. Processor 111 may be, for example, a central processing unit (CPU) or other programmable general-purpose or special-purpose microcontroller unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics unit (GPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field-programmable gate array (FPGA), or other similar devices or combinations thereof. Processor 111 may be coupled to storage medium 112, transceiver 113, image extraction device 114, and display 115.

[0028] Storage medium 112 may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or similar elements or combinations thereof. Storage medium 112 may be a non-transitory computer-readable storage medium configured to record a plurality of executable computer programs, modules, or application programs to be loaded by processor 111 to perform the functions of HMD 110.

[0029] Transceiver 113 can be configured to transmit or receive wired or wireless signals. Transceiver 113 can also perform operations such as low-noise amplification, impedance matching, frequency mixing, up- or down-frequency conversion, and filtering amplification. Processor 111 can communicate with other devices (e.g., HMD 120, mobile device 210, or mobile device 230) via transceiver 113.

[0030] Image extraction device 114 may be a camera for extracting images. Image extraction device 114 may include a complementary metal oxide semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor.

[0031] Display 115 can be used to display image data, such as providing an XR environment or XR scene to a user wearing HMD 110. Display 115 may include a liquid-crystal display (LCD) or an organic light-emitting diode (OLED) display. In one embodiment, display 115 can provide an image beam to the user's eyes to form an image on the user's retina, allowing the user to see the XR scene created by HMD 110.

[0032] Mobile device 210 may be paired with and communicatively connected to HMD (e.g., HMD 110). Mobile device 210 may communicate to share the same coordinate system with the paired HMD. Mobile device 210 may include processor 211, storage medium 212, transceiver 213, and one or more light sources 214.

[0033] Processor 211 may be, for example, a CPU or other programmable general-purpose or special-purpose MCU, microprocessor, DSP, programmable controller, ASIC, GPU, ALU, CPLD, FPGA or other similar device or a combination thereof. Processor 211 may be coupled to storage medium 212, transceiver 213 and light source 214.

[0034] Storage medium 212 may be, for example, any type of fixed or removable RAM, ROM, flash memory, HDD, SSD, or similar element or combination thereof. Storage medium 212 may be a non-transitory computer-readable storage medium configured to record a plurality of executable computer programs, modules, or applications to be loaded by processor 211 to perform the functions of mobile device 210.

[0035] Transceiver 213 can be configured to transmit or receive wired or wireless signals. Transceiver 213 can also perform operations such as low-noise amplification, impedance matching, frequency mixing, up- or down-frequency conversion, and filtering amplification. Processor 211 can communicate with other devices (e.g., HMD 110, HMD 120, or mobile device 230) via transceiver 213.

[0036] Light source 214 is used to output light. Multiple light sources 214 can be used to output light patterns. HMD 110 can extract an image of the light pattern and can calculate the rotation or translation of the moving device 210 based on the image using a perspective-n-point (PnP) algorithm. The rotation or translation of the moving device 210 can be used as a reference position for HMD 110 and the moving device 210. Light source 214 is, for example, an infrared light-emitting diode (IRLED). The light emitted by an infrared light-emitting diode is invisible light with high concealment and does not affect the user's visual experience.

[0037] In one embodiment, HMD 120 may have a similar construction or function to HMD 110. HMD 120 may use the same or a different coordinate system as HMD 110 or mobile device 210, or it may use the same or a different coordinate system than mobile device 230. Mobile device 230 may have a similar construction or function to mobile device 210. Mobile device 230 may use the same or a different coordinate system as HMD 110 or mobile device 210, or it may use the same or a different coordinate system than HMD 210.

[0038] Figure 2 A flowchart illustrating coordinate system synchronization is presented according to an embodiment of the present invention, wherein the flowchart may be derived from, for example... Figure 1The synchronization system 100 shown is implemented.

[0039] In step S201, the mobile device 210 may receive a user instruction to enter marker mode. During marker mode, the mobile device 210 may output light (e.g., infrared light) in a specific pattern through a set of light sources 214, wherein the pattern may be associated with one or more activation light sources 214 in the set of light sources 214, the intensity of the output light, or the frequency of flashing, etc. The pattern of light output by the mobile device 210 based on the specific pattern can be used to indicate the identity of the mobile device 210, and the light output by the mobile device 210 can be used to indicate the location of the mobile device 210.

[0040] In step S202, the HMD 120 can detect the light output by the mobile device 210 through the image extraction device. If the light output by the mobile device 210 is detected, the process proceeds to step S203. If the light output by the mobile device 210 is not detected, the process proceeds to step S206.

[0041] In one embodiment, assuming the field contains multiple mobile devices in marking mode, HMD 120 can identify the identity of a mobile device based on the pattern of light emitted by the mobile device, and then determine the synchronization target from among the multiple mobile devices. For example, suppose mobile device 210 in marking mode emits light in a first mode, and mobile device 230 in marking mode emits light in a second mode. After detecting the light emitted by mobile device 210 or mobile device 230, HMD 120 can identify the identity of the mobile device corresponding to the detected light pattern based on information pre-stored in the storage medium, and then generate an identification result. HMD 120 can also determine the position of the mobile device based on the detected light. Assuming the synchronization target of HMD 120 is mobile device 210, HMD 120 can select mobile device 210 from the detected mobile devices 210 and 230 based on the identification result, and obtain the position information of mobile device 210.

[0042] In step S203, HMD 120 can use the position of mobile device 210 as a reference point to synchronize its coordinate system. HMD 120 can synchronize its coordinate system with that of mobile device 210 (or HMD 110) based on the reference point. In one embodiment, HMD 120 can communicate with mobile device 210 (or HMD 110) to obtain the coordinate system of mobile device 210 (or HMD 110). Then, HMD 120 can synchronize the two coordinate systems based on the reference point (i.e., the position of mobile device 210) so that the coordinate system of HMD 120 is the same as that of mobile device 210 (or HMD 110).

[0043] In step S204, HMD 120 can determine whether to perform content sharing between HMD 120 and HMD 110. If HMD 120 determines to perform content sharing, it proceeds to step S205. If HMD 120 determines not to perform content sharing, it proceeds to step S206.

[0044] In step S205, HMD 120 performs content sharing between HMD 120 and HMD 110. Specifically, HMD 120 can communicatively connect to HMD 110 and can transmit data to or receive data from HMD 110. In one embodiment, HMD 120 can communicate with HMD 110 through an external communication device such as a cloud server, and thus perform content sharing through the external communication device.

[0045] In step S206, HMD 120 can determine whether the marking mode of mobile device 210 has ended. If the marking mode has ended, the process stops. If the marking mode has not ended, step S202 is re-executed. Specifically, HMD 120 can stop synchronizing the coordinate system of HMD 120 with the coordinate system of mobile device 210 (or HMD 110) in response to the light source 214 of mobile device 210 stopping outputting light based on a specific mode.

[0046] In one embodiment, HMD 120 may periodically synchronize its coordinate system with that of mobile device 210 (or HMD 110) based on the light output by mobile device 210.

[0047] Figure 3 A flowchart illustrating a synchronization method for a coordinate system according to an embodiment of the present invention is shown, wherein the synchronization method may be as follows: Figure 1The synchronization system 100 shown is implemented. In step S301, a first head-mounted display device is communicatively connected to a first mobile device, wherein the first mobile device includes a first light source, and the first mobile device and the first head-mounted display device correspond to a first coordinate system. In step S302, the first light source outputs first light based on a first mode. In step S303, a second head-mounted display device identifies the first mobile device based on the first light, wherein the second head-mounted display device corresponds to a second coordinate system. In step S304, the second head-mounted display device uses the first mobile device as a reference point to synchronize the second coordinate system with the first coordinate system.

[0048] In summary, the mobile device in the synchronization system of this invention can output light for identity recognition based on a specific pattern. When multiple mobile devices exist in the application area of ​​the synchronization system, the head-mounted display device can determine that the mobile device is the target based on the pattern of the light output by the mobile device, and then use the mobile device as a reference point for coordinate system synchronization. Compared with traditional coordinate system synchronization methods, this invention can quickly locate the synchronization target in the application area or select the synchronization target from many mobile devices for the head-mounted display device, and then perform coordinate system synchronization. Furthermore, users of this invention do not need to set markers in the application area of ​​the synchronization system. The light source of the mobile device of this invention can adapt to various lighting conditions. The light output by the light source can be invisible light with good concealment. Therefore, the synchronization system can complete synchronization without interfering with the visual experience of individual users.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A synchronization system for a coordinate system, characterized in that, include: The first moving device includes a first light source; A first head-mounted display device is communicatively connected to the first mobile device, wherein the first mobile device and the first head-mounted display device correspond to a first coordinate system; as well as The second head-mounted display device corresponds to the second coordinate system, wherein The first light source outputs first light based on a first mode, wherein The second head-mounted display device identifies the first moving device based on the first light and uses the first moving device as a reference point to synchronize the second coordinate system with the first coordinate system.

2. The synchronization system according to claim 1 further includes: The second mobile device includes a second light source, wherein the second light source outputs second light based on a second mode different from the first mode, wherein... The second head-mounted display device identifies the first moving device and the second moving device based on the first light and the second light respectively, to generate an identification result, wherein The second head-mounted display device selects the first mobile device from the first mobile device and the second mobile device as the reference point based on the recognition result.

3. The synchronization system according to claim 1, wherein In response to the first light source ceasing to output the first light based on the first mode, the second head-mounted display device ceasing to synchronize the second coordinate system with the first coordinate system.

4. The synchronization system according to claim 1, wherein The second head-mounted display device periodically synchronizes the second coordinate system with the first coordinate system according to the first optical synchronization.

5. The synchronization system according to claim 1, wherein... The second head-mounted display device is communicatively connected to at least one of the first head-mounted display device and the first mobile device, and obtains the first coordinate system from the first head-mounted display device and the first mobile device.

6. The synchronization system according to claim 1, wherein During the period when the first light source outputs the first light based on the first mode, the second head-mounted display device and the first head-mounted display device share the same content.

7. The synchronization system according to claim 6, wherein The second head-mounted display device and the first head-mounted display device share the content through a cloud server.

8. The synchronization system according to claim 1, wherein the first light source comprises an infrared light-emitting diode.

9. A method for synchronizing a coordinate system, characterized in that, include: A first head-mounted display device is communicatively connected to a first mobile device, wherein the first mobile device includes a first light source, and wherein the first mobile device and the first head-mounted display device correspond to a first coordinate system. The first light source outputs first light based on a first mode; The first moving device is identified by a second head-mounted display based on the first light, wherein the second head-mounted display corresponds to a second coordinate system; as well as The second head-mounted display device uses the first moving device as a reference point to synchronize the second coordinate system with the first coordinate system.

10. The synchronization method according to claim 9, further comprising: The second light source of the second mobile device outputs a second light based on a second mode different from the first mode; The second head-mounted display device identifies the first moving device and the second moving device based on the first light and the second light respectively, so as to generate an identification result; as well as The second head-mounted display device selects the first mobile device from the first mobile device and the second mobile device as the reference point based on the recognition result.

Images