Positioning information display method, wearable device, XR device and storage medium

Abstract

The embodiment of the invention discloses a positioning information display method, wearable equipment, XR equipment and a storage medium, and is used for enabling the XR equipment to display the positioning information on a display interface in an overlapping manner after the wearable equipment sends the positioning information to the XR equipment, so that a user does not need to switch back and forth between a real environment and a current display interface, and the user experience is improved. And the method is more convenient and real-time. The method provided by the embodiment of the invention comprises the following steps: acquiring positioning information of the wearable equipment; and sending the positioning information to XR equipment, wherein the positioning information is used for displaying the positioning information on a current display interface of the XR equipment in an overlapping manner.

Description

Technical Field

[0001] The present application relates to the field of positioning and tracking, and in particular to a method for displaying positioning information, a wearable device, an XR device, and a storage medium. Background Art

[0002] In the prior art, electronic devices with positioning functions such as mobile phones and wristbands are generally used for positioning and tracking. Taking mobile phones as an example, after the positioning function of mobile phone 1 is turned on and the positioning software of mobile phone 2 is opened, the specific location of mobile phone 1 can be viewed in real time, and the navigation function can also be selected to track mobile phone 1 in real time.

[0003] When mobile phone 1 sends positioning information to mobile phone 2, mobile phone 2 sends a prompt message to the observer after receiving the positioning information. If the observer sets mobile phone 2 to silent or do not disturb mode, or the observer is in a noisy environment and does not notice the prompt message of mobile phone 2, then the observer cannot receive the message from mobile phone 1 in time. Summary of the invention

[0004] The embodiments of the present application provide a method for displaying positioning information, a wearable device, an XR device, and a storage medium. After the wearable device sends positioning information to the XR device, the XR device can overlay and display the positioning information on the current display interface. The user does not need to switch back and forth between the real environment and the current display interface, which is more convenient and real-time.

[0005] A first aspect of the present application provides a method for displaying positioning information, which is applied to a wearable device. The method may include: obtaining the positioning information of the wearable device; sending the positioning information to an XR device to instruct the positioning information to be superimposed and displayed on a current display interface of the XR device.

[0006] A second aspect of the present application provides a method for displaying positioning information, which is applied to an XR device. The method may include: ： Receive positioning information sent by the wearable device; and overlay and display the positioning information on the current display interface.

[0007] A third aspect of the present application provides a wearable device, which may include:

[0008] An acquisition module is used to obtain the positioning information of the wearable device;

[0009] A sending module is used to send the positioning information to the XR device to instruct the positioning information to be superimposed and displayed on the current display interface of the XR device.

[0010] A fourth aspect of the present application provides an XR device, which may include:

[0011] A receiving module, used for receiving positioning information sent by a wearable device;

[0012] The display module is used to display the positioning information on a superimposed manner on the current display interface.

[0013] A fifth aspect of the present application provides a wearable device, which may include:

[0014] A memory storing executable program code;

[0015] a processor and a transceiver coupled to the memory;

[0016] The wearable device is used to execute the method described in the first aspect of the present application.

[0017] A sixth aspect of the present application provides an XR device, which may include:

[0018] A memory storing executable program code;

[0019] a processor and a transceiver coupled to the memory, and a display;

[0020] The XR device is used to execute the method described in the second aspect of the present application.

[0021] Another aspect of an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a processor, enables the processor to execute the method described in the first aspect of the present application.

[0022] Yet another aspect of an embodiment of the present application discloses a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the method described in the first aspect of the present application.

[0023] Another aspect of an embodiment of the present application discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes the method described in the first aspect of the present application.

[0024] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0025] In an embodiment of the present application, the wearable device obtains the positioning information of the wearable device; the wearable device sends the positioning information to the XR device to instruct the positioning information to be superimposed and displayed on the current display interface of the XR device. The XR device receives the positioning information sent by the wearable device; the XR device superimposes and displays the positioning information on the current display interface. After the wearable device sends the positioning information to the XR device, the XR device can superimpose and display the positioning information on the current display interface, without the user having to switch back and forth between the real environment and the current display interface, which is more convenient and real-time. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments and the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained based on these drawings.

[0027] Figure 1 A schematic diagram of an embodiment of a method for displaying positioning information in an embodiment of the present application;

[0028] Figure 2A A schematic diagram of a user wearing AR glasses in an embodiment of the present application;

[0029] Figure 2B A schematic diagram of a communication button provided on the AR glasses in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of an embodiment of a wearable device in an embodiment of the present application;

[0031] Figure 4 This is a schematic diagram of an embodiment of an XR device in an embodiment of the present application;

[0032] Figure 5 This is another schematic diagram of an embodiment of a wearable device in an embodiment of the present application;

[0033] Figure 6 Schematic diagram of another embodiment of the XR device in the embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present application provide a method for displaying positioning information, a wearable device, an XR device, and a storage medium. After the wearable device sends positioning information to the XR device, the XR device can overlay and display the positioning information on the current display interface. The user does not need to switch back and forth between the real environment and the current display interface, which is more convenient and real-time.

[0035] In order to make the technical personnel in the technical field better understand the scheme of the present application, the technical scheme in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, they should all fall within the scope of protection of the present application.

[0036] In the prior art, electronic devices with positioning functions such as mobile phones and bracelets are generally used for positioning and tracking. Taking mobile phones as an example, after the positioning function of mobile phone 1 is turned on and the positioning software is opened on mobile phone 2, the specific location of mobile phone 1 can be viewed in real time, or the navigation function can be selected to track mobile phone 1 in real time. Mobile phone 2 needs to open the positioning software and find the specific location of mobile phone 1 according to the prompt information, but the user needs to hold mobile phone 2 and keep looking at the prompt information of mobile phone 2.

[0037] The following is a brief description of some of the terms involved in this application, as follows:

[0038] 1. Augmented Reality (AR)

[0039] Simply put, it is to superimpose virtual information onto the real world, combine real scenes with virtual scenes, and experience them through electronic devices such as smartphones, tablets, and glasses.

[0040] 2. Virtual Reality (VR)

[0041] It is to use equipment to simulate a completely virtual world, enter the virtual world by wearing VR equipment, and achieve an immersive experience.

[0042] 3. Mixed Reality (MR)

[0043] VR and AR have not reached their respective peaks, but there are signs of integration, namely mixed reality (MR), that is, MR = VR*AR, which is to mix the real world and the virtual world together to create a new visual environment. The environment contains both physical entities and virtual information, and it must be "real-time". The relative position of virtual objects in AR devices will move with the movement of the device, but MR will not. Under ideal conditions, virtual objects created by AR devices can be clearly seen as virtual, while virtual objects and real objects seen by MR device users are almost indistinguishable from real objects.

[0044] 4. Extended Reality

[0045] XR is a new concept, which refers to a real and virtual environment that can be interacted with by humans and machines, generated by computer technology and wearable devices. Extended reality includes augmented reality (AR), virtual reality (VR), mixed reality (MR) and other forms. In other words, to avoid confusion, XR is actually a general term that includes AR, VR, and MR. XR is divided into multiple levels, from a virtual world with limited sensor input to a fully immersive virtual world. Therefore, XR devices refer to those that can provide AR, VR, and MR experiences.

[0046] Some common XR devices may include:

[0047] VR head-mounted display: VR head-mounted display is a display device worn on the head, which allows users to experience an immersive virtual world. For example, Oculus Rift, HTC Vive, etc. are common VR head-mounted displays.

[0048] AR glasses: AR glasses are glasses with AR functions that can overlay virtual content on the real world to provide an augmented reality experience. For example, Google Glass, Microsoft HoloLens, etc. are common AR smart glasses.

[0049] MR head-mounted display: MR head-mounted display is a device that combines VR and AR technologies, allowing users to switch freely between the virtual and real worlds, providing a mixed reality experience. For example, Magic Leap One and others are common MR head-mounted displays.

[0050] AR / VR helmet: AR / VR helmet is a device that combines AR and VR technologies, which can overlay virtual content on the real world or provide an immersive virtual reality experience. For example, Samsung Gear VR, Google Daydream, etc. are common AR / VR helmets.

[0051] The technical solution of the present application is further described below by way of embodiments. Figure 1 As shown, it is a schematic diagram of an embodiment of a method for displaying positioning information in an embodiment of the present application, which may include:

[0052] 101. The wearable device obtains positioning information of the wearable device.

[0053] Optionally, the wearable device obtaining the positioning information of the wearable device may include: the wearable device responding to an operation of a user turning on a positioning function to obtain the positioning information of the wearable device.

[0054] It is understandable that the user's operation to enable the positioning function can be manual operation or voice operation, which is not limited here.

[0055] Optionally, the wearable device obtains the positioning information of the wearable device, which may include but is not limited to the following implementation methods:

[0056] (1) The wearable device obtains the positioning information of the wearable device through the positioning function of the Global Positioning System (GPS). Modern wearable devices have built-in GPS chips. By opening a map application or a navigation application on the wearable device, the current location information, i.e., positioning information, can be obtained.

[0057] (2) The wearable device obtains the positioning information of the wearable device through the base station. If the wearable device does not have the GPS positioning function turned on, it can still obtain approximate location information by connecting to a nearby base station. Although this method is not accurate enough, it is sufficient for some simple navigation and location confirmation.

[0058] (3) The wearable device obtains the positioning information of the wearable device through wireless fidelity (WiFi). WiFi positioning relies on the wireless network signal to which the wearable device is connected. By matching with the pre-collected WiFi hotspot database, more accurate location information can be obtained.

[0059] (4) The wearable device obtains the positioning information of the wearable device through the Internet Protocol (IP) address. By obtaining the IP address of the network where the wearable device is located and performing an IP address positioning query, the approximate location information can be obtained.

[0060] Optionally, the type of the positioning information includes text, picture, audio or video.

[0061] Optionally, wearable devices can also be called wearable smart devices, which are a general term for the use of wearable technology to intelligently design and develop wearable devices for daily wear, such as glasses (which may include AR glasses, MR glasses, VR glasses, etc.), gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and independent of smartphones to achieve complete or partial functions, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0062] 102. The wearable device sends the positioning information to the XR device to instruct the positioning information to be superimposed and displayed on a current display interface of the XR device.

[0063] The XR device receives the positioning information sent by the wearable device.

[0064] Optionally, the positioning information is also used to instruct the XR device to perform positioning tracking in combination with the current real environment.

[0065] 103. The XR device displays the positioning information on a current display interface.

[0066] Optionally, the method may further include: when the XR device displays the positioning information in a superimposed manner on the current display interface, outputting the positioning information through a speaker.

[0067] Optionally, when the XR device superimposes and displays the positioning information on the current display interface, outputting the positioning information through the speaker may include: when the XR device superimposes and displays the positioning information on the current display interface, responding to a user's voice output operation, outputting the positioning information through the speaker.

[0068] Optionally, the XR device superimposes and displays the positioning information on the current display interface, which may include but is not limited to the following implementation methods:

[0069] (1) When the XR device is an AR device or an MR device, the positioning information is superimposed and displayed when the current real environment image is displayed on the current display interface; or,

[0070] (2) When the XR device is an AR device or an MR device, when the current real environment image is displayed on the current display interface, the positioning information is superimposed and displayed, and the positioning information is output through a speaker; or,

[0071] (3) When the XR device is a VR or MR device, when a virtual image is displayed on the current display interface, the positioning information is superimposed and displayed; or,

[0072] (4) When the XR device is a VR or MR device, when a virtual image is displayed on the current display interface, the positioning information is superimposed and displayed, and the positioning information is output through a speaker.

[0073] Optionally, in a driving scenario, the output methods (1) and (2) above are also applicable. Because the user's hands are driving the vehicle, the positioning information can be superimposed on the current real environment image displayed on the current display interface through the XR device worn, so that the user can see the positioning information in time.

[0074] Optionally, the XR device superimposing and displaying the positioning information on the current display interface may include:

[0075] When the current real environment is a noisy environment, when the XR device is an AR device or an MR device, the positioning information is superimposed and displayed when the current real environment picture is displayed through the current display interface, or the positioning information is superimposed and displayed when the current real environment picture is displayed through the current display interface, and the positioning information is output through the speaker; or,

[0076] When the current real environment is a noisy environment, when the XR device is a VR device or an MR device, the positioning information is superimposed and displayed when the virtual image is displayed through the current display interface, or the positioning information is superimposed and displayed when the virtual image is displayed through the current display interface, and the positioning information is output through the speaker; wherein, the noisy environment is an environment in which the sound is greater than a preset sound threshold.

[0077] It is understandable that the sound of a noisy environment can be called environmental noise, which generally refers to the sound generated in industrial production, construction, transportation and social life that disturbs the surrounding living environment. Environmental noise is a regulation of the permissible range of noise to protect people's health and living environment. The formulation principles should be based on protecting people's hearing, sleep, rest, conversation and thinking, and should be advanced, scientific and realistic. The basic standard of environmental noise is the basic basis of the environmental noise standard.

[0078] Optionally, the method may further include: the XR device performing positioning tracking according to the positioning information and the current real environment.

[0079] Optionally, the XR device performs positioning tracking according to the positioning information, which may include: the XR device generates a tracking path according to the positioning information and the current real environment; and the XR device superimposes and displays the tracking path on the current display interface.

[0080] Optionally, when the XR device overlays and displays the tracking path on the current display interface, it may also output the tracking path through a speaker.

[0081] Optionally, the XR device generates a tracking path according to the positioning information and the current real environment, which may include: the XR device responds to a user's start operation, starts a navigation mode, and generates a tracking path according to the positioning information and the current real environment.

[0082] Optionally, the XR device superimposing and displaying the tracking path on the current display interface may include:

[0083] When the XR device is an AR device or an MR device, the tracking path is superimposed and displayed when the current real environment image is displayed through the current display interface, or the positioning information and the tracking path are superimposed and displayed when the current real environment image is displayed through the current display interface. Or,

[0084] When the XR device is an AR device or an MR device, when the current real environment is a noisy environment, the tracking path is superimposed when the current real environment image is displayed through the current display interface, or the positioning information and the tracking path are superimposed when the current real environment image is displayed through the current display interface. Or,

[0085] When the XR device is an AR device or an MR device, when the current scene belongs to a driving scene, the tracking path is superimposed and displayed when the current real environment picture is displayed through the current display interface, or the positioning information and the tracking path are superimposed and displayed when the current real environment picture is displayed through the current display interface, or the tracking path is superimposed and displayed when the current real environment picture is displayed through the current display interface, and the tracking path is output through the speaker, or the positioning information and the tracking path are superimposed and displayed when the current real environment picture is displayed through the current display interface, and the tracking path is output through the speaker, or the positioning information and the tracking path are superimposed and displayed when the current real environment picture is displayed through the current display interface, and the positioning information is output through the speaker. Because the user's hands are driving the vehicle, the current real environment picture can be displayed through the current display interface of the worn AR device or MR device; optionally, at least one of the positioning information and the tracking path can also be displayed; optionally, at least one of the positioning information and the tracking path can also be output through the speaker, so that the user can obtain the positioning information and the tracking path in time. Or,

[0086] When the XR device is a VR or MR device, the tracking path is superimposed and displayed when the virtual image is displayed through the current display interface, or the positioning information and the tracking path are superimposed and displayed when the virtual image is displayed through the current display interface. Or,

[0087] When the XR device is a VR or MR device, when the current real environment is a noisy environment, the tracking path is superimposed and displayed when the virtual image is displayed through the current display interface, or the positioning information and the tracking path are superimposed and displayed when the virtual image is displayed through the current display interface.

[0088] Optionally, the method may also include: when the XR device is an AR device or an MR device, when the current real environment picture is displayed through the current display interface, the tracking path is superimposed and displayed, and the tracking path is output through the speaker; or, when the current real environment picture is displayed through the current display interface, the positioning information and the tracking path are superimposed and displayed, and the tracking path is output through the speaker.

[0089] Optionally, the method may also include: when the XR device is a VR device or an MR device, when the virtual image is displayed through the current display interface, the tracking path is superimposed and displayed, and the tracking path is output through the speaker; or, when the virtual image is displayed through the current display interface, the positioning information and the tracking path are superimposed and displayed, and the tracking path is output through the speaker.

[0090] It is understandable that the description is made by taking the wearable device as AR glasses 1 and the XR device as AR glasses 2 as an example. Figure 2A As shown, it is a schematic diagram of a user wearing AR glasses in an embodiment of the present application. AR glasses 1 have a positioning function, and AR glasses 2 can display the positioning information of AR glasses 1 in addition to displaying the current real environment picture. If you need to track AR glasses 1, you can select navigation mode and find AR glasses 1 according to the tracking path (also called navigation route). Optionally, AR glasses 2 can also display the tracking path. Compared with the positioning tracking of mobile phones, the technical solution of the present application not only frees the user's hands from holding the mobile phone, but also the user's line of sight does not need to always switch back and forth between the current real environment and the mobile phone screen. It only needs to observe the display interface of AR glasses 2.

[0091] Since the glasses are relatively close to the user's ears, communication can also be carried out through sound. For example, the wearer of AR glasses 1 turns on the communication function through the communication button to record or make a real-time voice call. AR glasses 1 transmits the received voice information to AR glasses 2, and plays the voice information through the speaker of AR glasses 2. Because the speaker is very close to the human ear, it has a similar function to headphones, so this voice information has a certain degree of confidentiality.

[0092] 104. The wearable device obtains help-seeking information.

[0093] Optionally, the help information is information including help keywords. Exemplarily, the help keywords may include 110, 120, help me, help, save my life, save me, police, fire, flood, traffic accident and other keywords.

[0094] Optionally, the wearable device obtains the help information, which may include but is not limited to the following implementations:

[0095] The wearable device responds to a request for help from a user of the wearable device. Or,

[0096] When the wearable device detects that the current real environment is a dangerous environment, the wearable device generates the help message.

[0097] When the wearable device detects that the user's heart rate exceeds a preset heart rate and the duration exceeds a preset duration, the wearable device generates the help message.

[0098] Optionally, the hazardous environment includes water environment, roads, intersections, stations, subway stations, parking lots, rainy days, typhoons, snowy days, hail, deserts, seas, etc.

[0099] The wearable device can detect, based on positioning information, maps, and weather forecasts, that the wearable device is currently in a water environment, road, intersection, bus station, subway station, parking lot, desert, or sea, and can determine that the current real environment is a dangerous environment, and automatically generate a message for help; or, if it detects that the current weather is rainy, typhoon, snowy, hail, etc., it can also determine that the current real environment is a dangerous environment, and automatically generate a message for help, and send it to the XR device, so that users using the XR device can obtain the help information in a timely manner and take corresponding measures.

[0100] Optionally, the type of the help information includes text, picture, audio or video, etc.

[0101] Optionally, the wearable device responding to a help message sent by a user of the wearable device may include: the wearable device generating the help message in response to an input operation of the user of the wearable device.

[0102] Optionally, the wearable device generates the help information in response to an input operation of a user of the wearable device, which may include: the wearable device turns on a communication function in response to an operation of the user of the wearable device turning on a communication button, and generates a text help information in response to a text input operation of the user;

[0103] The wearable device sending the help information to the XR device may include: the wearable device sending the text help information to the XR device.

[0104] Optionally, the wearable device generates the help information in response to an input operation of a user of the wearable device, which may include: the wearable device turns on a communication function in response to an operation of the user of the wearable device turning on a communication button; generates a picture help information in response to a user's photo taking operation or a user's operation of selecting a picture from an album;

[0105] The wearable device sending the help information to the XR device may include: the wearable device sending the picture help information to the XR device.

[0106] Optionally, the wearable device generates the help information in response to an input operation of a user of the wearable device, which may include: the wearable device responds to the user of the wearable device turning on a communication button to turn on a communication function; responds to a recording operation of the user, or turns on an audio call operation; the wearable device generates the recording help information or the audio call help information;

[0107] The wearable device sending the help information to the XR device may include: the wearable device sending the recorded help information or the audio call help information to the XR device.

[0108] Optionally, the wearable device generates the help information in response to an input operation of a user of the wearable device, which may include: the wearable device responds to the user of the wearable device turning on a communication button to turn on a communication function; responds to the user turning on a video call; the wearable device generates the video call help information;

[0109] The wearable device sending the help information to the XR device may include: the wearable device sending the video call help information to the XR device.

[0110] Optionally, the communication button is set at any location of the wearable device. For example, assuming that the wearable device is an AR glasses, the communication button can be set on the glasses leg of the AR glasses, or on the frame. Figure 2BAs shown, it is a schematic diagram of the communication button set on the AR glasses in the embodiment of the present application. Assuming that the wearable device is a smart bracelet, the communication button can be set on the wristband of the smart bracelet, or on the side of the display screen of the smart bracelet. Assuming that the wearable device is a smart watch, the communication button can be set on the wristband of the smart watch, or on the side of the dial of the smart watch.

[0111] Optionally, the wearer of the wearable device is an elderly person or a child, and the wearer of the XR device is the guardian of the elderly person or the child. The guardian of the elderly person may be a child, and the guardian of the child may be a parent.

[0112] Optionally, the method may further include: the wearable device obtains real-time information of the user, the real-time information may include but is not limited to departure information, arrival information, course start / end information, activity start / end information, and transportation information; and then sends the information to the XR device. Assuming that the wearer of the wearable device is an elderly person or a child, and the wearer of the XR device is the guardian of the elderly person or the child, the dynamics of the elderly person or the child can be grasped in time to achieve the effect of guardianship.

[0113] For example, the wearable device is AR glasses 1, and the XR device is AR glasses 2. When the wearer of AR glasses 1 needs help, he can manually input help information on AR glasses 1, and AR glasses 1 sends the help information to AR glasses 2; or, when AR glasses 1 recognizes the current real environment, that is, the surrounding environment is a dangerous environment, it can automatically generate help information and send it to AR glasses 2. The wearer of AR glasses 2 can quickly receive the help information sent by AR glasses 1 and make corresponding processing according to the actual situation.

[0114] 105. The wearable device sends the help information to the XR device, where the help information is used to instruct the XR device to perform positioning tracking according to the positioning information.

[0115] The XR device receives the help information sent by the wearable device.

[0116] 106. The XR device displays the help information superimposed on the current display interface.

[0117] Optionally, the method may further include: the XR device outputting the help information through a speaker.

[0118] Optionally, the XR device outputting the help information through a speaker may include: the XR device responding to a voice output operation of the user and outputting the positioning information through the speaker.

[0119] It can be understood that the XR device superimposes and displays the help information on the current display interface; or, the XR device superimposes and displays the help information on the current display interface, and outputs the help information through the speaker.

[0120] Optionally, the XR device performs positioning tracking according to the positioning information and the current real environment, which may include: generating a tracking path according to the positioning information and the current real environment when a request for help is received; and superimposing and displaying the tracking path on the current display interface.

[0121] Optionally, the XR device superimposes and displays the positioning information on the current display interface, which may include but is not limited to the following implementation methods:

[0122] (1) When the XR device is an AR device or an MR device, the help information is displayed superimposed on the current display interface while displaying the current real environment image; or,

[0123] (2) When the XR device is an AR device or an MR device, the help information is displayed on a superimposed manner when the current real environment image is displayed on the current display interface, and the help information is output through a speaker; or

[0124] (3) When the XR device is a VR or MR device, when a virtual image is displayed on the current display interface, the help information is superimposed and displayed; or,

[0125] (4) When the XR device is a VR or MR device, when a virtual image is displayed on the current display interface, the help information is superimposed and displayed, and the help information is output through a speaker.

[0126] Optionally, in a driving scenario, the output methods (1) and (2) above are also applicable. Because the user's hands are driving the vehicle, the help information can be superimposed on the current real environment image displayed on the current display interface through the XR device worn, so that the user can see the help information in time.

[0127] Optionally, the XR device superimposing and displaying the help information on the current display interface may include:

[0128] When the current real environment is a noisy environment, when the XR device is an AR device or an MR device, the help information is superimposed and displayed when the current real environment picture is displayed through the current display interface, or the help information is superimposed and displayed when the current real environment picture is displayed through the current display interface, and the help information is output through the speaker; or

[0129] When the current real environment is a noisy environment, when the XR device is a VR device or an MR device, the help message is superimposed and displayed when the virtual image is displayed through the current display interface, or the help message is superimposed and displayed when the virtual image is displayed through the current display interface, and the help message is output through the speaker; wherein, the noisy environment is an environment in which the sound is greater than a preset sound threshold.

[0130] Optionally, when the XR device is an AR device or an MR device, when the current real environment picture is displayed on the current display interface, at least one of the positioning information, the help information, and the tracking path can be superimposed and displayed.

[0131] Optionally, when the XR device is a VR device or an MR device, when a virtual image is displayed on the current display interface, at least one of the positioning information, the help information, and the tracking path may be superimposed and displayed.

[0132] In the technical solution of the present application, the AR glasses 2 can receive the positioning information and help information sent by the AR glasses 1 in a timely manner. For example, in some specific application scenarios, it is still very practical. For example, an adult is the wearer of the AR glasses 2, and a child is the wearer of the AR glasses 1, and the adult can monitor the child in a timely manner.

[0133] When an adult needs to find a child, they can directly find the child through the positioning of the wearable device and the navigation function of the XR device; or, when the child needs to send a message for help through the wearable device, the adult can receive the message for help and respond in time through the XR device; or, the wearable device can autonomously identify the surrounding environment. When it is determined that the child’s environment is a dangerous environment, it can send a message for help to the XR device, and the adult can rush to the child in time to avoid bad things from happening.

[0134] Exemplarily, the technical solutions of the present application can all use AR glasses for communication. AR glasses can display the superposition of the real environment picture and positioning information in real time, making positioning tracking more convenient and real-time. For some special scenarios, users who receive positioning information or help information can receive positioning information or help information in a timely manner through both picture and sound signals. It is understandable that some users do not wear AR glasses, such as children who are not convenient to wear glasses, but wear wearable devices such as smart watches or smart bracelets, and can also communicate with XR devices through wearable devices such as smart bracelets or smart watches.

[0135] It should be noted that steps 104-106 are optional steps. The timing of steps 101-103 and steps 104-106 is not limited. For example, steps 101-103 may be performed before steps 104-106, or after steps 104-106, or steps 101 and 104 may be performed together, steps 102 and 105 may be performed together, and steps 103 and 106 may be performed together, or other timings may be performed, which are not specifically limited in this application.

[0136] In an embodiment of the present application, a wearable device obtains positioning information of the wearable device; the wearable device sends the positioning information to an XR device, and the positioning information is used to overlay and display the positioning information on a current display interface of the XR device. The XR device receives the positioning information sent by the wearable device; the XR device overlays and displays the positioning information on the current display interface. After the wearable device sends the positioning information to the XR device, the XR device can overlay and display the positioning information on the current display interface, without the user having to switch back and forth between the real environment and the current display interface, which is more convenient and real-time.

[0137] like Figure 3 FIG. 1 is a schematic diagram of an embodiment of a wearable device in an embodiment of the present application, which may include:

[0138] An acquisition module 301 is used to acquire the positioning information of the wearable device;

[0139] The sending module 302 is used to send the positioning information to the XR device to instruct the positioning information to be superimposed and displayed on the current display interface of the XR device.

[0140] Optionally, the positioning information is also used to instruct the XR device to perform positioning tracking in combination with the current real environment.

[0141] Optionally, the acquisition module 301 is further configured to respond to a help message sent by a user of the wearable device; or, in the case where it is detected that the current real environment is a dangerous environment, generate the help message; or, in the case where it is detected that the heart rate of the user of the wearable device exceeds a preset heart rate and the duration exceeds a preset duration, generate the help message;

[0142] The sending module 302 is used to send the help information to the XR device, where the help information is used by the XR device to perform positioning tracking according to the positioning information.

[0143] Optionally, the type of the help information includes text, picture, audio or video.

[0144] like Figure 4As shown, it is a schematic diagram of an embodiment of an XR device in an embodiment of the present application, which may include:

[0145] The receiving module 401 is used to receive the positioning information sent by the wearable device;

[0146] The display module 402 is used to display the positioning information on a current display interface.

[0147] Optionally, the XR device may also include:

[0148] The tracking module 403 is used to perform positioning tracking according to the positioning information and the current real environment.

[0149] Optionally, the receiving module 401 is further configured to receive a help message sent by the wearable device;

[0150] The display module 402 is used to display the help information on the current display interface.

[0151] Optionally, a tracking module 403 is specifically configured to generate a tracking path according to the positioning information and the current real environment when receiving the help information;

[0152] The display module 402 is further configured to overlay and display the tracking path on the current display interface.

[0153] Optionally, the type of the help information includes text, picture, audio or video.

[0154] like Figure 5 FIG. 1 is a schematic diagram of another embodiment of a wearable device in an embodiment of the present application, which may include:

[0155] Combine the following Figure 5 A detailed introduction to each component of the wearable device group:

[0156] The RF circuit 510 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 580 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 510 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 510 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0157] The memory 520 can be used to store software programs and modules. The processor 580 executes various functional applications and data processing of the wearable device by running the software programs and modules stored in the memory 520. The memory 520 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the wearable device (such as audio data, a phone book, etc.), etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0158] The input unit 530 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the wearable device. Specifically, the input unit 530 may include a touch panel 531 and other input devices 532. The touch panel 531, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 531 or near the touch panel 531 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 531 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 580, and can receive and execute commands sent by the processor 580. In addition, the touch panel 531 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 531, the input unit 530 may also include other input devices 532. Specifically, other input devices 532 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0159] The display unit 540 can be used to display information input by the user or information provided to the user and various menus of the wearable device. The display unit 540 may include a display panel 541. Optionally, the display panel 541 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 531 may cover the display panel 541. When the touch panel 531 detects a touch operation on or near it, it is transmitted to the processor 580 to determine the type of touch event. Subsequently, the processor 580 provides a corresponding visual output on the display panel 541 according to the type of touch event. Although in Figure 5 In the embodiment, the touch panel 531 and the display panel 541 are used as two independent components to implement the input and output functions of the wearable device, but in some embodiments, the touch panel 531 and the display panel 541 can be integrated to implement the input and output functions of the wearable device.

[0160] The wearable device may also include at least one sensor 550, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 541 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 541 and / or the backlight when the wearable device is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of wearable devices (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometers, tapping), etc.; as for other sensors that can be configured on wearable devices, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0161] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the wearable device. The audio circuit 560 can transmit the received audio data to the speaker 561 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and converted into audio data, and then the audio data is output to the processor 580 for processing, and then sent to another wearable device through the RF circuit 510, or the audio data is output to the memory 520 for further processing.

[0162] Wi-Fi is a short-range wireless transmission technology. The wearable device can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 570, which provides users with wireless broadband Internet access. Figure 5 A Wi-Fi module 570 is shown, but it is understandable that it is not a necessary component of the wearable device and can be omitted as needed without changing the essence of the invention.

[0163] The processor 580 is the control center of the wearable device. It uses various interfaces and lines to connect various parts of the entire wearable device. By running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520, it performs various functions of the wearable device and processes data, thereby monitoring the wearable device as a whole. Optionally, the processor 580 may include one or more processing units; preferably, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 580.

[0164] The wearable device also includes a power source 590 (such as a battery) for supplying power to each component. Preferably, the power source can be logically connected to the processor 580 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0165] Although not shown, the wearable device may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0166] In the embodiment of the present application, the processor 580 is used to obtain the positioning information of the wearable device;

[0167] The RF circuit 510 is used to send the positioning information to the XR device to instruct the positioning information to be superimposed and displayed on the current display interface of the XR device.

[0168] Optionally, the positioning information is also used to instruct the XR device to perform positioning tracking in combination with the current real environment.

[0169] Optionally, the processor 580 is further configured to respond to a help message sent by a user of the wearable device; or, in the case where it is detected that the current real environment is a dangerous environment, generate the help message; or, in the case where it is detected that the heart rate of the user of the wearable device exceeds a preset heart rate and the duration exceeds a preset duration, generate the help message;

[0170] The RF circuit 510 is used to send the help information to the XR device, where the help information is used by the XR device to perform positioning tracking according to the positioning information.

[0171] Optionally, the type of the help information includes text, picture, audio or video.

[0172] like Figure 6 FIG. 1 is a schematic diagram of an XR device in an embodiment of the present application, which may include:

[0173] A memory 601 storing executable program codes;

[0174] A processor 602 and a transceiver 603 coupled to the memory 601, and a display 604;

[0175] The transceiver 603 is used to receive the positioning information sent by the wearable device;

[0176] The display 604 is used to display the positioning information on a superimposed manner on the current display interface.

[0177] Optionally, the processor 602 is further configured to perform positioning tracking according to the positioning information and the current real environment.

[0178] Optionally, the processor 602 is specifically configured to generate a tracking path according to the positioning information and the current real environment; and output the tracking path.

[0179] Optionally, the transceiver 603 is further configured to receive a help message sent by the wearable device;

[0180] The display 604 is used to display the help information superimposed on the current display interface.

[0181] Optionally, the processor 602 is specifically configured to generate a tracking path according to the positioning information and the current real environment when receiving the help information;

[0182] The display 604 is further configured to overlay and display the tracking path on the current display interface.

[0183] Optionally, the type of the help information includes text, picture, audio or video.

[0184] AR is a technology that perfectly overlays virtual information with the real world, and can display virtual information such as text, images, videos, and 3D models to users without affecting real life. XR devices can include AR glasses. To achieve this effect, AR glasses (or AR head-mounted devices) usually have two display solutions: Video see-through and Optical see-through.

[0185] (1)Video see-through

[0186] Video see-through is a display solution based on video synthesis technology. This solution first needs to use a camera to obtain the picture information of the real scene, and then after information processing, it synthesizes virtual information such as text, images, videos, 3D models into the picture of the real scene, and finally displays the synthesized picture to the user through the display screen.

[0187] (2)Optical see-through

[0188] Optical see-through is a display solution based on optical components. In this solution, the micro-projection system (optical machine) first projects virtual information such as text, images, videos, 3D models, etc. directly onto the optical components, and then sends the virtual information to the human eye through reflection and total reflection. At the same time, the images in the real scene will directly enter the human eye through the optical components. In this way, users can see the virtual information while seeing the real scene.

[0189] Since AR glasses are highly integrated devices, no matter which display solution is chosen, multiple modules need to cooperate with each other to achieve the AR effect of virtual-real integration. AR devices may include: shooting unit, processor, display and interaction unit, and transceiver.

[0190] The shooting unit is equivalent to the "eyes" of AR glasses, which is mainly responsible for capturing the image information of the real environment and obtaining the user's gesture information, etc. Its hardware equipment mainly includes cameras.

[0191] AR glasses are usually equipped with multiple cameras, and each camera has a different division of labor. For example, when a user uses AR glasses, the camera used to shoot the real environment will obtain information about the real environment from the user's perspective in real time, so as to provide rich picture information to the processor.

[0192] The processor is equivalent to the "brain" of AR glasses, which is mainly responsible for processing all information transmitted by various devices. Its hardware devices mainly include processors and memory.

[0193] On the one hand, the processor needs to analyze and process the real environment information transmitted by the shooting unit, and generate corresponding virtual information at the correct location by combining intelligent recognition technology with Simultaneous Localization and Mapping (SLAM) technology. On the other hand, the processor also needs to analyze and process the instructions issued by the user, and then change the display of virtual information in real time according to the user's intention.

[0194] The display is the core unit of AR glasses, which is mainly responsible for sending virtual information to the human eye and producing a visual effect of virtual-real fusion. As for the Optical see-through display solution, the hardware equipment of its display mainly includes micro-projection system and optical components. The micro-projection system mainly includes: Liquid Crystal on Silicon (LCOS), Digital Light Processing (DLP), light-emitting diode (LED) miniaturization technology (Micro LED), etc. Optical components mainly include: prism, free-form surface, Birdbath, array optical waveguide, diffraction optical waveguide, etc.

[0195] Whether an AR glasses can achieve good AR effects depends on the performance of its display. From the perspective of user needs, the display of AR glasses has three key points and development trends: light and comfortable, strong immersion, and reasonable price.

[0196] Since the diffraction optical waveguide based on surface relief grating has the advantages of being light and thin, having a large field of view, a large eye movement range, and low mass production cost, it is generally considered to be the mainstream display technology route for AR glasses and is hailed by the industry as the "golden key to opening the AR glasses market."

[0197] The interaction unit is an indispensable unit in AR glasses. It is mainly responsible for collecting relevant information about user interaction so that users can interact with virtual information in real time. Its hardware devices mainly include microphones, eye tracking sensors, as well as rings, wristbands, etc. Based on rich interactive hardware, users can control the virtual images projected by AR glasses through voice, gestures, eye movements and other interactive methods. As a result, AR glasses can not only free the user's hands, but also bring users a touchable and interactive upgraded interactive experience.

[0198] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0200] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0201] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0202] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0203] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0204] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for displaying positioning information, applied to a wearable device, characterized in that: The method comprises: Obtaining positioning information of the wearable device; The positioning information is sent to the XR device to instruct the positioning information to be superimposed and displayed on a current display interface of the XR device.

2. The method according to claim 1, characterized in that: The method further comprises: In response to a help message sent by the user of the wearable device; or, in the case where it is detected that the current real environment is a dangerous environment, the help message is generated; or, in the case where it is detected that the heart rate of the user of the wearable device exceeds a preset heart rate and the duration exceeds a preset duration, the help message is generated; The help information is sent to the XR device, where the help information is used to instruct the XR device to perform positioning tracking according to the positioning information.

3. A method for displaying positioning information, applied to an XR device, characterized in that: The method comprises: Receive positioning information sent by wearable devices; The positioning information is displayed superimposed on the current display interface.

4. The method according to claim 3, characterized in that The method further comprises: Positioning and tracking are performed according to the positioning information and the current real environment.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: Receiving a help message sent by the wearable device; The help information is displayed in a superimposed manner on the current display interface.

6. The method according to claim 5, characterized in that Performing positioning tracking according to the positioning information and the current real environment includes: When receiving the help information, generating a tracking path according to the positioning information and the current real environment; The tracking path is displayed in a superimposed manner on the current display interface.

7. A wearable device, characterized in that: include: An acquisition module is used to obtain the positioning information of the wearable device; A sending module is used to send the positioning information to the XR device, and the positioning information is used to overlay and display the positioning information on the current display interface of the XR device.

8. An XR device, characterized in that: include: A receiving module, used for receiving positioning information sent by a wearable device; The display module is used to display the positioning information on a superimposed manner on the current display interface.

9. A wearable device, characterized in that: include: A memory storing executable program code; a processor and a transceiver coupled to the memory; The wearable device is used to execute the method according to claim 1 or 2.

10. An XR device, characterized in that: include: A memory storing executable program code; a processor and a transceiver coupled to the memory, and a display; The XR device is used to execute the method according to any one of claims 3 to 6.

11. A computer-readable storage medium comprising instructions, which, when executed on a processor, causes the processor to execute the method according to any one of claims 1 or 2, or 3-6.

Images