Positioning method of wearable device, wearable device and electronic device

Abstract

The application discloses a positioning method of a wearable device, a wearable device and an electronic device, and belongs to the field of XR. The method comprises the following steps: an electronic device acquires relevant information of a ray emitted by a first wearable device, wherein the relevant information comprises direction information of the ray, length information of the ray, and first position information of a first light spot formed by the ray; second position information of a second light spot formed by a light-emitting device of the first wearable device in a light-emitting state is acquired; three-degree-of-freedom position information of the first wearable device is determined according to the relevant information of the ray and the second position information; and six-degree-of-freedom pose information of the first wearable device is determined according to three-degree-of-freedom attitude information of the first wearable device and the three-degree-of-freedom position information.

Description

Technical Field

[0001] This application belongs to the field of communication technology, specifically relating to a positioning method for wearable devices, wearable devices, and electronic devices. Background Technology

[0002] In extended reality (XR) applications, the most basic requirement for devices is the ability to provide lightweight and natural interaction methods. "Lightweight" emphasizes a compact physical structure, while "natural" requires the ability to perceive spatial pose. Currently, lightweight XR devices such as rings often only provide three degrees of freedom (DoF) attitude information due to the integration drift problem caused by networked inertial measurement units (IMUs), meaning they lack position information. Summary of the Invention

[0003] The purpose of this application is to provide a positioning method for wearable devices, a wearable device, and an electronic device, in order to solve the problem that the prior art cannot determine the location information of lightweight devices in the XR field.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a positioning method for a wearable device, including:

[0006] The electronic device acquires relevant information about the rays emitted by the first wearable device, including: direction information of the rays, length information of the rays, and first position information of the first spot formed by the rays;

[0007] Obtain the second position information of the light-emitting device of the first wearable device forming a second light spot when it emits light;

[0008] Based on the relevant information of the ray and the second position information, the three-degree-of-freedom position information of the first wearable device is determined;

[0009] Based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is determined.

[0010] Secondly, embodiments of this application provide a wearable device, including:

[0011] Laser emitter: Used to emit rays towards a target object;

[0012] Laser receiver: used to receive the reflected signal of the ray from the target object;

[0013] Light-emitting devices;

[0014] Distance sensor: used to determine the length information of the ray;

[0015] Inertial measurement unit: used to determine the direction information of the ray;

[0016] The first transmitting module is used to transmit the length information and direction information of the ray to the electronic device.

[0017] Thirdly, embodiments of this application provide an electronic device, including:

[0018] The first acquisition module is used to acquire relevant information about the rays emitted by the first wearable device, including: the direction information of the rays, the length information of the rays, and the first position information of the first light spot formed by the rays;

[0019] The second acquisition module is used to acquire the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light.

[0020] The first determining module is used to determine the three-degree-of-freedom position information of the first wearable device based on the relevant information of the ray and the second position information;

[0021] The second determining module is used to determine the six-degree-of-freedom pose information of the first wearable device based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information of the first wearable device.

[0022] Fourthly, embodiments of this application provide a positioning system, including an electronic device and a first wearable device;

[0023] The first wearable device emits a ray towards the target object via a laser emitter, determines the length of the ray via a ranging sensor, and determines the direction of the ray via an inertial measurement unit; it then sends the length and direction information of the ray to the electronic device; the first wearable device controls a light-emitting device to emit light.

[0024] The electronic device acquires the length information of the ray, the direction information of the ray, and the first position information of the first light spot formed by the ray, and acquires the second position information of the second light spot formed by the light-emitting device when it emits light;

[0025] The electronic device determines the three-degree-of-freedom position information of the first wearable device based on the length information of the ray, the direction information of the ray, the first position information, and the second position information; and determines the six-degree-of-freedom pose information of the first wearable device based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information.

[0026] Fifthly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0027] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0028] In a seventh aspect, embodiments of this application also provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0029] In the embodiments of this application, the electronic device acquires relevant information about the rays emitted by the first wearable device and the position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light. Based on the relevant information about the rays and the position information of the second light spot, the three-degree-of-freedom position information of the first wearable device is determined. Combined with the three-degree-of-freedom attitude information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is obtained. This realizes the acquisition of three-dimensional information of the position and attitude of the first wearable device, thereby enabling motion-based interaction of the first wearable device. Attached Figure Description

[0030] Figure 1 This is one of the flowcharts illustrating the positioning method of a wearable device according to an embodiment of this application;

[0031] Figure 2 This is a schematic diagram of an XR application scenario according to an embodiment of this application;

[0032] Figure 3 This is a schematic diagram of the algorithm framework of the first wearable device according to an embodiment of this application;

[0033] Figure 4 This is a schematic diagram illustrating the visual positioning principle of an embodiment of this application;

[0034] Figure 5 One of the schematic diagrams showing the structure of the ring according to an embodiment of this application;

[0035] Figure 6 The second schematic diagram illustrating the structure of the ring according to an embodiment of this application;

[0036] Figure 7 This is a schematic diagram showing the structure of a helmet system according to an embodiment of this application;

[0037] Figure 8 A second schematic flowchart illustrating the positioning method of a wearable device according to an embodiment of this application;

[0038] Figure 9 The third flowchart illustrates the positioning method of a wearable device according to an embodiment of this application;

[0039] Figure 10 This is a schematic diagram showing the control waveforms of an embodiment of this application;

[0040] Figure 11 A schematic diagram illustrating the structure of a positioning device for a wearable device according to an embodiment of this application;

[0041] Figure 12 One of the schematic diagrams illustrating the structure of an electronic device according to an embodiment of this application;

[0042] Figure 13 This is a second schematic diagram illustrating the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The terms "first" and "second" in the specification and claims of this application may explicitly or implicitly include one or more of the features. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0045] like Figure 1 As shown, this application provides a positioning method for a wearable device, including:

[0046] Step 101: The electronic device acquires relevant information about the rays emitted by the first wearable device, including: the direction information of the rays, the length information of the rays, and the first position information of the first light spot formed by the rays;

[0047] Step 102: Obtain the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light;

[0048] Step 103: Determine the three-degree-of-freedom position information of the first wearable device based on the relevant information of the ray and the second position information;

[0049] Step 104: Determine the six-degree-of-freedom pose information of the first wearable device based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information.

[0050] In this embodiment, the positioning method of the wearable device is executed by an electronic device, which may be a wireless computing platform with a camera, such as a second wearable device, a personal computer (PC), or a mobile terminal. The second wearable device may be a head-mounted display device, such as a virtual reality (VR) headset. The first wearable device may be a lightweight device, such as an augmented reality (AR) / VR ring.

[0051] Optionally, the first wearable device includes a laser rangefinder, a laser emitter, a laser receiver, and a light-emitting device. The laser emitter emits a ray towards a target object, the laser receiver receives the reflected signal, and the laser rangefinder measures the distance based on the signals from the laser emitter and receiver to determine the length of the ray emitted by the first wearable device. The first wearable device also includes an IMU and an MCU. The IMU determines the three-degree-of-freedom (DOF) attitude information of the first wearable device, allowing the MCU to determine the direction of the emitted ray based on this information. The first wearable device also includes a light-emitting device, such as a light-emitting diode (LED). The electronic device can acquire the position of the light spot emitted by this device.

[0052] The electronic device can obtain the direction and length of the ray emitted by the first wearable device, as well as the position of the first light spot formed by the ray mapping onto the target object. It can also obtain the position of the second light spot formed by the light-emitting device of the first wearable device when it emits light. Thus, it can calculate the three-degree-of-freedom position information of the first wearable device. Combined with the obtained three-degree-of-freedom attitude information of the first wearable device, it can obtain the six-degree-of-freedom pose information of the first wearable device.

[0053] The six-degree-of-freedom pose information describes the three-degree-of-freedom position information (x, y, z) and the three-degree-of-freedom pose information (α, β, γ), which together constitute a complete 3D spatial coordinate system description. In general, the three degrees of freedom specifically refers to the three-degree-of-freedom pose information, which lacks the three-degree-of-freedom position information. Therefore, the embodiments of this application can locate the six-degree-of-freedom pose information of the first wearable device, and can realize the motion-type interaction of the first wearable device.

[0054] In the embodiments of this application, the electronic device acquires relevant information about the rays emitted by the first wearable device and the position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light. Based on the relevant information about the rays and the second position information, the three-degree-of-freedom position information of the first wearable device is determined. Combined with the three-degree-of-freedom attitude information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is obtained. This realizes the acquisition of three-dimensional information of the position and attitude of the first wearable device, thereby enabling motion-based interaction of the first wearable device.

[0055] Optionally, the method further includes: receiving three-degree-of-freedom attitude information sent by the first wearable device. The IMU of the first wearable device can obtain the three-degree-of-freedom attitude information of the first wearable device.

[0056] As an optional embodiment, obtaining the direction and length information of a ray emitted by the first wearable device includes: receiving the direction and length information of the ray sent by the first wearable device; wherein the direction information of the ray is determined based on the three-degree-of-freedom attitude information of the first wearable device, and the length information of the ray is obtained by measurement by a ranging sensor.

[0057] In this embodiment, the IMU can obtain the three-degree-of-freedom attitude information of the first wearable device, the direction information and length information of the ray emitted by the first wearable device. The first wearable device can send one or more of the above information to the electronic device, or it can directly send the data read by the IMU (such as the direction information and length information of the ray) to the electronic device, and the electronic device can calculate the three-degree-of-freedom attitude information and the three-degree-of-freedom position information of the first wearable device.

[0058] In this embodiment, a typical XR application scenario is as follows: Figure 2As shown in the diagram, the electronic device is a head-mounted display (HUD) 200, and the first wearable device is a ring-shaped main structure 202 with LEDs. An infrared laser emitting module 203 (for emitting rays) is set on the ring-shaped main structure 202. The rays emitted by the infrared laser emitting module 203 are mapped onto the surface contour 201 of the XR application scene to form a first light spot 205. 204 is the observation line from the ring LED to the HUD camera, 206 is the observation line from the first light spot to the HUD camera, and 207 is the laser ranging line. The positioning scheme for the first wearable device is Visual Inertial Odometry (VIO), so the positioning process can include three implementation processes: inertial navigation, vision, and fusion.

[0059] For the inertial navigation process, the core component of this part is the IMU of the first wearable device, such as... Figure 3 As shown, the IMU consists of an accelerometer, a gyroscope, and a magnetometer. Two types of information can be obtained from the IMU's observation data: a) transient motion information directly observable through readings (mainly angular velocity ω, acceleration a, and geomagnetic field m); b) three-degree-of-freedom (DOF) attitude information of the first wearable device (rotation angles α, β, and γ about the Earth's surface coordinates) that can be indirectly calculated. The IMU can calculate the DDF attitude information of the first wearable device and send it to the head-mounted display (HMD), or the first wearable device can directly send the information observed in step a) to the HMD, which will then calculate the DDF attitude information. The method by which the HMD obtains the DDF attitude information of the first wearable device is not limited here and can be flexibly adjusted according to the system computing power, power consumption, latency, and other requirements of both the HMD and the first wearable device.

[0060] The calculation for a) can be obtained by linearly transforming the raw readings from the device datasheet. The calculation for b) can be estimated using algorithms similar to those in Attitude and Heading Reference Systems (AHRS), such as... Figure 3 The algorithm shown is an Extended Kalman Filter (EKF) class algorithm.

[0061] Since the position of the laser emitter relative to the IMU is determined by the ring-shaped rigid body structure, the emission direction of the laser emitter can be calculated using the extrinsic parameter [R|t] between the two after estimating the three-degree-of-freedom attitude of the IMU. This extrinsic parameter can be obtained through offline calibration at the factory, and will not be elaborated upon here.

[0062] As an optional embodiment, obtaining the first position information of the first light spot formed by the ray emitted by the first wearable device includes: extracting the first position information of the first light spot formed by the ray mapping onto the target object based on the scene image.

[0063] In this embodiment, the electronic device can acquire a scene image of the current XR scene, which may include a visible light image and an infrared image. Specifically, the laser emitter of the first wearable device emits a ray that forms a first light spot on the target object. The electronic device can determine the position of this first light spot based on the acquired infrared image, and thus determine the position of the first wearable device in the infrared image.

[0064] As an optional embodiment, obtaining the second position information of the second light spot formed by the light-emitting device of the first wearable device under illumination includes: determining the imaging point mapped onto the camera by the light-emitting device under illumination based on the scene image; and determining that the position information of the imaging point is the second position information of the second light spot formed by the light-emitting device. In this embodiment, the light-emitting device disposed on the first wearable device maps to form a second light spot in the imaging area of ​​the camera, and the electronic device can determine the position of the second light spot based on the visible light image.

[0065] For the visual component of the positioning process in this application embodiment, the core device is a multi-view camera system, such as the multi-view camera system of an AR helmet. This multi-view camera system can be installed on the electronic device, for example, on a helmet. The helmet can detect the infrared and visible light images of the first wearable device through this multi-view camera system. For simplicity, it is abstracted here as follows: Figure 4 The single-camera system shown. Where X... c Y c Z c Let X be the abstracted helmet camera coordinate system. w Y w Z w Using the world coordinate system, X I Y I Z I Let L be the laser emission line, and I be the IMU coordinate system. I and I B These are the imaging points on the camera for the first wearable device (such as the ring body) and the first spot of the laser emission line. The electronic device can determine the specific location of the imaging point based on the scene image of the current scene.

[0066] As an optional embodiment, determining the three-degree-of-freedom position information of the first wearable device based on the relevant information of the ray and the second position information includes: determining an observation plane based on the camera origin, the first position information, and the second position information; determining the depth of the first wearable device based on the observation plane; and determining the three-degree-of-freedom position information of the first wearable device based on the depth.

[0067] In this embodiment, the electronic device can determine the observation plane based on the detected infrared and visible light images. The electronic device can determine the depth of the first wearable device based on the observation plane and the length information of the rays, and determine the three-degree-of-freedom position information of the first wearable device based on the depth of the first wearable device and the position of the second light spot formed by the light-emitting device of the first wearable device in the image.

[0068] like Figure 4 As shown, the camera origin O c and imaging point I I I B This constitutes a ray O c I I and O c I B The observation plane π is formed; the laser emission line L must fall on the observation plane π. Since the IMU of the first wearable device can acquire three-degree-of-freedom attitude information, it can determine the direction of ray L. The ray length is different at different positions on the observation plane π, therefore, the length of ray L needs to be determined to further determine the depth of the first wearable device, and thus determine the three-degree-of-freedom position of the first wearable device. Fusing the three-degree-of-freedom attitude information with the three-degree-of-freedom position information yields a six-degree-of-freedom pose, which includes both the attitude and position of the first wearable device.

[0069] The following example, using a lightweight ring as the first wearable device and an AR helmet system as the electronic device, illustrates the positioning process of the lightweight ring. The AR helmet system includes a head-mounted XR device.

[0070] The structure of a lightweight ring is as follows Figure 5As shown, the lightweight ring includes: an electronic component structure 500, a laser receiver 501, a laser emitter 502, a ring-shaped battery 503, a finger sleeve structure 504, and a light-emitting device 505. The light-emitting device can be an LED. The electronic component structure 500 may include: a touch sensor (Touch / IO), an IMU, and a microcontroller unit (MCU). The laser receiver 501 can be a complementary metal-oxide-semiconductor (CMOS) or a single-photon avalanche diode (SPAD); the laser emitter 502 can be a vertical-cavity surface-emitting laser (VCSEL). The lightweight ring also includes a laser rangefinder sensor, which can perform ray ranging based on the laser receiver 501 and the laser emitter 502.

[0071] The system structure of this lightweight ring is as follows: Figure 6 As shown, it includes a central processing unit (CPU), storage module, IMU, power supply module, laser rangefinder module, I / O, and LED driver module. An AR helmet system is as follows: Figure 7 As shown, it includes a CPU, storage module, IMU, power supply module, camera module, I / O, and digital signal processor (DSP).

[0072] The helmet system can fuse information transmitted through the ring and information acquired by the camera to output a final six-degree-of-freedom positioning result. The information transmitted by the ring includes: ray length information, direction information, the first position information of the first light spot formed by the ray, the second position information of the second light spot formed by the light-emitting device, and three-degree-of-freedom attitude information. The information acquired by the camera includes infrared images and visible light images. The positioning principle is as follows:

[0073] 1) Without loss of generality, we assume that the system operates under the premise that the depth of the monocular camera is unknown.

[0074] 2) Camera origin O c and imaging point I I I B This constitutes a ray O c I I and O c I B The observation plane π is formed; the laser emission line L emitted by the ring must fall on the observation plane π; I IIt refers to the imaging point on the camera mapped by the light-emitting device of the ring; I B It is the imaging point on the camera that is mapped onto the first spot B formed by the laser emission line L.

[0075] 3) The ring's IMU can provide the estimation results of the three degrees of freedom attitude, that is, the three degrees of freedom attitude information of the ring. The ring or helmet system calculates the direction of the laser emission line L based on the three degrees of freedom attitude estimation results.

[0076] 4) Since the depth of the ring or the ring's inertial measurement unit is unknown, any straight line parallel to the L direction on the observation plane π satisfies the above 2) and 3). Therefore, additional constraints need to be introduced to determine the position of the ring or the ring's IMU.

[0077] 5) In this system, the constraint used to determine the ring's position originates from the laser rangefinder's measurement of the length of ray L; that is, the laser rangefinder on the ring can obtain the length information of ray L. Based on the observation plane π, only at a specific depth can ray L simultaneously satisfy both direction and length constraints, thus determining the position of the ring or its IMU on ray O. c I I The actual location on it.

[0078] 6)O I O I ′ The IMU indicating a ring or ring in the X-ray O c I I The possible position on the ring, its coordinates are in the camera coordinate system, and the actual position of the ring or the ring's IMU is determined to be O. I Then, O I Projected onto the world coordinate system XYZ w In the middle, we get O I The position coordinates (x, y, z) in the world coordinate system represent the three-degree-of-freedom position information of the ring. Combined with the three-degree-of-freedom attitude information of the ring provided by the IMU, the complete 6DoF positioning result can be obtained, that is, the six-degree-of-freedom pose information of the ring can be obtained.

[0079] The complete positioning calculation process is as follows: Figure 8 As shown, the ranging module, specifically the laser rangefinder of the ring, measures the length of ray L. The ring's IMU measures the three-degree-of-freedom attitude information of the ring and determines the direction of ray L. The helmet system extracts the position of the second light spot formed by the LED emission on the ring from the visible light image obtained by the camera system, and extracts the position of the first light spot mapped from ray L onto the target object from the infrared image obtained by the camera system. The observation plane π is determined based on the position of the second light spot and the position of the first light spot of ray L.

[0080] The helmet system can determine the depth of the ring based on the length and direction of ray L and the observation plane π, and calculate the three-degree-of-freedom position information of the ring based on the ring depth. Based on the three-degree-of-freedom position information and the three-degree-of-freedom attitude information of the ring, the six-degree-of-freedom pose information of the ring is determined.

[0081] The workflow of the ring system and helmet system in this application is as follows: Figure 9 As shown, for the ring system side, the process includes: configuring drive parameters, reading IMU data, calculating three-degree-of-freedom (DOF) attitude information, and transmitting the three-DOF attitude information to the helmet system (or adjusting the signal parameters if they do not meet predetermined conditions before wireless transmission). For the helmet system side, the process includes: loading internal and external parameters, configuring drive parameters, calculating the position of the light spot and the three-DOF position information of the ring through image processing and vision algorithms, receiving the three-DOF attitude information sent by the ring system, and obtaining the six-DOF pose information of the ring by fusing the three-DOF position information and the three-DOF attitude information.

[0082] It should be noted that, in the embodiments of this application, the color, frequency and intensity of the light emitted by the light-emitting device on the first wearable device can be adjusted as needed while meeting the requirements of visible light imaging.

[0083] Because the camera system of electronic devices needs to simultaneously observe the infrared light emitted by the light-emitting device and the laser emitter, a common driving method is to adjust the exposure control to an interleaved mode. Taking an LED as the light-emitting device as an example, specifically, a longer time can be reserved in odd-numbered frames to ensure good observation of visible light, while the LED is lit and the laser is turned off to prevent the identification of the visible light of the ring LED; when even-numbered frames arrive, the laser is lit and the ring LED is turned off, and the exposure time is significantly shortened to improve the signal-to-noise ratio of infrared imaging. A typical control waveform is as follows: Figure 10 As shown.

[0084] Therefore, the following points can be considered when controlling light-emitting devices:

[0085] 1) The illumination time window for the light-emitting device should fall within the odd-numbered frame exposure time window; otherwise, the image will be too dark or even unobservable. Similarly, the exposure time for the laser should also fall within the even-numbered frame exposure window. In special cases, even-numbered frames can also be used to attempt to illuminate the light-emitting device to obtain more optional configurations, but obviously, even-numbered frame exposure windows should be avoided.

[0086] 2) The duty cycle and intensity of the light-emitting device pulse jointly determine the appearance brightness. Under the premise of meeting 1) and power consumption requirements, the ratio of the two can be appropriately balanced.

[0087] 3) The color selection of the light-emitting device can refer to the wavelength gain curve of the helmet camera module. Try to select the band (color) with higher gain to improve energy utilization efficiency.

[0088] In the embodiments of this application, the electronic device acquires relevant information about the rays emitted by the first wearable device and the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light. Based on the relevant information about the rays and the second position information, the three-degree-of-freedom position information of the first wearable device is determined. Combined with the three-degree-of-freedom attitude information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is obtained. This realizes the acquisition of three-dimensional information of the position and attitude of the first wearable device, thereby enabling motion-based interaction of the first wearable device.

[0089] This application also provides a wearable device, which may be a first wearable device, such as an AR / VR ring.

[0090] like Figure 5 As shown, the wearable device includes:

[0091] Laser emitter 502: Used to emit rays toward a target object;

[0092] Laser receiver 501: Used to receive the reflected signal of the ray from the target object;

[0093] Light-emitting device 505; the light-emitting device is, for example, an LED;

[0094] Distance sensor ( Figure 5 (Not shown in the image): Information used to determine the length of the ray;

[0095] Inertial Measurement Unit (IMU): Used to determine the direction information of the ray;

[0096] First sending module ( Figure 5 (Not shown in the image), used to send the length information and direction information of the ray to the electronic device.

[0097] Optionally, the wearable device controls the laser emitter to emit rays toward the target object and controls the light-emitting device to emit light;

[0098] The wearable device determines the length information of the ray through the ranging sensor and the direction information of the ray through the inertial measurement unit;

[0099] The wearable device transmits the length information and direction information of the ray.

[0100] Optionally, determining the length information of the ray using the ranging sensor includes:

[0101] The laser receiver receives the reflection signal of the ray from the target object;

[0102] The range sensor is controlled to determine the length information of the ray based on the ray and the reflected signal.

[0103] In this embodiment, a laser emitter can emit a ray towards a target object, a laser receiver can receive the reflected signal, and a laser rangefinder can measure the distance based on the signals from the laser emitter and receiver to determine the length of the ray emitted by the first wearable device. An IMU can determine the three-degree-of-freedom attitude information of the wearable device. Optionally, the wearable device may further include an MCU, which can determine the direction information of the ray emitted by the first wearable device based on the three-degree-of-freedom attitude information. The wearable device transmits the length and direction information of the ray to an electronic device via a first transmitting module.

[0104] An MCU can control a light-emitting device to emit light, and electronic devices can determine the position of a second light spot formed when the light-emitting device emits light.

[0105] Optionally, determining the direction information of the ray through the inertial measurement unit includes:

[0106] Determine the three-degree-of-freedom attitude information of the wearable device; determine the direction information of the ray based on the three-degree-of-freedom attitude information.

[0107] In this embodiment, the IMU can read the three-degree-of-freedom (DOF) attitude information of the first wearable device and determine the direction of the ray based on the DDF attitude information. Optionally, the MCU can determine the direction information of the ray emitted by the wearable device based on the DDF attitude information.

[0108] Optionally, the wearable device may also include an MCU, which can determine the direction information of the rays emitted by the wearable device based on the three-degree-of-freedom attitude information.

[0109] In an embodiment of this application, an inertial measurement unit is provided on the wearable device. The inertial measurement unit can determine the three-degree-of-freedom attitude information of the wearable device, the direction information and length information of the rays emitted by the wearable device, and send the above information to an electronic device, which may be a head-mounted display device. The electronic device determines the three-degree-of-freedom attitude information and three-degree-of-freedom position information of the wearable device based on the above information, thereby determining the six-degree-of-freedom pose information of the wearable device.

[0110] like Figure 11 As shown in the illustration, this application also provides an electronic device 1100, comprising:

[0111] The first acquisition module 1110 is used to acquire relevant information about the rays emitted by the first wearable device. The relevant information includes: the direction information of the rays, the length information of the rays, and the first position information of the first light spot formed by the rays.

[0112] The second acquisition module 1120 is used to acquire the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light.

[0113] The first determining module 1130 is used to determine the three-degree-of-freedom position information of the first wearable device based on the relevant information of the ray and the second position information;

[0114] The second determining module 1140 is used to determine the six-degree-of-freedom pose information of the first wearable device based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information of the first wearable device.

[0115] Optionally, the first acquisition module includes:

[0116] A receiving unit is used to receive the direction and length information of the ray transmitted by the first wearable device;

[0117] The direction information of the ray is determined based on the three-degree-of-freedom attitude information of the first wearable device, and the length information of the ray is obtained by measuring the distance sensor.

[0118] Optionally, the first acquisition module includes:

[0119] The information extraction unit is used to extract the first position information of the first light spot formed by the ray mapping onto the target object based on the scene image.

[0120] Optionally, the second acquisition module includes:

[0121] The first determining unit is used to determine, based on the scene image, the imaging point that the light-emitting device maps onto the camera when it is emitting light;

[0122] The second determining unit is used to determine that the position information of the imaging point is the second position information of the second light spot formed by the light-emitting device.

[0123] Optionally, the first determining module includes:

[0124] The third determining unit is used to determine the observation plane based on the camera origin, the first position information, and the second position information;

[0125] The fourth determining unit is used to determine the depth of the first wearable device based on the observation plane;

[0126] The fifth determining unit is used to determine the three-degree-of-freedom position information of the first wearable device based on the depth.

[0127] Optionally, the electronic device further includes:

[0128] The receiving module is used to receive the three-degree-of-freedom attitude information sent by the first wearable device.

[0129] In the embodiments of this application, the electronic device acquires relevant information about the rays emitted by the first wearable device and the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light. Based on the relevant information about the rays and the second position information, the three-degree-of-freedom position information of the first wearable device is determined. Combined with the three-degree-of-freedom attitude information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is obtained. This realizes the acquisition of three-dimensional information of the position and attitude of the first wearable device, thereby enabling motion-based interaction of the first wearable device.

[0130] The wearable devices and head-mounted displays in this application embodiment can be electronic devices or components within electronic devices, such as integrated circuits or chips. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0131] The electronic device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0132] The electronic device provided in this application embodiment can achieve... Figures 1 to 10 The various processes that can be implemented by electronic devices in the method embodiments are not described in detail here to avoid repetition.

[0133] Optional, such as Figure 12 As shown, this application embodiment also provides an electronic device 1200, including a processor 1201, a memory 1202, and a program or instructions stored in the memory 1202 and executable on the processor 1201. When the program or instructions are executed by the processor 1201, they implement the various processes of the above-described wearable device positioning method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0134] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0135] Figure 13 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0136] The electronic device 1300 includes, but is not limited to, components such as: radio frequency unit 1301, network module 1302, audio output unit 1303, input unit 1304, sensor 1305, display unit 1306, user input unit 1307, interface unit 1308, memory 1309, and processor 1310.

[0137] Those skilled in the art will understand that the electronic device 1300 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1310 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 13 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0138] The processor 1310 is used to acquire relevant information about the rays emitted by the first wearable device, including: the direction information of the rays, the length information of the rays, and the first position information of the first spot formed by the rays;

[0139] Obtain the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light;

[0140] Based on the relevant information of the ray and the second position information, the three-degree-of-freedom position information of the first wearable device is determined;

[0141] Based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is determined.

[0142] Optionally, the radio frequency unit 1301 is specifically used for:

[0143] Receive the direction and length information of the ray sent by the first wearable device;

[0144] The direction information of the ray is determined based on the three-degree-of-freedom attitude information of the first wearable device, and the length information of the ray is obtained by measuring the distance sensor.

[0145] Optionally, the processor is specifically configured to: extract first position information of the first light spot formed by the ray mapping onto the target object based on the scene image.

[0146] Optionally, the processor is specifically used for:

[0147] The imaging point on the camera when the light-emitting device is emitting light is determined based on the scene image;

[0148] The location information of the imaging point is determined to be the second location information of the second light spot formed by the light-emitting device.

[0149] Optionally, the processor is further configured to:

[0150] The observation plane is determined based on the camera origin, the first position information, and the second position information;

[0151] The depth of the first wearable device is determined based on the observation plane;

[0152] The three-degree-of-freedom position information of the first wearable device is determined based on the depth.

[0153] Optionally, the radio frequency unit is further configured to: receive three-degree-of-freedom attitude information sent by the first wearable device.

[0154] In the embodiments of this application, the electronic device acquires relevant information about the rays emitted by the first wearable device and the second position information of the second light spot formed by the light-emitting device of the first wearable device when it emits light. Based on the relevant information about the rays and the second position information, the three-degree-of-freedom position information of the first wearable device is determined. Combined with the three-degree-of-freedom attitude information of the first wearable device, the six-degree-of-freedom pose information of the first wearable device is obtained. This realizes the acquisition of three-dimensional information of the position and attitude of the first wearable device, thereby enabling motion-based interaction of the first wearable device.

[0155] It should be understood that, in this embodiment, the input unit 1304 may include a graphics processing unit (GPU) 13041 and a microphone 13042. The GPU 13041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1306 may include a display panel 13061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1307 includes a touch panel 13071 and at least one of other input devices 13072. The touch panel 13071 is also called a touch screen. The touch panel 13071 may include a touch detection device and a touch controller. Other input devices 13072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0156] The memory 1309 can be used to store software programs and various data. The memory 1309 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1309 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1309 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0157] Processor 1310 may include one or more processing units; optionally, processor 1310 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1310.

[0158] This application also provides a positioning system, including an electronic device and a first wearable device;

[0159] The first wearable device emits a ray towards the target object via a laser emitter, determines the length of the ray via a ranging sensor, and determines the direction of the ray via an inertial measurement unit; it then sends the length and direction information of the ray to the electronic device; the first wearable device controls a light-emitting device to emit light.

[0160] The electronic device acquires the length information of the ray, the direction information of the ray, and the first position information of the first light spot formed by the ray, and acquires the second position information of the second light spot formed by the light-emitting device when it emits light;

[0161] The electronic device determines the three-degree-of-freedom position information of the first wearable device based on the length information of the ray, the direction information of the ray, the first position information, and the second position information; and determines the six-degree-of-freedom pose information of the first wearable device based on the three-degree-of-freedom attitude information and the three-degree-of-freedom position information.

[0162] Optionally, the electronic device is a second wearable device or a personal computer (PC).

[0163] The structure of the electronic device and the first wearable device, as well as the operations they perform, will not be described in detail here.

[0164] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described wearable device positioning method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0165] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0166] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described wearable device positioning method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0167] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0168] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the positioning method embodiment of the wearable device described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0169] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

Claims

1. A positioning method of a wearable device, characterized by, The method comprises: An electronic device obtains related information of a ray emitted by a first wearable device, the related information comprising: direction information of the ray, length information of the ray, and first position information of a first light spot formed by the ray; Obtaining second position information of a second light spot formed by a light emitting device of the first wearable device in a light emitting state; Determining three-degree-of-freedom position information of the first wearable device according to the related information of the ray and the second position information; Determining six-degree-of-freedom pose information of the first wearable device according to three-degree-of-freedom pose information of the first wearable device and the three-degree-of-freedom position information.

2. The method of claim 1, wherein, Obtaining direction information and length information of a ray emitted by a first wearable device comprises: Receiving direction information and length information of the ray sent by the first wearable device; The direction information of the ray is determined according to three-degree-of-freedom pose information of the first wearable device, and the length information of the ray is obtained by a distance measuring sensor.

3. The method of claim 1, wherein, Obtaining first position information of a first light spot formed by a ray emitted by a first wearable device comprises: Extracting first position information of a first light spot formed by the ray mapping to a target object according to a scene image.

4. The method of claim 1, wherein, Obtaining second position information of a second light spot formed by a light emitting device of the first wearable device in a light emitting state comprises: Determining an imaging point of the light emitting device mapping to a camera in a light emitting state according to a scene image; Determining position information of the imaging point as second position information of a second light spot formed by the light emitting device.

5. The method of claim 1, wherein, The method further comprises: Receiving three-degree-of-freedom pose information sent by the first wearable device. The method comprises: A laser transmitter for emitting a ray to a target object; 6. The method of claim 1, wherein, A laser receiver for receiving a reflection signal of the ray by the target object; A light emitting device; 7. A wearable device, comprising: A distance measuring sensor for determining length information of the ray according to the ray and the reflection signal; An inertial measurement unit for determining three-degree-of-freedom pose information of a wearable device, and determining direction information of the ray according to the three-degree-of-freedom pose information; A first sending module for sending the length information of the ray and the direction information of the ray to an electronic device; The length information of the ray and the direction information of the ray are used to determine three-degree-of-freedom position information of the wearable device.

8. The wearable device of claim 7, wherein: The wearable device controls the laser transmitter to emit a ray to a target object and controls the light emitting device to emit light; The wearable device determines the length information of the ray by the distance measuring sensor, and determines the direction information of the ray by the inertial measurement unit; The wearable device sends the length information of the ray and the direction information of the ray. ​ ​ ​ ​ 9. An electronic device, comprising: The method comprises the following steps: A first acquisition module is configured to acquire relevant information of a ray emitted by a first wearable device, wherein the relevant information comprises direction information of the ray, length information of the ray, and first position information of a first light spot formed by the ray; A second acquisition module is configured to acquire second position information of a second light spot formed by a light emitting device of the first wearable device in a light emitting state; A first determination module is configured to determine three-degree-of-freedom position information of the first wearable device according to the relevant information of the ray and the second position information; A second determination module is configured to determine six-degree-of-freedom pose information of the first wearable device according to three-degree-of-freedom attitude information of the first wearable device and the three-degree-of-freedom position information.

10. A positioning system, characterized by The method comprises the following steps: An electronic device and a first wearable device are provided; The first wearable device emits a ray to a target object through a laser emitter, determines length information of the ray through a distance measuring sensor, and determines direction information of the ray through an inertial measurement unit; the first wearable device sends the length information and the direction information of the ray to the electronic device; the first wearable device controls a light emitting device to emit light; The electronic device acquires the length information of the ray, the direction information of the ray, and first position information of a first light spot formed by the ray, and acquires second position information of a second light spot formed by the light emitting device in a light emitting state; 11. The positioning system of claim 10, wherein, The electronic device determines three-degree-of-freedom position information of the first wearable device according to the length information of the ray, the direction information of the ray, the first position information, and the second position information; the electronic device determines six-degree-of-freedom pose information of the first wearable device according to three-degree-of-freedom attitude information of the first wearable device and the three-degree-of-freedom position information. The electronic device is a second wearable device or a personal computer (PC).

Images