Energy-saving tracking system and tracking method

Abstract

The invention provides an energy-saving tracking system and an energy-saving tracking method. The tracking method includes: adjusting, by a first mobile device, at least one of a first power consumption of a sensor of the first mobile device and a second power consumption of a first light source of the first mobile device; the first mobile device performs at least one of the following: obtaining sensing data according to the adjusted first power consumption; and adjusting illumination of the first light source according to the adjusted second power consumption; and tracking, by the second mobile device, the first mobile device according to at least one of the sensing data and illumination of the first light source.

Description

Technical Field

[0001] This invention relates to extended reality (XR) technology, and more particularly to an energy-efficient tracking system and tracking method. Background Technology

[0002] In XR systems, head-mounted displays (HMDs) are often used in conjunction with portable devices. To interact with the portable device, the HMD needs to track its position. However, the tracking technology used in XR systems can be affected by environmental factors and may not perform well. Furthermore, tracking the portable device can consume a significant amount of power from either the HMD or the portable device. Summary of the Invention

[0003] This invention provides an energy-saving tracking system and method that can save power consumption of tracking mobile devices.

[0004] An energy-saving tracking system of the present invention includes a first mobile device and a second mobile device. The first mobile device includes a sensor and a first light source. The second mobile device is communicatively connected to the first mobile device. The first mobile device adjusts at least one of a first power consumption of the sensor and a second power consumption of the first light source, and performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption. The second mobile device tracks the first mobile device based on at least one of the sensing data and the illumination of the first light source.

[0005] In an embodiment of the present invention, the second mobile device described above includes an image capturing device, wherein the second mobile device detects illumination to determine whether the first light source is located in the field of view of the image capturing device, wherein in response to the first light source being located in the field of view, the first mobile device reduces the first power consumption of the sensor.

[0006] In an embodiment of the present invention, in response to the first light source being located outside the field of view, the first moving device reduces the second power consumption of the first light source.

[0007] In an embodiment of the present invention, the first moving device adjusts the second power consumption of the first light source so that the second power consumption is proportional to the distance between the first moving device and the second moving device.

[0008] In an embodiment of the present invention, the aforementioned sensing data includes the moving speed of the first mobile device, wherein the first mobile device adjusts the first power consumption of the sensor so that the first power consumption is proportional to the moving speed.

[0009] In an embodiment of the present invention, the aforementioned sensing data includes the moving speed of the first moving device, wherein the first moving device adjusts the second power consumption of the first light source so that the second power consumption is proportional to the moving speed.

[0010] In an embodiment of the present invention, the second mobile device includes an image capturing device, and the sensing data includes the moving speed of the first mobile device, wherein the second mobile device adjusts the third power consumption of the image capturing device so that the third power consumption is proportional to the moving speed.

[0011] In an embodiment of the present invention, the aforementioned sensing data includes the attitude of the first mobile device, wherein the first mobile device adjusts the second power consumption of the first light source according to the attitude.

[0012] In an embodiment of the present invention, the second mobile device described above includes an image capturing device, and the sensing data includes the attitude of the first mobile device, wherein the second mobile device adjusts the third power consumption of the image capturing device according to the attitude.

[0013] An energy-saving tracking method of the present invention includes: adjusting at least one of a first power consumption of a sensor of the first mobile device and a second power consumption of a first light source of the first mobile device by a first mobile device; performing at least one of the following by the first mobile device: acquiring sensing data based on the adjusted first power consumption; adjusting the illumination of the first light source based on the adjusted second power consumption; and tracking the first mobile device by a second mobile device based on at least one of the sensing data and the illumination of the first light source.

[0014] Based on the above, the tracking system of the present invention can dynamically adjust the power consumption of the sensors or light sources of the mobile device according to environmental information or the status of the mobile device, so as to achieve the purpose of accurately tracking the mobile device and saving energy. Attached Figure Description

[0015] Figure 1 A schematic diagram of an energy-saving tracking system is shown according to an embodiment of the present invention.

[0016] Figure 2 A schematic diagram illustrating the location of the light source within the field of view of the image capturing device, according to an embodiment of the present invention.

[0017] Figure 3 A schematic diagram illustrating a light source located outside the field of view of an image capturing device, according to an embodiment of the present invention.

[0018] Figure 4 According to an embodiment of the present invention, a schematic diagram is shown showing multiple light sources located in and out of the field of view of the image capturing device.

[0019] Figure 5A schematic diagram illustrating the relative positions of the mobile devices is shown in the embodiment of the present invention.

[0020] Figure 6 A flowchart of an energy-saving tracking method is illustrated according to an embodiment of the present invention. Detailed Implementation

[0021] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element symbols are used in the drawings and description to denote the same or similar parts.

[0022] Figure 1 A schematic diagram of an energy-efficient tracking system 10 according to an embodiment of the present invention is illustrated. The tracking system 10 may include a mobile device 100 and a mobile device 200, wherein the mobile device 200 is communicatively connected to the mobile device 100. In an embodiment, the mobile device 200 may be a head-mounted display device, and the mobile device 100 may be a portable device (e.g., a wireless controller) for operating or interacting with the head-mounted display device. The head-mounted display device may be worn on a user's head and may provide the user with an XR environment (or XR scene), such as a virtual reality (VR) environment, an augmented reality (AR) environment, or a mixed reality (MR) environment. The user may operate the portable device to transmit control commands to the head-mounted display device to interact with it. In an embodiment, the portable device may be used to capture the user's movements. The head-mounted display device may determine the user's current movement based on the portable device carried by the user.

[0023] Mobile device 100 may include processor 110, storage medium 120, transceiver 130, one or more light sources 140, and one or more sensors 150. Processor 110 may be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field-programmable gate array (FPGA), or other similar elements or combinations thereof. The processor 110 can be coupled to the storage medium 120, transceiver 130, light source 140 and sensor 150, and access and execute multiple modules and various applications stored in the storage medium 120.

[0024] Storage medium 120 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar components or combinations thereof, for storing multiple modules or various applications that can be executed by processor 110.

[0025] Transceiver 130 transmits or receives signals wirelessly or via a wired connection. Transceiver 130 may also perform operations such as low-noise amplification, impedance matching, mixing, up- or down-frequency conversion, filtering, amplification, and similar operations. Processor 110 can communicate with mobile device 200 via transceiver 130. Processor 110 can use transceiver 130 to transmit wireless signals (e.g., sensing data acquired by sensor 150) to or receive wireless signals (e.g., information acquired by image capture device 240 or sensing data acquired by sensor 250) from mobile device 200.

[0026] The light source 140 of the mobile device 100 can provide illumination that will be tracked by the mobile device 200. The light source 140 may include, but is not limited to, an infrared light-emitting diode (IR LED) or a laser device. The processor 110 can dynamically adjust the power consumption of the light source 140.

[0027] Sensor 150 can generate sensing data. Sensor 150 may include, but is not limited to, an image capture device, an electronic compass, a magnetometer, or an accelerometer (e.g., an inertial measurement unit, IMU). The mobile device 100 can perform self-tracking or pose estimation based on the sensing data. Processor 110 can dynamically adjust the power consumption of sensor 150. For example, processor 110 can reduce the power consumption of sensor 150 by reducing the frame rate or gain of sensor 150.

[0028] The mobile device 200 may include a processor 210, a storage medium 220, a transceiver 230, and an image capture device 240. In embodiments, the mobile device 200 may further include one or more sensors 250.

[0029] Processor 210 is, for example, a CPU, or other programmable general-purpose or special-purpose MCU, microprocessor, DSP, programmable controller, ASIC, GPU, ISP, IPU, ALU, CPLD, FPGA, or other similar components or combinations thereof. Processor 210 may be coupled to storage medium 220, transceiver 230, image capture device 240, and sensor 250, and access and execute multiple modules and various applications stored in storage medium 220.

[0030] Storage medium 220 may be, for example, any type of fixed or removable RAM, ROM, flash memory, HDD, SSD or similar element or combination thereof, for storing multiple modules or various applications that can be executed by processor 210.

[0031] Transceiver 230 transmits or receives signals wirelessly or via a wired connection. Transceiver 230 can also perform operations such as low-noise amplification, impedance matching, mixing, up- or down-frequency conversion, filtering, amplification, and similar functions. Processor 210 can communicate with mobile device 100 via transceiver 230. Processor 210 can use transceiver 230 to transmit wireless signals to or receive wireless signals from mobile device 100.

[0032] Image capture device 240 is, for example, a camera used to capture images. Image capture device 240 may include a photosensitive element such as a complementary metal oxide semiconductor (CMOS) or a charge-coupled device (CCD). Processor 210 can dynamically adjust the power consumption of image capture device 240. For example, processor 110 can reduce the power consumption of image capture device 240 by reducing the frame rate or gain of image capture device 240. In embodiments, the exposure time of image capture device 240 may be proportional to or inversely proportional to the gain of image capture device 240.

[0033] Sensor 250 can generate sensing data. Sensor 250 may include, but is not limited to, an image capture device, an electronic compass, a magnetometer, or an accelerometer. Mobile device 200 can perform self-tracking or attitude prediction based on the sensing data. Processor 210 can dynamically adjust the power consumption of sensor 250. For example, processor 210 can reduce the power consumption of sensor 250 by reducing the frame rate or gain of sensor 250.

[0034] When the mobile device 200 is an HMD, it may further include a display coupled to the processor 210. The display can be used to display image data, such as providing an XR environment or XR scene to a user wearing the HMD. The display may include a liquid crystal display (LCD) or an organic light-emitting diode (OLED) display. In one embodiment, the display can provide an image beam to the user's eyes to form an image on the user's retina, allowing the user to see the XR scene created by the HMD.

[0035] Mobile device 100 can acquire sensing data through sensor 150 and transmit the sensing data to mobile device 200. Mobile device 200 can determine the state of mobile device 100 or obtain environmental information about the location of mobile device 100 from the sensing data. Mobile device 200 can track the position or attitude of mobile device 100 based on the sensing data. Mobile device 200 can detect the illumination of light source 140 to track the position or attitude of mobile device 100. That is, mobile device 200 can track the position or attitude of mobile device 100 based on sensing data from mobile device 100 or illumination from light source 140.

[0036] In some cases, tracking the mobile device 100 based on sensor data or illumination from the light source 140 may produce undesirable results. The mobile device 100 can adjust the power consumption of the sensor 150 or the light source 140 accordingly. For example, when the mobile device 100 or the mobile device 200 determines that the current environment will distort the sensor data, the mobile device 100 can reduce the power consumption of the sensor 150 or not reduce the power consumption of the light source 140 (e.g., maintain or increase the power consumption of the light source 140) to avoid the distorted sensor data excessively affecting the tracking results of the mobile device 100. As another example, when the mobile device 100 or the mobile device 200 determines that the current posture of the mobile device 100 will make the illumination of the light source 140 difficult to detect, the mobile device 100 can reduce the power consumption of the light source 140 or not reduce the power consumption of the sensor 150 to enhance the influence of the sensor data on the tracking results of the mobile device 100.

[0037] In one embodiment, the image capture device 240 of the mobile device 200 can detect the illumination provided by the light source 140 to determine whether the light source 140 is within the field of view (FOV) of the image capture device 240 and generate a determination result. The mobile device 200 can transmit the determination result to the mobile device 100. The mobile device 100 can adjust the power consumption of the light source 140 or the power consumption of the sensor 150 according to the determination result. The mobile device 200 can adjust the power consumption of the image capture device 240 according to the determination result.

[0038] Figure 2 A schematic diagram illustrating the location of the light source 140 within the field of view of the image capturing device 240 according to an embodiment of the present invention is shown. When the light source 140 is within the field of view of the image capturing device 240, the moving device 100 may reduce the power consumption of the sensor 150 or not reduce the power consumption of the light source 140.

[0039] Figure 3A schematic diagram illustrating a light source 140 located outside the field of view of an image capturing device 240 is shown according to an embodiment of the present invention. When the light source 140 is outside the field of view of the image capturing device 240, the moving device 100 may reduce the power consumption of the light source 140 or not reduce the power consumption of the sensor 150.

[0040] Figure 4 A schematic diagram illustrating a plurality of light sources 140 located within and outside the field of view of an image capturing device 240, according to an embodiment of the present invention, is provided. Assume that the plurality of light sources 140 includes light source 141 and light source 142, wherein light source 141 is located within the field of view of the image capturing device 240, but light source 142 is located outside the field of view of the image capturing device 240. The moving device 100 can reduce the power consumption of light source 142 without reducing the power consumption of light source 141. That is, when multiple light sources 140 are present, the moving device 100 can selectively reduce the power consumption of a subset of the multiple light sources 140.

[0041] In this embodiment, the mobile device 100 can adjust the power consumption of the light source 140 or the sensor 150 based on the distance between the mobile device 100 and the mobile device 200. The mobile device 200 can adjust the power consumption of the image capture device 240 or the sensor 250 based on the aforementioned distance. The aforementioned distance can be measured by the sensor 150 of the mobile device 100 or by the image capture device 240 or the sensor 250 of the mobile device 200.

[0042] Figure 5 A schematic diagram illustrating the relative positions of the moving device 100 and the moving device 200 according to an embodiment of the present invention is shown. The moving device 100 can adjust the power consumption of the light source 140 so that the power consumption of the light source 140 is proportional to the distance between the moving device 100 and the moving device 200. For example, when the moving device 100 is in position A, the moving device 100 can adjust the power consumption of the light source 140 to 100%. When the moving device 100 is in position B, the moving device 100 can adjust the power consumption of the light source 140 to 50%.

[0043] In one embodiment, the mobile device 100 can adjust the power consumption of the light source 140 or the sensor 150 according to the moving speed of the mobile device 100, so that the power consumption is proportional to the moving speed of the mobile device 100. On the other hand, the mobile device 200 can adjust the power consumption of the image capturing device 240 or the sensor 250 according to the moving speed of the mobile device 100, so that the power consumption is proportional to the moving speed of the mobile device 100. The moving speed of the mobile device 100 can be measured by the sensor 150 of the mobile device 100 or by the image capturing device 240 or the sensor 250 of the mobile device 200.

[0044] For example, when the mobile device 100 moves at high speed, it can adjust the power consumption of the light source 140 to 50% and the frame rate of the sensor 150 to 60 frames per second (FPS). The mobile device 200 can adjust the frame rate of the image capture device 240 to 60 FPS, adjust the gain of the image capture device 240 to high gain, and adjust the exposure time of the image capture device 240 to 3 milliseconds. On the other hand, when the mobile device 100 moves at low speed, it can adjust the power consumption of the light source 140 to 20% and the frame rate of the sensor 150 to 15 FPS. The mobile device 200 can adjust the frame rate of the image capture device 240 to 30 FPS, adjust the gain of the image capture device 240 to low gain, and adjust the exposure time of the image capture device 240 to 6 milliseconds.

[0045] In this embodiment, the mobile device 100 can adjust the power consumption of the light source 140 or the sensor 150 according to the attitude of the mobile device 100 (e.g., a six-degree-of-freedom attitude). The mobile device 200 can adjust the power consumption of the image capture device 240 or the sensor 250 according to the attitude of the mobile device 100. The attitude of the mobile device 100 can be detected by the sensor 140 of the mobile device 100 or by the image capture device 240 or the sensor 250 of the mobile device 200.

[0046] For example, suppose the orientation of the mobile device 100 makes it difficult for the light source 140 of the mobile device 100 to be detected by the image capture device 240 of the mobile device 200. The mobile device 100 can reduce the power consumption of the light source 140 without reducing the power consumption of the sensor 150. The mobile device 200 can reduce the power consumption or frame rate of the image capture device 240.

[0047] In this embodiment, mobile device 100 may adjust the power consumption of light source 140 or sensor 150 according to ambient brightness. Mobile device 200 may adjust the power consumption of image capture device 240 or sensor 250 according to ambient brightness. Ambient brightness may be measured by sensor 150 of mobile device 100 or by image capture device 240 or sensor 250 of mobile device 200.

[0048] For example, suppose sensor 150 is an image capture device. When the ambient brightness is too high, the illumination from light source 140 is difficult to detect. Accordingly, mobile device 100 can reduce the power consumption of sensor 150 (or the gain of sensor 150, such as light sensitivity) without reducing the power consumption of light source 140. When the ambient brightness is moderate, the illumination from light source 140 is easier to detect. Accordingly, mobile device 100 can reduce the power consumption of light source 140 without reducing the power consumption of sensor 150 (or the gain of sensor 150, such as light sensitivity). When the ambient brightness is too low, the illumination from light source 140 is easier to detect, while sensor 150 may not be able to detect any objects. Accordingly, mobile device 100 can further reduce the power consumption of light source 140 and disable sensor 150.

[0049] Assume sensor 150 is an image capture device. In an embodiment, mobile device 100 can reduce the power consumption of sensor 150 based on environmental factors such as lack of texture, reflection, repetitive patterns, or open space. Lack of texture: The surface of objects surrounding sensor 150 lacks texture, making it difficult for sensor 150 to detect the objects. Reflection: The ambient brightness around sensor 150 is too high, making it difficult for sensor 150 to detect the objects. Repetitive patterns: The texture of the surface of objects surrounding sensor 150 is a repeating pattern, making it difficult for sensor 150 to recognize the object. Open space: The field in which sensor 150 is located is too large, making it impossible to correctly locate mobile device 100 or mobile device 200 based on the captured image.

[0050] Assume sensor 150 is an electronic compass. In this embodiment, the moving device 100 can reduce the power consumption of sensor 150 based on environmental factors such as metal layout, magnetic field distribution, or electric field distribution. Metal layout: Too many metal objects around sensor 150 may make it difficult for sensor 150 to detect moving device 200. Magnetic field or electric field distribution: A strong magnetic field or electric field may exist around sensor 150, causing the electronic compass to malfunction.

[0051] In this embodiment, the mobile device 100 can reduce the power consumption of the light source 140 based on environmental factors including electromagnetic wave distribution. For example, if there is strong sunlight, infrared light, or reflected light in the environment surrounding the mobile device 100, the illumination of the light source 140 may be difficult for the image capture device 240 to detect. Accordingly, the mobile device 100 can reduce the power consumption of the light source 140. The electromagnetic wave distribution can be measured by the sensor 150 of the mobile device 100 or by the image capture device 240 or sensor 250 of the mobile device 200.

[0052] Figure 6 A flowchart of an energy-saving tracking method is illustrated according to an embodiment of the present invention, wherein the tracking method may be as follows: Figure 1The tracking system 10 is implemented as follows: In step S601, the first mobile device adjusts at least one of the first power consumption of its sensor and the second power consumption of its first light source. In step S602, the first mobile device performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption. In step S603, the second mobile device tracks the first mobile device based on at least one of the sensing data and the illumination of the first light source.

[0053] In summary, the tracking system of this invention can track mobile devices using various methods such as sensors or light sources. Therefore, even if the application scenario of the tracking system changes, the tracking results of the mobile device can still remain accurate. Furthermore, the tracking system can determine which tracking technology performs best in the current application scenario based on environmental information or the status of the mobile device. The tracking system can dynamically adjust the power consumption of each tracking technology based on the determination results to achieve accurate tracking of the mobile device and energy saving.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An energy-saving tracking system, characterized in that, include: The first mobile device includes a sensor and a first light source; as well as The second mobile device is communicatively connected to the first mobile device, wherein The first mobile device adjusts at least one of a first power consumption of the sensor and a second power consumption of the first light source, and performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption, wherein... The second mobile device tracks the first mobile device based on the sensing data and at least one of the illuminations from the first light source.

2. The tracking system according to claim 1, wherein the second moving device includes an image capturing device, wherein The second mobile device detects the illumination to determine whether the first light source is within the field of view of the image capturing device, wherein In response to the first light source being located within the field of view, the first moving device reduces the first power consumption of the sensor.

3. The tracking system according to claim 2, wherein In response to the first light source being outside the field of view, the first mobile device reduces the second power consumption of the first light source.

4. The tracking system according to claim 1, wherein The first mobile device adjusts the second power consumption of the first light source so that the second power consumption is proportional to the distance between the first mobile device and the second mobile device.

5. The tracking system of claim 1, wherein the sensing data includes the moving speed of the first mobile device, wherein The first moving device adjusts the first power consumption of the sensor so that the first power consumption is proportional to the moving speed.

6. The tracking system of claim 1, wherein the sensing data includes the moving speed of the first mobile device, wherein The first moving device adjusts the second power consumption of the first light source so that the second power consumption is proportional to the moving speed.

7. The tracking system of claim 1, wherein the second moving device includes an image capturing device, and the sensing data includes the moving speed of the first moving device, wherein... The second moving device adjusts the third power consumption of the image capturing device so that the third power consumption is proportional to the moving speed.

8. The tracking system of claim 1, wherein the sensing data includes the attitude of the first mobile device, wherein The first mobile device adjusts the second power consumption of the first light source according to the posture.

9. The tracking system of claim 1, wherein the second moving device includes an image capturing device, and the sensing data includes the attitude of the first moving device, wherein... The second mobile device adjusts the third power consumption of the image capture device according to the posture.

10. An energy-saving tracking method, characterized in that, include: The first mobile device adjusts at least one of the first power consumption of the sensor of the first mobile device and the second power consumption of the first light source of the first mobile device; The first mobile device performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption; and The second mobile device tracks the first mobile device based on the sensing data and at least one of the illumination from the first light source.

Images