Tracking system and tracking method for power saving

Abstract

An energy-saving tracking system and tracking method are provided. The tracking method includes: adjusting at least one of a first power consumption of a sensor of the first mobile device and a second power consumption of a first light source of the first mobile device; performing at least one of the following by the first mobile device: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption; and tracking the first mobile device by a second mobile device based on at least one of the sensing data and the illumination of the first light source.

Description

[Technical Field]

[0001] This invention relates to an extended reality (XR) technology, and more particularly to an energy-efficient tracking system and tracking method. [Previous Technology]

[0002] In XR systems, head-mounted displays (HMDs) are often used in conjunction with portable devices. To interact with the portable device, the HMD needs to track its position. However, the tracking technology used in XR systems may be affected by environmental factors and fail to perform well. Furthermore, tracking the portable device can consume a significant amount of power from either the HMD or the portable device. [Summary of the Invention]

[0003] The present invention provides an energy-saving tracking system and tracking method, which can save power consumption of tracking mobile devices.

[0004] An energy-saving tracking system of the present invention includes a first mobile device and a second mobile device. The first mobile device includes a sensor and a first light source. The second mobile device is communicatively connected to the first mobile device. The first mobile device adjusts at least one of a first power consumption of the sensor and a second power consumption of the first light source, and performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption. The second mobile device tracks the first mobile device based on at least one of the sensing data and the illumination of the first light source.

[0005] In one embodiment of the present invention, the second action device described above includes an image capturing device, wherein the second action device detects illumination to determine whether the first light source is located in the field of view of the image capturing device, wherein in response to the first light source being located in the field of view, the first action device reduces the first power consumption of the sensor.

[0006] In one embodiment of the present invention, in response to the first light source being located outside the field of view, the first action device reduces the second power consumption of the first light source.

[0007] In one embodiment of the present invention, the first action device adjusts the second power consumption of the first light source so that the second power consumption is proportional to the distance between the first action device and the second action device.

[0008] In one embodiment of the present invention, the sensing data includes the moving speed of the first mobile device, wherein the first mobile device adjusts the first power consumption of the sensor so that the first power consumption is proportional to the moving speed.

[0009] In one embodiment of the present invention, the sensing data includes the moving speed of the first mobile device, wherein the first mobile device adjusts the second power consumption of the first light source so that the second power consumption is proportional to the moving speed.

[0010] In one embodiment of the present invention, the second mobile device includes an image capturing device and the sensing data includes the movement speed of the first mobile device, wherein the second mobile device adjusts the third power consumption of the image capturing device so that the third power consumption is proportional to the movement speed.

[0011] In one embodiment of the present invention, the sensing data includes the posture of the first mobile device, wherein the first mobile device adjusts the second power consumption of the first light source according to the posture.

[0012] In one embodiment of the present invention, the second mobile device includes an image capturing device, and the sensing data includes the posture of the first mobile device, wherein the second mobile device adjusts the third power consumption of the image capturing device according to the posture.

[0013] An energy-saving tracking method of the present invention includes: adjusting at least one of a first power consumption of a sensor of the first mobile device and a second power consumption of a first light source of the first mobile device by a first mobile device; performing at least one of the following by the first mobile device: acquiring sensing data based on the adjusted first power consumption; adjusting the illumination of the first light source based on the adjusted second power consumption; and tracking the first mobile device by a second mobile device based on at least one of the sensing data and the illumination of the first light source.

[0014] Based on the above, the tracking system of the present invention can dynamically adjust the power consumption of the sensors or light sources of the mobile device according to environmental information or the status of the mobile device, so as to achieve the purpose of accurately tracking the mobile device and saving energy.

Implementation Method

[0016] FIG1 illustrates a schematic diagram of an energy-efficient tracking system 10 according to an embodiment of the present invention. The tracking system 10 may include a mobile device 100 and a mobile device 200, wherein the mobile device 200 is communicatively connected to the mobile device 100. In one embodiment, the mobile device 200 may be a head-mounted display device, and the mobile device 100 may be a portable device (e.g., a wireless controller) for operating or interacting with the head-mounted display device. The head-mounted display device may be worn on the user's head and may provide the user with an XR environment (or XR scene), such as a virtual reality (VR) environment, an augmented reality (AR) environment, or a mixed reality (MR) environment. The user may operate the portable device to transmit control commands to the head-mounted display device to interact with the head-mounted display device. In one embodiment, the portable device may be used to capture the user's movements. The head-mounted display device may determine the user's current movement based on the portable device carried by the user.

[0017] The mobile device 100 may include a processor 110, a storage medium 120, a transceiver 130, one or more light sources 140, and one or more sensors 150. The processor 110 may be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements or combinations thereof. The processor 110 can be coupled to the storage medium 120, transceiver 130, light source 140 and sensor 150, and access and execute multiple modules and various applications stored in the storage medium 120.

[0018] The storage medium 120 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or similar element or combination of the above elements, for storing multiple modules or various applications that can be executed by the processor 110.

[0019] Transceiver 130 transmits or receives signals wirelessly or via a wired connection. Transceiver 130 may also perform operations such as low-noise amplification, impedance matching, mixing, up- or down-frequency conversion, filtering, amplification, and similar operations. Processor 110 can communicate with mobile device 200 via transceiver 130. Processor 110 can use transceiver 130 to transmit wireless signals (e.g., sensing data acquired by sensor 150) to mobile device 200 or receive wireless signals (e.g., information acquired by image capturing device 240 or sensing data acquired by sensor 250) from mobile device 200.

[0020] The light source 140 of the mobile device 100 can provide illumination that will be tracked by the mobile device 200. The light source 140 may include, but is not limited to, an infrared light-emitting diode (IR LED) or a laser device. The processor 110 can dynamically adjust the power consumption of the light source 140.

[0021] Sensor 150 can generate sensing data. Sensor 150 may include, but is not limited to, an image capturing device, an electronic compass, a magnetometer, or an accelerometer (e.g., an inertial measurement unit, IMU). The mobile device 100 can perform self-tracking or pose estimation based on the sensing data. Processor 110 can dynamically adjust the power consumption of sensor 150. For example, processor 110 can reduce the power consumption of sensor 150 by reducing the frame rate or gain of sensor 150.

[0022] The mobile device 200 may include a processor 210, a storage medium 220, a transceiver 230, and an image capturing device 240. In one embodiment, the mobile device 200 may further include one or more sensors 250.

[0023] The processor 210 is, for example, a CPU, or other programmable general-purpose or special-purpose MCU, microprocessor, DSP, programmable controller, ASIC, GPU, ISP, IPU, ALU, CPLD, FPGA, or other similar components or combinations thereof. The processor 210 may be coupled to the storage medium 220, transceiver 230, image capturing device 240, and sensor 250, and access and execute multiple modules and various applications stored in the storage medium 220.

[0024] The storage medium 220 is, for example, any type of fixed or removable RAM, ROM, flash memory, HDD, SSD or similar element or combination of the above elements, for storing multiple modules or various applications that can be executed by the processor 210.

[0025] Transceiver 230 transmits or receives signals wirelessly or via a wired connection. Transceiver 230 may also perform operations such as low-noise amplification, impedance matching, mixing, up- or down-frequency conversion, filtering, amplification, and similar operations. Processor 210 can communicate with mobile device 100 via transceiver 230. Processor 210 can use transceiver 230 to transmit wireless signals to or receive wireless signals from mobile device 100.

[0026] The image capturing device 240 is, for example, a camera used to capture images. The image capturing device 240 may include a photosensitive element such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD). The processor 210 may dynamically adjust the power consumption of the image capturing device 240. For example, the processor 110 may reduce the power consumption of the image capturing device 240 by reducing the frame rate or gain of the image capturing device 240. In one embodiment, the exposure time of the image capturing device 240 may be proportional to or inversely proportional to the gain of the image capturing device 240.

[0027] Sensor 250 can generate sensing data. Sensor 250 may include, but is not limited to, an image capturing device, an electronic compass, a magnetometer, or an accelerometer. Mobile device 200 can perform self-tracking or self-attitude prediction based on the sensing data. Processor 210 can dynamically adjust the power consumption of sensor 250. For example, processor 210 can reduce the power consumption of sensor 250 by reducing the frame rate or gain of sensor 250.

[0028] When the mobile device 200 is an HMD, the mobile device 200 may further include a display coupled to the processor 210. The display may be used to display image data, such as providing an XR environment or XR scene to a user wearing the HMD. The display may include a liquid-crystal display (LCD) or an organic light-emitting diode (OLED) display. In one embodiment, the display may provide an image beam to the user's eyes to form an image on the user's retina, allowing the user to see the XR scene created by the HMD.

[0029] Mobile device 100 can acquire sensing data through sensor 150 and transmit the sensing data to mobile device 200. Mobile device 200 can determine the state of mobile device 100 or obtain environmental information about the location of mobile device 100 from the sensing data. Mobile device 200 can track the position or posture of mobile device based on sensing data. Mobile device 200 can detect the illumination of light source 140 to track the position or posture of mobile device 100. That is, mobile device 200 can track the position or posture of mobile device 100 based on sensing data from mobile device 100 or illumination from light source 140.

[0030] In some cases, tracking the mobile device 100 based on sensing data or based on the illumination of the light source 140 may produce undesirable results. The mobile device 100 may adjust the power consumption of the sensor 150 or the power consumption of the light source 140 accordingly. For example, when the mobile device 100 or the mobile device 200 determines that the current environment will distort the sensing data, the mobile device 100 may reduce the power consumption of the sensor 150 or not reduce the power consumption of the light source 140 (e.g., maintain or increase the power consumption of the light source 140) to avoid the distorted sensing data excessively affecting the tracking results of the mobile device 100. As another example, when the mobile device 100 or the mobile device 200 determines that the current posture of the mobile device 100 will make the illumination of the light source 140 difficult to detect, the mobile device 100 may reduce the power consumption of the light source 140 or not reduce the power consumption of the sensor 150 to enhance the impact of the sensing data on the tracking results of the mobile device 100.

[0031] In one embodiment, the image capturing device 240 of the mobile device 200 can detect the illumination provided by the light source 140 to determine whether the light source 140 is located within the field of view (FOV) of the image capturing device 240 and generate a determination result. The mobile device 200 can transmit the determination result to the mobile device 100. The mobile device 100 can adjust the power consumption of the light source 140 or the power consumption of the sensor 150 according to the determination result. The mobile device 200 can adjust the power consumption of the image capturing device 240 according to the determination result.

[0032] FIG2 is a schematic diagram illustrating the location of the light source 140 in the field of view of the image capturing device 240 according to an embodiment of the present invention. When the light source 140 is in the field of view of the image capturing device 240, the mobile device 100 may reduce the power consumption of the sensor 150 or not reduce the power consumption of the light source 140.

[0033] FIG3 is a schematic diagram illustrating that the light source 140 is located outside the field of view of the image capturing device 240 according to an embodiment of the present invention. When the light source 140 is located outside the field of view of the image capturing device 240, the mobile device 100 may reduce the power consumption of the light source 140 or not reduce the power consumption of the sensor 150.

[0034] FIG4 illustrates a schematic diagram according to an embodiment of the present invention, showing that a plurality of light sources 140 are respectively located within and outside the field of view of the image capturing device 240. Assume that the plurality of light sources 140 includes light source 141 and light source 142, wherein light source 141 is located within the field of view of the image capturing device 240, but light source 142 is located outside the field of view of the image capturing device 240. The mobile device 100 can reduce the power consumption of light source 142 without reducing the power consumption of light source 141. That is, when a plurality of light sources 140 are present, the mobile device 100 can selectively reduce the power consumption of a portion of the plurality of light sources 140.

[0035] In one embodiment, the mobile device 100 may adjust the power consumption of the light source 140 or the sensor 150 according to the distance between the mobile device 100 and the mobile device 200. The mobile device 200 may adjust the power consumption of the image capturing device 240 or the sensor 250 according to the aforementioned distance. The aforementioned distance may be measured by the sensor 150 of the mobile device 100 or by the image capturing device 240 or the sensor 250 of the mobile device 200.

[0036] FIG. 5 is a schematic diagram illustrating the relative positions of mobile device 100 and mobile device 200 according to an embodiment of the present invention. Mobile device 100 can adjust the power consumption of light source 140 so that the power consumption of light source 140 is proportional to the distance between mobile device 100 and mobile device 200. For example, when mobile device 100 is in position A, mobile device 100 can adjust the power consumption of light source 140 to 100%. When mobile device 100 is in position B, mobile device 100 can adjust the power consumption of light source 140 to 50%.

[0037] In one embodiment, the mobile device 100 may adjust the power consumption of the light source 140 or the sensor 150 according to the moving speed of the mobile device 100, so that the power consumption is proportional to the moving speed of the mobile device 100. On the other hand, the mobile device 200 may adjust the power consumption of the image capturing device 240 or the sensor 250 according to the moving speed of the mobile device 100, so that the power consumption is proportional to the moving speed of the mobile device 100. The moving speed of the mobile device 100 may be measured by the sensor 150 of the mobile device 100 or by the image capturing device 240 or the sensor 250 of the mobile device 200.

[0038] For example, when the mobile device 100 moves at high speed, the mobile device 100 can adjust the power consumption of the light source 140 to 50% and the frame rate of the sensor 150 to 60 frames per second (FPS). The mobile device 200 can adjust the frame rate of the image capturing device 240 to 60 FPS, adjust the gain of the image capturing device 240 to high gain, and adjust the exposure time of the image capturing device 240 to 3 milliseconds. On the other hand, when the mobile device 100 moves at low speed, the mobile device 100 can adjust the power consumption of the light source 140 to 20% and the frame rate of the sensor 150 to 15 FPS. The mobile device 200 can adjust the frame rate of the image capturing device 240 to 30 FPS, adjust the gain of the image capturing device 240 to low gain, and adjust the exposure time of the image capturing device 240 to 6 milliseconds.

[0039] In one embodiment, the mobile device 100 may adjust the power consumption of the light source 140 or the sensor 150 according to the posture of the mobile device 100 (e.g., a six-degree-of-freedom posture). The mobile device 200 may adjust the power consumption of the image capturing device 240 or the sensor 250 according to the posture of the mobile device 100. The posture of the mobile device 100 may be detected by the sensor 140 of the mobile device 100 or by the image capturing device 240 or the sensor 250 of the mobile device 200.

[0040] For example, suppose the posture of the mobile device 100 makes it difficult for the light source 140 of the mobile device 100 to be detected by the image capturing device 240 of the mobile device 200. The mobile device 100 can reduce the power consumption of the light source 140 without reducing the power consumption of the sensor 150. The mobile device 200 can reduce the power consumption or frame rate of the image capturing device 240.

[0041] In one embodiment, the mobile device 100 may adjust the power consumption of the light source 140 or the sensor 150 according to the ambient brightness. The mobile device 200 may adjust the power consumption of the image capturing device 240 or the sensor 250 according to the ambient brightness. The ambient brightness may be measured by the sensor 150 of the mobile device 100 or by the image capturing device 240 or the sensor 250 of the mobile device 200.

[0042] For example, suppose sensor 150 is an image capturing device. When the ambient brightness is too high, the illumination of light source 140 is more difficult to detect. Accordingly, mobile device 100 can reduce the power consumption of sensor 150 (or the gain of sensor 150, such as light sensitivity) without reducing the power consumption of light source 140. When the ambient brightness is moderate, the illumination of light source 140 is easier to detect. Accordingly, mobile device 100 can reduce the power consumption of light source 140 without reducing the power consumption of sensor 150 (or the gain of sensor 150, such as light sensitivity). When the ambient brightness is too low, the illumination of light source 140 is easier to detect, while sensor 150 may not be able to detect any objects. Accordingly, mobile device 100 can further reduce the power consumption of light source 140 and disable sensor 150.

[0043] Assume that sensor 150 is an image capturing device. In one embodiment, mobile device 100 can reduce the power consumption of sensor 150 based on environmental factors such as lack of texture, reflection, repetitive patterns, or open space. Lack of texture: The surface of objects surrounding sensor 150 lacks texture, making it difficult for sensor 150 to detect the objects. Reflection: The ambient brightness around sensor 150 is too high, making it difficult for sensor 150 to detect the objects. Repetitive patterns: The texture of the surface of objects surrounding sensor 150 is a repetitive pattern, making it difficult for sensor 150 to identify the objects. Open space: The area where sensor 150 is located is too large, making it impossible to correctly locate mobile device 100 or mobile device 200 based on the captured image.

[0044] Assume sensor 150 is an electronic compass. In one embodiment, mobile device 100 can reduce the power consumption of sensor 150 based on environmental factors such as metal layout, magnetic field distribution, or power plant distribution. Metal layout: Too many metal objects around sensor 150 make it difficult for sensor 150 to detect mobile device 200. Magnetic field or electric field distribution: A strong magnetic field or electric field around sensor 150 causes the electronic compass to malfunction.

[0045] In one embodiment, the mobile device 100 may reduce the power consumption of the light source 140 based on environmental factors including electromagnetic wave distribution. For example, if there is strong sunlight, infrared light, or reflected light in the environment surrounding the mobile device 100, the illumination of the light source 140 may be difficult for the image capturing device 240 to detect. Accordingly, the mobile device 100 may reduce the power consumption of the light source 140. The electromagnetic wave distribution may be measured by the sensor 150 of the mobile device 100 or by the image capturing device 240 or the sensor 250 of the mobile device 200.

[0046] FIG. 6 illustrates a flowchart of an energy-saving tracking method according to an embodiment of the present invention, wherein the tracking method may be implemented by the tracking system 10 as shown in FIG. 1. In step S601, at least one of a first power consumption of a sensor of the first mobile device and a second power consumption of a first light source of the first mobile device is adjusted by the first mobile device. In step S602, at least one of the following is performed by the first mobile device: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption. In step S603, the second mobile device tracks the first mobile device based on at least one of the sensing data and the illumination of the first light source.

[0047] In summary, the tracking system of the present invention can track mobile devices using various methods such as sensors or light sources. Therefore, even if the application scenario of the tracking system changes, the tracking results of the mobile device can still remain accurate. Furthermore, the tracking system can determine which tracking technology performs better in the current application scenario based on factors such as environmental information or the status of the mobile device. The tracking system can dynamically adjust the power consumption of each tracking technology based on the determination results to achieve accurate tracking of the mobile device and energy saving. [Simplified Explanation of the Diagram]

[0015] Figure 1 illustrates a schematic diagram of an energy-saving tracking system according to an embodiment of the present invention. Figure 2 illustrates a schematic diagram of a light source located within the field of view of an image capturing device according to an embodiment of the present invention. Figure 3 illustrates a schematic diagram of a light source located outside the field of view of an image capturing device according to an embodiment of the present invention. Figure 4 illustrates a schematic diagram of multiple light sources located within and outside the field of view of an image capturing device according to an embodiment of the present invention. Figure 5 illustrates a schematic diagram of the relative positions of mobile devices according to an embodiment of the present invention. Figure 6 illustrates a flowchart of an energy-saving tracking method according to an embodiment of the present invention.

Claims

1. An energy-saving tracking system, comprising: The first action device includes a sensor and a first light source; and a second mobile device communicatively connected to the first mobile device, wherein the first mobile device adjusts at least one of a first power consumption of the sensor and a second power consumption of the first light source, and performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption, wherein the second mobile device tracks the first mobile device based on at least one of the sensing data and the illumination of the first light source.

2. The tracking system of claim 1, wherein the second action device includes an image capturing device, wherein the second action device detects the illumination to determine whether the first light source is within the field of view of the image capturing device, wherein in response to the first light source being within the field of view, the first action device reduces the first power consumption of the sensor.

3. The tracking system as claimed in claim 2, wherein in response to the first light source being outside the field of view, the first motion device reduces the second power consumption of the first light source.

4. The tracking system of claim 1, wherein the first action device adjusts the second power consumption of the first light source such that the second power consumption is proportional to the distance between the first action device and the second action device.

5. The tracking system of claim 1, wherein the sensing data includes the movement speed of the first moving device, wherein the first moving device adjusts the first power consumption of the sensor such that the first power consumption is proportional to the movement speed.

6. The tracking system of claim 1, wherein the sensing data includes the movement speed of the first moving device, wherein the first moving device adjusts the second power consumption of the first light source such that the second power consumption is proportional to the movement speed.

7. The tracking system of claim 1, wherein the second action device includes an image capturing device, and the sensing data includes the movement speed of the first action device, wherein the second action device adjusts a third power consumption of the image capturing device such that the third power consumption is proportional to the movement speed.

8. The tracking system of claim 1, wherein the sensing data includes the attitude of the first motion device, wherein the first motion device adjusts the second power consumption of the first light source according to the attitude.

9. The tracking system of claim 1, wherein the second action device includes an image capturing device, and the sensing data includes the posture of the first action device, wherein the second action device adjusts a third power consumption of the image capturing device according to the posture.

10. An energy-efficient tracking method, comprising: The first mobile device adjusts at least one of a first power consumption of a sensor of the first mobile device and a second power consumption of a first light source of the first mobile device; the first mobile device performs at least one of the following: acquiring sensing data based on the adjusted first power consumption; and adjusting the illumination of the first light source based on the adjusted second power consumption; and the second mobile device tracks the first mobile device based on the sensing data and the illumination of the first light source, wherein the second mobile device is communicatively connected to the first mobile device.

Images