Methods, apparatus, devices, and storage media for object tracking

Abstract

Embodiments of the present invention provide a method, apparatus, device, and storage medium for object tracking. The method includes determining an optical configuration associated with an operating mode of a first device, wherein the first device comprises a motion sensor and at least one light-emitting element, the optical configuration being used to control the light emission of at least one light-emitting element, and configuring the first device to perform object tracking by tracking the light emitted by at least one light-emitting element based on the optical configuration and / or based on sensing data from the motion sensor. This allows for different configurations depending on different operating modes. In this way, a tracking device that can meet various tracking requirements and is highly scalable can be realized.

Description

[Technical Field]

[0001] (Cross-Reference to Related Applications) This application claims the priority of a Chinese patent application filed on August 31, 2023, with the invention title "Method, Apparatus, Device and Storage Medium for Object Tracking", and application number 2023111205911. The entire content disclosed in the above-mentioned patent application is incorporated into this application by reference.

[0002] (Field of the Invention) Exemplary embodiments of the present invention generally relate to the field of computers, and in particular, relate to a method, an apparatus, a device for object tracking, and a computer-readable storage medium. [Background Art]

[0003] Extended Reality Reality, XR) technology includes Virtual Reality, VR), Augmented Reality (AR), Mixed Reality, MR) and other technologies. With the development of XR technology, the experience content of XR devices has become increasingly abundant. For example, all-in-one VR devices supporting 6 Degrees of Freedom (DoF) have improved rapidly in terms of game content and user experience. [Summary of the Invention]

[0004] A first aspect of the present invention provides a method for object tracking. The method includes determining an optical configuration associated with an operating mode of a first device, wherein the first device comprises a motion sensor and at least one light-emitting element, the optical configuration being used to control the light emission of at least one light-emitting element, and configuring the first device to perform object tracking by tracking the light emitted by at least one light-emitting element and / or based on sensing data from the motion sensor, based on at least the optical configuration.

[0005] In a second aspect of the present invention, an apparatus for object tracking is provided. The apparatus comprises an optical configuration determination module configured to determine an optical configuration associated with an operating mode of a first device, the first device comprising a motion sensor and at least one light-emitting element, the optical configuration being used to control the light emission of at least one light-emitting element, and a device configuration module configured to configure the first device to perform object tracking by tracking light emitted by at least one light-emitting element based on at least the optical configuration and / or based on sensing data from the motion sensor.

[0006] A third aspect of the present invention provides an electronic device comprising at least one processing unit and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device causes the device to perform the method of the first aspect.

[0007] A fourth embodiment of the present invention provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method of the first embodiment can be realized.

[0008] It should be understood that the information described in the summary of the present invention is not intended to limit the main or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will be readily apparent from the following description. [Brief explanation of the drawing]

[0009] Referring to the following detailed description in conjunction with the drawings will further clarify the above-mentioned features and other features, advantages, and aspects of each embodiment of the present invention. In the drawings, the same or similar reference numerals indicate the same or similar elements.

[0010] [Figure 1] A schematic diagram of an exemplary environment in which embodiments of the present invention can be realized is shown.

[0011] [Figure 2A] The diagrams show schematic representations of several exemplary configurations of the first device according to some embodiments of the present invention. [Figure 2B] The diagrams show schematic representations of several exemplary configurations of the first device according to some embodiments of the present invention. [Figure 2C] The diagrams show schematic representations of several exemplary configurations of the first device according to some embodiments of the present invention.

[0012] [Figure 3A] A schematic diagram shows an example of a mounting method for the first device according to some embodiments of the present invention.

[0013] [Figure 3B] A schematic diagram shows an example of a mounting method for the first device according to some embodiments of the present invention.

[0014] [Figure 4] A schematic diagram of the light emission timing of a light-emitting element according to some embodiments of the present invention is shown.

[0015] [Figure 5]Shows a flowchart of an example process for identifying a tracker according to some embodiments of the present invention.

[0016] [Figure 6] Shows a flowchart of an example process for connecting a plurality of trackers according to some embodiments of the present invention.

[0017] [Figure 7] Shows a schematic diagram of communication timing between a plurality of trackers and a head-mounted device according to some embodiments of the present invention.

[0018] [Figure 8] Shows a schematic diagram of an exemplary signaling diagram of back connection and frequency hopping configuration between a tracker and a head-mounted device according to some embodiments of the present invention.

[0019] [Figure 9] Shows a flowchart of a process for object tracking according to some embodiments of the present invention.

[0020] [Figure 10] Shows a block diagram of an apparatus for object tracking according to some embodiments of the present invention.

[0021] [Figure 11] Shows a block diagram of a device of a plurality of embodiments capable of implementing the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] It should be understood that before using the technical solutions disclosed in each embodiment of the present invention, relevant users should be notified of the type, scope of use, and application scenarios of the information involved in the present invention in an appropriate manner in accordance with relevant laws and regulations, and the consent of relevant users should be obtained.

[0023] For example, when responding to an unauthorized request from a user, prompt information is sent to the user in question to explicitly prompt them that the requested operation requires the acquisition and use of their personal information. This allows the user to independently choose, based on the prompt information, whether or not to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present invention.

[0024] As a selective but non-restrictive implementation, a method for sending prompt information to a user in response to receiving an unsolicited request from the user may, for example, utilize a pop-up window, where the prompt information can be displayed in text form. The pop-up window may also include a selection control for the user to choose whether to "agree" or "disagree" to providing personal information to the electronic device.

[0025] The notification and user authorization process described above is merely a general overview and does not limit the embodiments of the present invention. It is understood that other methods that comply with relevant laws and regulations may also be applied to embodiments of the present invention.

[0026] It is understood that data related to this technical solution (including, but not limited to, the data itself, its acquisition, or its use) must comply with applicable laws and related designated requirements.

[0027] The embodiments of the present invention will be described in more detail below with reference to the drawings. Although specific embodiments of the present invention are shown in the drawings, the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present invention. The drawings and embodiments of the present invention are for illustrative purposes only and should not be used to limit the scope of protection of the present invention.

[0028] It should be noted that the headings of any section / subsection provided herein are not limiting. Various examples are described throughout this specification, and any type of example may be included in any section / subsection. In addition, any example described in any section / subsection may be combined in any way with any other example described in the same section / subsection and / or different sections / subsections.

[0029] In the description of embodiments of the present invention, the term “including” and similar terms are open-ended inclusions meaning “including, but not limited to, ~”. The term “based on” should be understood as “based on at least part of”. The term “one embodiment” or “the embodiment” should be understood as “at least one embodiment”. The term “some embodiments” should be understood as “at least some embodiments”. The following specification may contain other explicit and implicit definitions. Terms such as “first,” “second,” etc., may refer to different or the same subject matter. The following specification may contain other explicit and implicit definitions.

[0030] As briefly mentioned above, with the development of XR technology, some game applications, sports applications, and medical rehabilitation applications are beginning to support XR devices. For example, some VR 6DoF games and applications rely primarily on the 6DoF functionality of head-mounted display devices and joysticks. Head-mounted display devices and joysticks can complete the basic actions of game and application interaction. However, some body movements such as walking, running, sitting, squatting, dancing, kicking a ball, and doing yoga cannot be performed. The inability to track body movements significantly impacts the user experience.

[0031] Some tracking solutions include outside-in tracking technology, which uses external cameras or locators to capture and track body movements. For example, a laser light tower is installed externally to identify the location of body parts. This location information is then transmitted to a PC or all-in-one machine via USB or wirelessly to enable tracking of body movements. However, this implementation has several obvious drawbacks. Firstly, the installation process is complex and requires the use of multiple locators (at least two) that form a 360° range to awaken three-dimensional location information. Secondly, it is complex to use, and especially in the case of all-in-one machines, the positional relationship between the all-in-one machine and the locators needs to be calibrated. Thirdly, there is a large delay in the transmission of location data. Fourthly, accurate location cannot be obtained if the subject being tracked is far from the scanning range of the locators or is hidden by an object.

[0032] Some other tracking solutions involve attaching multiple trackers to the body, each containing both light-emitting elements and inertial measurement units (IMUs), and using a combination of optical and inertial tracking techniques to collaboratively identify the pose of each tracker, thereby inferring body movements or performing other actions. However, different XR application scenarios require different inputs related to body movement and pose to be acquired. For example, dance or fitness applications may focus on how to acquire the movement and pose of the user's lower limbs. Similarly, ball or shooting game applications may focus on how to acquire the movement and pose of the user's arms and the position of handheld objects (e.g., rods or handles). There are significant differences in how trackers are attached and positioned, how they communicate with head-mounted displays, how the light-emitting elements are controlled, and the processing logic for image data and IMU data in each scenario. Given this, it is not possible to apply a single system solution to multiple scenarios. However, designing optical tracking and IMU trackers tailored to each scenario and application presents numerous problems, including high costs, limited scalability, and user inconvenience.

[0033] Embodiments of the present invention propose a solution for object tracking. According to various embodiments of the present invention, an optical configuration associated with the operating mode of a first device is determined. The first device comprises a motion sensor and at least one light-emitting element. The optical configuration is used to control the light emission of at least one light-emitting element. Based on at least the determined optical configuration, the first device is configured to perform object tracking by tracking the light emitted by at least one light-emitting element and / or based on sensing data from the motion sensor, for example, to track a person wearing the first device or the first device itself. This allows for different configurations depending on different operating modes by controlling the light emission of the light-emitting element in association with the operating mode. In this way, a highly scalable tracking device can be realized that satisfies various tracking requirements.

[0034] Exemplary environment

[0035] Figure 1 shows a schematic diagram of an exemplary environment 100 that can realize an embodiment of the present invention. In environment 100, the subject 130 (also called the user) wears a second device 120. The second device 120 may be a head-mounted or wearable display device such as a head-mounted display or smart glasses that supports technologies such as VR, AR, and MR. The second device 120 can communicate with a remote device (not shown) to reconstruct a virtual scene for the subject 130 or merge virtual content with a realistic scene. In some embodiments, the second device 120 may be designed integrally with the remote device.

[0036] Environment 100 is also related to the first device 110. In some embodiments, the first device 110 may be attached by an object 130. For example, the first device 110 may be attached by an object 130 to a part of the body such as the wrist, arm, waist, knee, top of the foot, or ankle. Environment 100 may include multiple first devices 110. For example, multiple first devices 110 may be attached by an object 140 to the left wrist, right wrist, left ankle, and right ankle, respectively. In such cases, the first device 110 may be in the style of a bracelet, handle, belt, anklet, etc.

[0037] In some embodiments, the first device 110 may be attached to the object 130 in any suitable manner. For example, the first device 110 may be held by the object 130, or it may be placed on a fixed or movable object near the object 130. In such cases, the object 130 may be an object that can be attached to the first device 110, such as a club, handle, or robotic dog.

[0038] The first device 110 may establish communication with the second device 120 via a wired or wireless link. The second device 120 can track the object 130 based on sensing data collected by itself and / or data uploaded by the first device 120.

[0039] The structure and function of environment 100 are described for illustrative purposes only and should not be understood as limiting the scope of the invention. Furthermore, the above-described descriptions of the first device 110 and the second device 120 are merely illustrative and not limiting. The first device 110 and the second device 120 may be implemented as devices of various forms, structures, or categories, and embodiments of the invention are not limited in this respect.

[0040] The following description will continue with reference to the attached drawings, illustrating some exemplary embodiments of the present invention.

[0041] Exemplary configuration of the first device

[0042] In environment 100, the first device 110 may comprise a motion sensor and at least one light-emitting element. The light-emitting element may include an active light-emitting element and / or a passive light-emitting element. An active light-emitting element includes, for example, a visible light-emitting diode, an infrared light-emitting diode, etc. A passive light-emitting element includes, for example, a marker formed from a reflective material. The second device 120 may include an image sensor (e.g., a camera). The second device 120 performs object tracking by tracking the light emitted by the light-emitting element on the first device 110 and / or based on sensing data from the motion sensor. For example, it may track the first device 110 itself and / or further track an object 130 to which the first device 110 is attached. Exemplarily, the second device 120 can determine the orientation of the first device 110 relative to the second device 120 by the light emitted by the light-emitting element on the first device 110 and / or based on sensing data from the motion sensor. Furthermore, the second device 120 can determine the orientation of the first device 110 in space based on its own orientation in space. This allows the first device 110 to be tracked. In addition, the first device 110 can track an object to which it is attached or mounted based on the connection relationship between the first device 110 and that object. The tracked object may include, but is not limited to, a human body, a part of the body, or an object (such as a racket or handle).

[0043] To gain a clearer understanding of the target tracking solution according to the embodiment of the present invention, an exemplary configuration of the first device 110 will be described below with reference to Figures 2A to 2C.

[0044] In the example shown in Figure 2A, the first device 110 includes at least a main body 210 (also called a body). In some embodiments, the first device 110 may be mounted on a base 220 (also called a bracket), for example, by the main body 210. Alternatively, in some embodiments, the base 220 may be part of the first device 110. The main body 210 includes at least one light-emitting element 212, a communication interface 214, and a wireless communication module (not shown). For the purposes of the following description only, the light-emitting element on the main body will also be referred to as the first light-emitting element. Such a first light-emitting element is located on the side of the main body 210 facing the second device 120 so that the first device 120 can easily track the light emitted by the first light-emitting element. In some embodiments, the number of first light-emitting elements is greater than a predetermined number, for example, two. Multiple first light-emitting elements may be staggered to facilitate identification by the second device 120. The communication interface 214 is located on a side or bottom surface facing the base 220. The main unit 210 may establish a wireless communication connection with the second device 120 via a wireless communication module in order to transmit various data such as sensing data and configuration data.

[0045] In the example shown in Figure 2A, the main body 210 and the base 220 are detachable. This allows the same main body to be fitted to different bases, and different main bodies to be fitted to the same base. In this way, the split design allows for flexibility in meeting the needs of different scenarios. Methods for attaching the main body 210 and the base 220 include, but are not limited to, elastic fasteners, magnetic adsorption, rotary fasteners, push-in locks and push-in pop-outs. The left diagram of Figure 2A shows a schematic diagram of the main body 210 and the base 220 in a separated state, and the right diagram shows a schematic diagram of the main body 210 attached to the base 220. Based on different mounting positions, the base 220 may include rigid materials (e.g., PVC or metal), flexible materials (e.g., elastic straps), or a combination thereof to enhance wearing comfort.

[0046] In some embodiments, the base 220 may include at least one light-emitting element 222. For the purposes of the following description only, the light-emitting element on the base will also be referred to as the second light-emitting element. At least one second light-emitting element is mounted on the side of the base 220 facing the second device 120 so that the second device 120 can easily track the light emitted by the second light-emitting element. In some embodiments, the number of second light-emitting elements is greater than a predetermined number, for example, two. Multiple second light-emitting elements are provided at both ends of the base 220 or are arranged offset from the first light-emitting elements to facilitate identification by the second device 120.

[0047] The base 220 may further include a communication interface 224 and a wireless communication module (not shown). When the main body 210 is mounted on the base 220, the communication interfaces 224 and 214 may be connected to each other. The connection between the communication interfaces 224 and 214 may be any suitable form of connection, including wired connections (e.g., hardwired connections, slotted connections) and / or wireless connections (e.g., near-field communication, radio frequency identification techniques). This allows the base 220 to establish a communication connection with the first device 110. The base 220 transmits signals, transmission data, etc., to the first device 110 (e.g., the main body 210). In some embodiments, the base 220 may establish a wireless communication connection with a second device 120 via a wireless communication module to transmit sensing data, configuration data, etc.

[0048] The base 220 may further include a vibration motor 226. The vibration motor 226 is used to provide vibration feedback. For example, in a game, when an arm wearing the first device 110 performs a striking motion, the vibration motor 226 can provide strong vibration feedback to enhance the immersion of the game. Alternatively, for example, when it is necessary to draw attention or when an abnormal situation occurs, the vibration motor 226 may vibrate slightly to draw the user's attention.

[0049] The first device 110 may further include a motion sensor (not shown) for collecting sensing data related to the movement of the first device 110. In some embodiments, the motion sensor may include a three-axis gyroscope for detecting rotation angles such as pitch, roll, and heading, so that the first device 110 supports 3DoF. Additionally or alternatively, the motion sensor may include a three-axis accelerometer so that the first device 110 supports 6DoF. Additionally or alternatively, the motion sensor may include a magnetometer for detecting directions such as east, west, south, and north relative to the Earth's magnetic field. Additionally or alternatively, the motion sensor may include an IMU.

[0050] The base 220 may further include a battery 228. The battery 228 is used to power the second light-emitting element, the vibration motor 226, and the wireless communication module. In some embodiments, the battery 228 may be used to power the main body 210 via communication interfaces 224 and 214 to drive the first light-emitting element to emit light.

[0051] In the example in Figure 2B, the first device 110 generally includes a main body 230, which is mounted to the base 240 via the main body 230. In some embodiments, the first device 110 may include both a main body 230 and a base 240. The main body 230 includes at least one light-emitting element 232, a communication interface 234, and a wireless communication module (not shown). Alternatively, the base 240 may include at least one light-emitting element 242, a communication interface 244, a wireless communication module (not shown), and a motion sensor (not shown). The difference from the example in Figure 2A is the mounting location of the vibration motor 236 and battery 238. The main body 230 can establish a wireless communication connection with the second device 120 via the wireless communication module to transmit various data such as sensing data and configuration data.

[0052] In some embodiments, referring to Figure 2B, the main body 230 may further include a vibration motor 236 and a battery 238. The battery 238 may be used to power the light-emitting element 232 and may also be used to power the base 240 via communication interfaces 234 and 244. That is, the first device 110 can power the base 220. The power supplied by the first device 110 may be used to drive the light-emitting element 242 to emit light.

[0053] In the examples in Figures 2A and 2B, the first device 110 may be mounted on the object 130. In the example in Figure 2C, the first device 110 may be provided on an object in the vicinity of the object 130. For example, the first device 110 may include a main body 250 and be mounted on a base 260, or the first device 110 may include a base 260. The base 260 is substantially annular and includes two light-emitting elements 262. The main body 250 is substantially cylindrical and includes one light-emitting element 252.

[0054] The structure of the first device 110 has been described above using the examples in Figures 2A to 2C. Please understand that this is merely illustrative and does not constitute a limitation of the present invention.

[0055] Exemplary operating modes

[0056] The object tracking method according to an embodiment of the present invention may be performed by the first device 110, by the second device 120, or jointly by both. Alternatively or additionally, the object tracking method according to an embodiment of the present invention may also be performed by a remote device. For the purposes of this invention only and without limitation, each embodiment will be described below with reference to an example performed by the first device 110.

[0057] The optical configuration is used to control the light emission of at least one light-emitting element in the first device 110. In embodiments of the present invention, the optical configuration is associated with the operating mode of the first device 110. In some embodiments, the operating mode is associated with a scene of object tracking. The scene may depend on the object tracking application or a specific part of the application. For example, different applications may correspond to different scenes. Also, for example, different types of games, game scenes, or game levels within the same application may correspond to different scenes. Exemplarily, the corresponding operating mode can be determined based on instructions regarding the scene from the application layer. For example, in the example in Figure 3A, the first device 110 may be applied to a shooting type game. Also, for example, in the example in Figure 3B, the first device 110 may be applied to a running type application.

[0058] Alternatively or additionally, in some embodiments, the operating mode may be related to a base to which the first device 110 is mounted or a base included in the first device 110. In such embodiments, the mounted or included base can be determined based on an identification signal from the base. After the main body of the first device 110 is paired with a base, a signal from the base may be used as the basis for the operating mode. Such a signal may indicate information corresponding to the mounting or attachment position of the base. This allows the first device 110 to determine its operating mode.

[0059] If a second device 120 is present, the second device 120 is typically attached to a predetermined location on an object 130, also called a mounting position, such as the head. In view, the operating mode may be related to the position of the base relative to the second device 120. In some embodiments, the operating mode corresponding to a first base located at a first predetermined location is a short-range mode, and the operating mode corresponding to a second base located at a second predetermined location is a long-range mode. The first and second predetermined locations are associated with an object 130 to which the second device 120 is attached, with the second predetermined location being further from the mounting position of the second device 120 on the object 130, such as the head, than the first predetermined location.

[0060] The first predetermined position is a position associated with the object 130. In some embodiments, the first predetermined position includes the mounting position of the base or attached base on the first device 110. For example, referring to Figure 2A, if the first device 110 is attached to the wrist of the object 130 via the base 220, the first predetermined position may refer to the wrist. In some embodiments, the first predetermined position includes a position corresponding to the mounting position of the base on the first device 110. For example, in the example in Figure 3A, the first device 110 includes a body 310 and a base 320. If such a first device 110 is held by the object 130, the first predetermined position may refer to the position of the body 310.

[0061] The second predetermined location is a location associated with the object 130. In some embodiments, the second predetermined location includes the mounting location of the base or attached base on the first device 110. For example, referring to Figure 3B, the first device 110 includes a main body 330 and a base 340. If the first device 110 is mounted on the ankle of the object 130 via the base 340, the second predetermined location may refer to the ankle.

[0062] When the first device 110 is located in a first predetermined position and / or a second predetermined position, the second device 120 tracks the object 130 (e.g., a body part of the object being tracked 130 or a device associated with it) by tracking the light emitted by a light-emitting element mounted on the first device 110 (e.g., by collecting a point image using an image sensor). Such a mode may also be called an optical tracking mode.

[0063] Additionally, in some embodiments, the sensing configuration of a motion sensor (e.g., IMU) associated with the operating mode can be determined, and the first device 110 can be configured based on the sensing configuration and optical configuration. In such embodiments, the second device 120 can receive the motion sensor sensing data transmitted by the first device 110 and perform object tracking, for example, tracking of object 130, based on the light emitted by the light-emitting element and the sensing data. In this way, the combination of optical tracking and motion sensing data can improve the accuracy of the tracking.

[0064] In some embodiments, if the operating mode indicates that the first device 110 is outside the optical tracking range for target tracking, for example, if the first device 110 is located at a third predetermined position outside the optical tracking range, at least one light-emitting element in the first device 110 can be disabled. That is, in such cases, the first device 110 operates in a non-optical tracking mode. For example, the first device 110 is mounted on the rear waist (behind the waist) of the target 130, thus exceeding the field of view of the image sensor. In such cases, the second device 120 can receive the motion sensor sensing data transmitted by the first device 110 and track the target 130 based solely on that sensing data. In this way, the working scene of the first device 110 can be distinguished by the optical tracking range for the second device 120. Furthermore, in such embodiments, since the first device 110 is no longer within the optical tracking range of the second device 120, continuing to enable the light-emitting element in the first device 110 does not improve tracking accuracy, but rather increases the energy consumption of the first device 110. Therefore, in such embodiments, the energy consumption of the first device 110 can be reduced by controlling it to a non-optical tracking mode.

[0065] In some embodiments, after pairing the first device 110 with a base, the attached base can be determined based on an identification signal from the base, and its operating mode can also be determined. Such an identification signal may specifically indicate that the base is a leg base, a waist base, or a hand base.

[0066] The operating modes of the first device 110 have been described above through each embodiment. Various operating modes are determined by the scene for target tracking, the optical tracking range for the second device 120, and the identification signal from the base. Furthermore, the first device 110 can determine the optical configuration associated with the operating mode. Such an optical configuration can control the light emission of multiple light-emitting elements mounted on the first device 110, thereby achieving the effects of power saving and improved range while ensuring stable tracking.

[0067] In embodiments of the present invention, determining the optical configuration based on a base is a hardware-based implementation. In such cases, the first device can be appropriately configured regardless of the base to which it is connected, thereby enabling convenient configurations. Furthermore, the hardware-based implementation is fast and reliable. Determining the optical configuration based on a scene is a software-based implementation. Such a method allows the first device to be adapted to any application scene and is therefore more flexible. By determining the optical configuration based on both the base and the scene, hardware and software implementations can be combined, further improving flexibility and scalability.

[0068] Exemplary optical configuration

[0069] The second device 120 collects light (e.g., visible light) emitted by the light-emitting elements using an image sensor. The light emitted by each light-emitting element forms a light spot. The distance between the light spots is determined by the proximity of the distances. If these light spots are too far from a predetermined position on the second device 120, they may be too close together to be distinguishable. To adapt to different operating modes, for example, all or some of the light-emitting elements can be selectively enabled or disabled. The control of light emission from the light-emitting elements is sometimes called optical configuration.

[0070] The optical configuration may include corresponding activation states of multiple light-emitting elements. In some embodiments, when the first device 110 is located in a first predetermined position, the first device 110 can disable some of the light-emitting elements. In this way, the density of light spots can be reduced.

[0071] In the example in Figure 2A, the main body 210 includes a plurality of light-emitting elements 212. The base 220 includes a plurality of light-emitting elements 222. The arrangement of the plurality of light-emitting elements 212 is denser than the arrangement of the plurality of light-emitting elements 222. When such a first device 110 is positioned in a first predetermined location, for example, when attached to the ankle of an object 130, the first device 110 can disable some or all of the plurality of light-emitting elements 212 on the main body 210 and enable the plurality of light-emitting elements 222 on the base 220. In this way, the distance between light points can be increased, thereby reducing the density of light points, which is useful for identification by the second device 120.

[0072] In some embodiments, when the first device 110 is located in a second predetermined position, the second light-emitting element on the base can be disabled. Continuing to refer to Figure 2A, when such a first device 110 is located in a second predetermined position, for example, when it is attached to the wrist of the subject 130, all light-emitting elements 222 can be disabled and some or all of the light-emitting elements 212 can be enabled. In this way, the range can be improved.

[0073] When fewer light-emitting elements are mounted on the first device 110, the optical configuration can be precisely set. In the example in Figure 2C, the main body 250 includes one light-emitting element 252. The base 260 includes two light-emitting elements 262. When such a first device 110 is located in a second predetermined position, only the light-emitting element 252 can be enabled and the light-emitting element 262 can be disabled. When such a first device 110 is located in a first predetermined position, the light-emitting element 262 can be enabled. In this way, the range of light spots can be increased, which is useful for identification by the second device 120.

[0074] Additionally or alternatively, the optical configuration may include the brightness of the light emitted by the activated light-emitting element. Thus, in addition to adjusting the density and range of the light spots, the brightness of the light emitted by the light-emitting element can also be adjusted, thereby achieving the effect of improving flight time. In some embodiments, when the first device 110 is located in a first predetermined position, it can be determined that the activated light-emitting element emits light at a higher brightness. When the first device 110 is located in a second predetermined position, it can be determined that the activated light-emitting element emits light at a lower brightness. This is because, when the distance from the predetermined position is closer, the light spot brightness can be easily identified by the second device 120 even if it is lower. Thus, when the first device 110 is located in a second predetermined position, the brightness of the light emitted by the light-emitting element can be appropriately reduced, for example, so that the light-emitting element emits light at 60% brightness. When the first device 110 is located in a first predetermined position, the brightness of the light emitted by the light-emitting element can be appropriately increased, for example, so that the light-emitting element emits light at 90% brightness.

[0075] Additionally or alternatively, the first device 110 can achieve an overall reduction or increase in brightness by determining the brightness of each activated light-emitting element. For example, if the first device 110 is located in a second predetermined position, the brightness of half of the activated light-emitting elements can be controlled to 50%, and the brightness of the remaining half of the light-emitting elements can be controlled to 70%.

[0076] In the example shown in Figure 3A, the main body 310 is mounted on a handle bracket. When such a first device 110 is held by the object 130, all light-emitting elements on the main body 310 can be enabled to facilitate tracking from each viewpoint by the second device 120. However, when the main body is mounted on a specific special base (e.g., a wrap-around stand), only some of the light-emitting elements can be enabled. In some embodiments, the first device 110 can determine, based on the base's identification signal, whether or not there are light-emitting elements designated as non-illuminated among the multiple light-emitting elements, thereby disabling these non-illuminated elements to improve range. For example, when the main body is mounted on a wrap-around stand, one or more light-emitting elements on the main body are shielded by the bracket. The base's identification information indicates the presence of the shielded light-emitting elements. The first device 110 can disabling the corresponding light-emitting elements based on such identification signals.

[0077] The optical configuration of the first device 110 may be determined based on whether the light-emitting elements are within the optical tracking range of the second device 120. In some embodiments, if the operating mode indicates that the first device 110 is outside the optical tracking range of the target tracking, the first device 110 can disable all of its light-emitting elements. This is because, since these light-emitting elements are outside the optical tracking range, the light emitted from them does not affect the tracking by the second device 120 based on the sensing data of the motion sensor. In this way, the range of the first device 110 can be improved.

[0078] Additionally or alternatively, the optical configuration may further include the timing of light emission from the activated light-emitting elements. The light emission timing may include, but is not limited to, one or more of the light emission moment, light emission frequency, or light emission duration. When multiple first devices 110 are operating simultaneously, they may be shielded from each other or appear simultaneously, making them difficult to distinguish by the second device 120. In particular, when the volume of the first device 110 is smaller, or when the optical shape of the first device 110 is kept constant to allow for later scalability and mass production costs, the activated light-emitting elements on each first device 110 can be controlled to emit light at different timings.

[0079] In some embodiments, when multiple first devices 110 operate simultaneously, the optical configuration of one first device 110 is associated with the corresponding operating mode of these first devices 110. In other words, in such cases, the optical configuration of one first device 110 is associated not only with its own operating mode but also with the operating modes of the other first devices 110. For example, when two first devices 110 operate simultaneously, for instance, one first device 110 operates at a first predetermined position (e.g., the left ankle) and another first device 110 operates at another first predetermined position (e.g., the right ankle), the optical configuration of one first device 110 is associated not only with its own operating mode but also with the operating mode of the other first device 110. This allows such two first devices 110 to be distinguished based on their optical configurations (e.g., different light spot patterns and / or different light emission timings).

[0080] In some embodiments, the optical configuration of one first device 110 may differ at least partially from the optical configuration of another first device 110. For example, the optical configuration may include a dot pattern formed by activated light-emitting elements on each first device 110. These dot patterns may not be exactly the same, or may differ at least partially, to facilitate the distinction of the first devices 110 based on the pattern by the second device 120. In some embodiments, if the two first devices 110 are located at either a first or second predetermined position that is generally symmetrical, different dot patterns may be used to facilitate the distinction by the second device 120. For example, the dot pattern of the first device 110 located at the left wrist of object 130 may be "

number

number

[0081] Figure 4 shows a schematic diagram of the light emission timing 400 of a light-emitting element according to some embodiments of the present invention. In the example in Figure 4, the first device 110 includes tracker 1 (for example, located on the user's left hand) and tracker 2 (for example, located on the user's right hand). The second device 120 includes a head-mounted display (abbreviated as head-mount, located on the user's head). Cameras 1, 2, 3, and 4 are provided on the head-mount. These cameras can collect image data at a predetermined frequency.

[0082] Referring to the light emission timing 400, at time t1, the head-mount transmits a synchronization signal. These cameras may start exposure at each time point and continue for a certain duration, but the center points of their respective exposure durations are ensured to align at time t2, thereby collecting tracking images of the head-mount position in different fields of view. At time t3, these cameras collect tracking images of the tracker position with the same exposure duration and aligned duration center points. At this time, tracker 1 emits light. The center point of tracker 1's light emission duration is not only aligned with the center point of the camera's exposure duration, but it is also ensured that the light emission duration is longer than the exposure duration. At time t4, the head-mount transmits a synchronization signal. At time t5, the camera collects tracking images of the head-mount position. At time t6, tracker 2 emits light. The center point of tracker 2's light emission duration is not only aligned with the center point of the camera's exposure duration, but it is also ensured that the light emission duration is longer than the exposure duration. This allows tracker 1 and tracker 2 to emit light at different times. In this way, head-mounted devices are easily identified.

[0083] In some embodiments, a first optical configuration can be determined based on the base to which the first device 110 is mounted or contained, and a second optical configuration different from the first optical configuration can be determined based on the object tracking scene. For example, the first and second optical configurations may be of different types. Exemplary, the first optical configuration may be the activation state of the light-emitting element, and the second optical configuration may be the light emission timing of the activated light-emitting element. For example, in a dance application scene, two first devices 110 operate simultaneously. One first device 110 operates at a first predetermined position (e.g., the left ankle), and the other first device 110 operates at another first predetermined position (e.g., the right ankle). The first optical configuration can be determined based on the bases to which these two first devices 110 are mounted (in this example, the left ankle and right ankle bases), so that, for example, each can activate one of its own light-emitting elements. Furthermore, the second optical configuration can be determined based on the dance application scene. For example, in a dance scene, it is necessary to track the movements of the left and right legs, that is, to distinguish between the left and right legs. Accordingly, the light emission timing of the activated light-emitting elements can be configured such that the light emission timings of the two first devices 110 are staggered from each other.

[0084] In some embodiments, the first optical configuration and the second optical configuration may be determined by the first device 110. In some embodiments, the first optical configuration and the second optical configuration may be determined by the second device 120. In some embodiments, the first optical configuration may be determined by the first device 110 and the second optical configuration may be determined by the second device 120.

[0085] In some embodiments, the light-emitting element on the base may emit light. Accordingly, the combined light point of the light emitted by the light-emitting element on the first device 110 and the light emitted by the light-emitting element on the base can be tracked. This allows for target tracking based on the tracking of the combined light point, for example, tracking a human body wearing the first device 110 and / or the first device 110 itself.

[0086] The optical configuration of the first device 110 has been described above. According to each embodiment of the present invention, the present invention can provide a system solution comprising one or more first devices 110 used in combination with a second device 120. The first device 110 may be a single-piece structure or a segmented structure. The first device 110 may, as desired, be mounted on the limbs of the object 130, or on a fixed or movable object in a realistic scene. Furthermore, for different scenes, the first device 110 may be optically configured to meet the specific tracking needs in that scene. Such optical configurations include, for example, configurations for optically enabling and disabling each light-emitting element on the body and base of the first device 110, configurations for the brightness of the emitted light, configurations for the light point pattern, and configurations for the light emission timing. Specific examples of mode control of multiple trackers in different scenes will be described below with reference to Figures 5 and 6.

[0087] Figure 5 shows a flowchart of an exemplary process 500 that identifies a tracker according to some embodiments of the present invention. The exemplary process 500 includes a tracker and head-mounted scene. The tracker is, for example, a split structure (i.e., a detachable structure) including a main body and a base. When the main body is attached to the base, relative position determination of the tracker is initiated in box 510.

[0088] In box 512, if it is a leg tracker, i.e., if the attached base is attached to a leg, proceed to box 514. In box 514, the subject pairs with the base, and the subject or base transmits a connection identifier. In box 516, the tracker connects with the head mount and determines whether to enter leg tracking mode. If it enters leg tracking mode, proceed to box 518. In box 518, the tracker or head mount can perform the optical configuration associated with leg mode. For example, enable some light-emitting elements at higher brightness and lower density. After the optical configuration, the enabled light-emitting elements on the tracker do not immediately emit light, but are temporarily in standby mode, and are controlled to emit light when they receive a light emission instruction (e.g., a signal transmitted by the head mount), thereby entering tracking mode.

[0089] If box 512 determines that it is not a leg tracker, process 500 proceeds to box 520. If box 520 determines that it is a waist tracker, i.e., if the attached base is attached to the waist, process 500 proceeds to box 522. In box 522, the subject pairs with the base, and the subject or base transmits a connection identifier. In box 524, the tracker connects to the head mount and determines whether to enter waist tracking mode. If it enters waist tracking mode, process 500 proceeds to box 526. In box 526, the tracker or head mount can perform optical configurations associated with waist mode, for example, disabling all light-emitting elements.

[0090] If box 520 determines that it is not a waist tracker, process 500 proceeds to box 528. In box 528, the subject pairs with the base, and the base transmits a connection identifier. In box 530, the tracker connects with the head mount and determines whether to enter a corresponding tracking mode, such as hand mode. If it enters a corresponding tracking mode, process 500 proceeds to box 526. In box 526, the tracker or head mount can perform the optical configuration associated with that tracking mode. For example, in hand mode, some light-emitting elements are enabled at lower brightness.

[0091] Figure 6 shows a flowchart of an exemplary process 600 for connecting multiple trackers according to some embodiments of the present invention. The exemplary process 600 includes a scene of tracker 1, tracker 2, and a head mount. For example, the exemplary process 500 determines that tracker 1 is in leg mode (e.g., corresponding to the user's left ankle) and tracker 2 is in leg mode (e.g., corresponding to the user's right ankle). Furthermore, based on the fact that tracker 1 and tracker 2 are located on the left and right ankles, respectively, it is possible to determine which light-emitting elements are activated for each of these two trackers and the corresponding light emission timings for these two trackers.

[0092] In box 610, tracker 1 connects to the head mount. When process 600 determines that it is moving from standby mode to tracking mode, it proceeds to box 612. In box 612, tracker 1 receives a synchronization signal transmitted by the head mount and synchronizes with the head mount. The head mount sets the light emission timing for tracker 1. In box 614, tracker 1 emits light at the corresponding timing based on the previously determined optical configuration and returns an illuminated flag and a timestamp.

[0093] In box 620, tracker 2 connects to the head mount. When it is determined that it is moving from standby mode to tracking mode, process 600 proceeds to box 622. In box 622, tracker 2 receives the synchronization signal transmitted by the head mount and synchronizes with the head mount. The head mount sets the light emission timing for tracker 2. In box 624, tracker 2 emits light at the corresponding timing based on the previously determined optical configuration and returns an illuminated flag and a timestamp.

[0094] In box 630, the head-mounted display collects images from tracker 1 and tracker 2 at corresponding times and performs tracking based on the corresponding images and IMU data.

[0095] Example communication timing

[0096] The system solution of the present invention may further include a configuration of a communication transmission method between a plurality of first devices 110 and second devices 120, or it may further include adjusting a corresponding tracking algorithm based on the configuration of the first device 110. Specific examples of communication configurations between the tracker and head-mounted display will be described below with reference to Figures 7 and 8.

[0097] Figure 7 shows a schematic diagram of the communication timing 700 of multiple trackers and a head mount according to some embodiments of the present invention. The communication timing 700 includes tracker 1, tracker 2, and the head mount. Tracker 1 and tracker 2 use their respective communication lists.

[0098] During a specific time period, Tracker 1, Tracker 2, and the head-mounted display are all operating on Tracker 1's channel 1. Tracker 1 is in a receiving state, receiving a beacon signal at t11 ​​and transmitting data at t12. Simultaneously, the head-mounted display also switches to a receiving state. After a specific time, both Tracker 1 and the head-mounted display hop to Tracker 1's channel 2 and operate accordingly. Tracker 2 operates similarly to Tracker 1.

[0099] By adjusting the pairing of the transmit and receive timings, tracker 1 and tracker 2 configure periodic timing transmission, such as cycle T1 shown in the figure. Cycle T1 determines the maximum data transmission frequency of the trackers. Cycle T1 and the number of trackers determine the communication transmission timing of the system. The head-mount and trackers determine the data transmission channel, data transmission time, and data reception time at their respective timings within a certain period of time. By adjusting cycle T2, interference from wireless communications to other frequency bands of the head-mount, such as WiFi and Bluetooth® devices, can be reduced.

[0100] Figure 8 shows a schematic diagram of exemplary signaling diagram 800 for a tracker and head mount back-connection and frequency-hopping configuration according to some embodiments of the present invention. Exemplary signaling diagram 800 includes a tracker and a head mount. The head mount and tracker have a fixed back-link channel. The back-link channel may be a single channel or a fixed channel list. In the case of a channel list, the connection logic includes an inconsistency between the head mount update channel cycle and the tracker update channel cycle if the head mount and tracker are not properly connected. After being properly connected, the head mount and tracker use the same cycle hopping to maintain channel-updated consistency.

[0101] Specifically, at 802, the head-mount device is activated. At 804, the tracker is activated. At 806, the head-mount device scans the channels to determine the data channel list, for example, a channel list that exceeds a predetermined noise floor includes channels 5 through 20. When the head-mount device connects to the tracker, it includes the determined data channel list in the connection information. For example, it broadcasts connection information on the connection channel, and such information includes the data channel list. At 808, the head-mount device transmits a broadcast signal. The broadcast signal includes the data channel list and one hopping count. If the tracker does not receive acknowledgment, it means that the head-mount device and the tracker are not properly connected. Therefore, the head-mount device continues to transmit the broadcast signal. For example, at 810, the head-mount device continues to transmit the broadcast signal. Each time the broadcast signal is transmitted, the hopping count decreases by 1. If the hopping count is not 0, for example, at 812, the head-mount device transmits a broadcast signal. At 814, the tracker replies with acknowledgment after receiving the data channel list. In 818, the head-mounted unit determines when to use the data channel list. In 816, the tracker determines when to use the data channel list. If the hopping count is 0, the head-mounted unit and the tracker use the data channel list that was checked simultaneously.

[0102] However, during the actual transmission process, new sources of interference may exist in the external environment. In 818, the head-mounted display scans the external environment for interference in real time and generates a new data channel list. For example, it broadcasts connection information on the connection channel, which includes the new data channel list and the updated hopping count. In 820, the head-mounted display sends a broadcast signal. The broadcast signal includes the data channel list and one hopping count. Each time a broadcast signal is sent, the hopping count is decremented by one. If the hopping count is not zero, for example in 822, the head-mounted display sends a broadcast signal. In 824, the tracker replies with acknowledgment after updating the data channel list. In 826, the head-mounted display determines when to use the new data channel list. In 828, the tracker determines when to use the new data channel list. If the hopping count is zero, the head-mounted display and the tracker use the new data channel list simultaneously.

[0103] Thus, as long as the data for one frame is successfully updated during channel updates, both the head-mounted display and the tracker will use the new channel list as determined. In this way, the problem of channel interference during transmission can be solved. Additionally or alternatively, the hopping count may use the inverse operation of adding 1.

[0104] Exemplary process

[0105] Figure 9 shows a flowchart of a process 900 for object tracking according to some embodiments of the present invention. Process 900 may be implemented in a first device 110, in a second device 120, jointly by both, or by another device that can communicate with the first device 110 and the second device 120.

[0106] In box 910, the optical configuration associated with the operating mode of the first device is determined. The first device comprises a motion sensor and at least one light-emitting element, and the optical configuration is used to control the light emission of at least one light-emitting element.

[0107] In box 920, the first device is configured to perform object tracking by tracking light emitted by at least one light-emitting element based on at least an optical configuration, and / or based on sensing data from a motion sensor.

[0108] In some embodiments, the operating mode is related to the scene of object tracking and at least one of the base to which the first device is mounted or the base included in the first device.

[0109] In some embodiments, target tracking is performed by a second device, where the operating mode corresponding to a first base located at a first predetermined position is a short-range mode, and the operating mode corresponding to a second base located at a second predetermined position is a long-range mode. The first and second predetermined positions are associated with an object to which the second device is attached, and the second predetermined position is further from the attachment position of the second device on the object than the first predetermined position.

[0110] In some embodiments, determining the optical configuration includes disabling a portion of at least one light-emitting element when the operating mode is the long-range mode, and disabling a light-emitting element on the first base when the operating mode is the short-range mode.

[0111] In some embodiments, determining the optical configuration includes determining that when the operating mode is far-range mode, the activated light-emitting element emits light at a first predetermined brightness, or when the operating mode is near-range mode, the activated light-emitting element emits light at a second predetermined brightness, wherein the first predetermined brightness is higher than the second predetermined brightness.

[0112] In some embodiments, process 900 further includes receiving an identification signal from a base and determining the base based on the identification signal.

[0113] In some embodiments, determining the optical configuration includes disabling at least one light-emitting element if the operating mode indicates that the first device is outside the optical tracking range of the target tracking.

[0114] In some embodiments, configuring the first device based at least on its optical configuration includes determining the sensing configuration of a motion sensor associated with an operating mode, and configuring the first device based on the sensing configuration and the optical configuration.

[0115] In some embodiments, the first device is one of several first devices, and the optical configuration is associated with the corresponding operating modes of the several first devices.

[0116] In some embodiments, the first device is one of a plurality of first devices, and the optical configuration of the first device is at least partially different from the optical configuration of the other devices among the plurality of first devices, excluding the first device.

[0117] In some embodiments, the optical configuration includes at least one of the following: a corresponding activation state of at least one light-emitting element; a light spot pattern formed by the activated light-emitting element among the at least one light-emitting element; the brightness of the light emitted by the activated light-emitting element; and the light emission timing of the activated light-emitting element.

[0118] In some embodiments, determining the optical configuration associated with the operating mode of the first device includes determining a first optical configuration based on a base and determining a second optical configuration different from the first optical configuration based on a scene.

[0119] In some embodiments, process 900 further includes supplying power to the base via the first device in response to the first device being mounted on the base.

[0120] In some embodiments, the process 900 further includes tracking a combined light point of light emitted by at least one light-emitting element and light emitted by a light-emitting element on a base, and performing target tracking based on the tracking of the combined light point.

[0121] In some embodiments, target tracking includes tracking of the first device and at least one of the human body wearing the first device.

[0122] Exemplary apparatus and devices

[0123] Figure 10 shows a schematic structural block diagram of a device 1000 for object tracking according to a specific embodiment of the present invention. The device 1000 may be implemented as a first device 110 and / or a second device 120, or may be included in the first device 110 and / or the second device 120. Each module / component within the device 1000 may be implemented by hardware, software, firmware, or any combination thereof.

[0124] As shown in the drawings, the apparatus 1000 includes an optical configuration determination module 1010 configured to determine an optical configuration associated with the operating mode of a first device, the first device comprising a motion sensor and at least one light-emitting element, the optical configuration being used to control the light emission of at least one light-emitting element. The apparatus 1000 further includes a device configuration module 1020 configured to configure the first device to perform object tracking by tracking light emitted by at least one light-emitting element based on at least the optical configuration and / or based on sensing data from the motion sensor.

[0125] In some embodiments, the operating mode is related to the scene of object tracking and at least one of the base to which the first device is mounted or the base included in the first device.

[0126] In some embodiments, target tracking is performed by a second device, where the operating mode corresponding to a first base located at a first predetermined position is a short-range mode, and the operating mode corresponding to a second base located at a second predetermined position is a long-range mode. The first and second predetermined positions are associated with an object to which the second device is attached, and the second predetermined position is further from the attachment position of the second device on the object than the first predetermined position.

[0127] In some embodiments, the optical configuration determination module 1010 is further configured to disable a portion of at least one light-emitting element when the operating mode is far-range mode, and to disable a light-emitting element on the first base when the operating mode is near-range mode.

[0128] In some embodiments, the optical configuration determination module 1010 is further configured to determine that when the operating mode is long-distance mode, the activated light-emitting element emits light at a first predetermined brightness, or when the operating mode is short-distance mode, the activated light-emitting element emits light at a second predetermined brightness, wherein the first predetermined brightness is higher than the second predetermined brightness.

[0129] In some embodiments, the apparatus 1000 further comprises a signal receiving module configured to receive an identification signal from a base, and a base determination module that determines the base based on the identification signal.

[0130] In some embodiments, the optical configuration determination module 1010 is further configured to disable at least one light-emitting element if the operating mode indicates that the first device is outside the optical tracking range of the target tracking.

[0131] In some embodiments, the device configuration module 1020 is further configured to determine the sensing configuration of the motion sensor associated with the operating mode, and to configure the first device based on the sensing configuration and the optical configuration.

[0132] In some embodiments, the first device is one of several first devices, and the optical configuration is associated with the corresponding operating modes of the several first devices.

[0133] In some embodiments, the first device is one of a plurality of first devices, and the optical configuration of the first device is at least partially different from the optical configuration of the other devices among the plurality of first devices, excluding the first device.

[0134] In some embodiments, the optical configuration includes at least one of the following: a corresponding activation state of at least one light-emitting element; a light spot pattern formed by the activated light-emitting element among the at least one light-emitting element; the brightness of the light emitted by the activated light-emitting element; and the light emission timing of the activated light-emitting element.

[0135] In some embodiments, the optical configuration determination module 1010 is further configured to determine a first optical configuration based on a base and a second optical configuration different from the first optical configuration based on a scene.

[0136] In some embodiments, the apparatus 1000 further includes a power supply module configured to supply power to the base via the first device in response to the first device being mounted on the base.

[0137] In some embodiments, the apparatus 1000 further comprises a light spot tracking module configured to track a combined light spot of light emitted by at least one light-emitting element and light emitted by a light-emitting element on a base, and a target tracking module configured to perform target tracking based on the tracking of the combined light spot.

[0138] In some embodiments, target tracking includes tracking of the first device and at least one of the human body wearing the first device.

[0139] Figure 11 shows a block diagram of an electronic device 1100 that can carry out one or more embodiments of the present invention. It should be understood that the electronic device 1100 shown in Figure 11 is illustrative and should not limit the function and scope of the embodiments described herein. The electronic device 1100 shown in Figure 11 may be used to carry out the first device 110 and / or the second device 120 of Figure 1.

[0140] As shown in Figure 11, the electronic device 1100 is in the form of a general-purpose electronic device. The components of the electronic device 1100 may include, but are not limited to, one or more processors or processing units 1110, memory 1120, storage device 1130, one or more communication units 1140, one or more input devices 1150, and one or more output devices 1160. The processing unit 1110 may be an actual or virtual processor and can perform various processes based on a program stored in memory 1120. In a multiprocessor system, the parallel processing capability of the electronic device 1100 is improved by having multiple processing units execute computer executable instructions in parallel.

[0141] The electronic device 1100 generally includes multiple computer storage media. Such media may include, but are not limited to, volatile and non-volatile media, removable and non-removable media, and may be any obtainable media accessible by the electronic device 1100. Memory 1120 may be volatile memory (e.g., registers, fast cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or a specific combination thereof. Storage device 1130 may be removable or non-removable media, and may include machine-readable media such as flash memory drives, magnetic disks, or any other media, which may be used to store information and / or data and may be accessible within the electronic device 1100.

[0142] The electronic device 1100 may further include other removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 11, a magnetic disk drive for reading from or writing to removable, non-volatile magnetic disks (e.g., “floppy disks”) and a removable optical disk drive for reading from or writing to non-volatile optical disks may be provided. In these cases, each drive may be connected to a path (not shown) by one or more data medium interfaces. The memory 1120 may also include a computer program product 1125 having one or more program modules, which are configured to perform various methods or operations of various embodiments of the present invention.

[0143] The communication unit 1140 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the electronic device 1100 may be implemented as a single computing cluster or multiple computing machines, which can communicate via a communication connection. Therefore, the electronic device 1100 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or other network nodes.

[0144] The input device 1150 may be one or more input devices such as a mouse, keyboard, or trackball. The output device 1160 may be one or more output devices such as a display, speaker, or printer. The electronic device 1100 may further communicate with one or more external devices (not shown) such as a storage device or display device via the communication unit 1140, as needed, or with one or more devices that enable a user to interact with the electronic device 1100, or with any device (e.g., a netbook card, modem) that enables the electronic device 1100 to communicate with one or more other computing devices. Such communication may be performed via an input / output (I / O) interface (not shown).

[0145] An exemplary embodiment of the present invention provides a computer-readable storage medium in which computer-executable instructions are stored, and the computer-executable instructions are executed by a processor to realize the above method. An exemplary embodiment of the present invention further provides a computer program product, the computer program product including computer-executable instructions, which is tangibly stored on a non-temporary computer-readable medium, and the computer-executable instructions are executed by a processor to realize the above method.

[0146] Herein, each aspect of the present invention has been described with reference to flowcharts and / or block diagrams of methods, apparatus, devices, and computer program products realized by the present invention. It should be understood that each box in the flowcharts and / or block diagrams, and each combination of boxes in the flowcharts and / or block diagrams, may be realized by computer-readable program instructions.

[0147] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device to generate a machine that, when these instructions are executed by the computer or other programmable data processing device's processing unit, generates a device for performing functions / operations specified in one or more boxes in a flowchart and / or block diagram. These computer-readable program instructions may be stored on a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other device to operate in a particular manner so that the computer-readable medium on which the instructions are stored constitutes a product containing instructions for performing each aspect of the functions / operations specified in one or more boxes in a flowchart and / or block diagram.

[0148] By loading computer-readable program instructions into a computer, other programmable data processing device, or other device, a series of operational steps are performed on the computer, other programmable data processing device, or other device to generate a computer-implemented process, thereby enabling the instructions executed on the computer, other programmable data processing device, or other device to perform a function / operation specified in one or more boxes in a flowchart and / or block diagram.

[0149] The flowcharts and block diagrams in the drawings illustrate the implementable architectures, functions, and operations of systems, methods, and computer program products of multiple implementations according to the present invention. In this regard, each box in a flowchart or block diagram may represent a module, program segment, or part of an instruction, and a module, program segment, or part of an instruction contains one or more executable instructions for implementing a specified logical function. In some implementations as replacements, the functions represented in the boxes may occur in a different order than those shown in the drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or in reverse order depending on the functions involved. It should also be noted that each box in a block diagram and / or flowchart, and combinations of boxes in a block diagram and / or flowchart, may be implemented by a special-purpose hardware-based system that performs a specified function or operation, or by a combination of special-purpose hardware and computer instructions.

[0150] While the various realizations of the present invention have been described above, the above descriptions are illustrative, not exhaustive, and not limited to the realizations disclosed. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the realizations described. The choice of terms used herein is intended to best interpret the principle, practical application, or improvement to the art in the market of each realization, or to enable those skilled in the art to understand each realization disclosed herein.

Claims

1. A method for tracking targets, The objective is to determine an optical configuration associated with the operating mode of a first device, wherein the first device comprises a motion sensor and at least one light-emitting element, and the optical configuration is used to control the light emission of the at least one light-emitting element. This includes configuring the first device, at least based on the optical configuration, to perform object tracking by tracking light emitted by the at least one light-emitting element and / or based on sensing data from the motion sensor, method.

2. The aforementioned operating mode is The scene of tracking the aforementioned target, and Related to at least one of the bases to which the first device is attached or the bases included in the first device, The method according to claim 1.

3. The aforementioned target tracking is performed by the second device. The operating mode corresponding to the first base located at a predetermined position is the short-range mode, and the operating mode corresponding to the second base located at a predetermined position is the long-range mode. The first predetermined position and the second predetermined position are associated with an object to which the second device is attached, and the second predetermined position is further from the attachment position of the second device on the object than the first predetermined position. The method according to claim 2.

4. Determining the aforementioned optical configuration is When the operating mode is the long-distance mode, the light-emitting elements of at least one of the light-emitting elements are disabled, If the operating mode is the short-range mode, the operation includes disabling the light-emitting element on the first base, The method according to claim 3.

5. Determining the aforementioned optical configuration is When the operating mode is the long-distance mode, the activated light-emitting element is determined to emit light at a first predetermined brightness, or When the operating mode is the short-range mode, the activated light-emitting element is determined to emit light at a second predetermined brightness, The first predetermined brightness is higher than the second predetermined brightness. The method according to claim 3.

6. Receiving an identification signal from the aforementioned base, The further includes determining the base based on the identification signal, The method according to claim 2.

7. Determining the aforementioned optical configuration is If the operating mode indicates that the first device is outside the optical tracking range of the target tracking, it includes disabling at least one light-emitting element, The method according to claim 1.

8. Setting up the first device based at least on the optical configuration is Determining the sensing configuration of the motion sensor associated with the aforementioned operating mode, This includes configuring the first device based on the sensing configuration and the optical configuration, The method according to claim 1.

9. The first device is one of a plurality of first devices, and the optical configuration is associated with the corresponding operating mode of the plurality of first devices. The method according to claim 1.

10. The first device is one of a plurality of first devices, and the optical configuration of the first device is at least partially different from the optical configuration of the other devices among the plurality of first devices. The method according to claim 1.

11. The aforementioned optical configuration is The corresponding activation state of the at least one light-emitting element, A light spot pattern formed by the activated light-emitting element among the at least one light-emitting element, The brightness of the light emitted by the activated light-emitting element, The light emission timing of the activated light-emitting element, and at least one of the above, The method according to claim 1.

12. Determining the optical configuration associated with the operating mode of the first device is, The first optical configuration is determined based on the aforementioned base, This includes determining a second optical configuration different from the first optical configuration based on the aforementioned scene, The method according to claim 2.

13. The further includes supplying power to the base via the first device in response to the first device being mounted on the base, The method according to claim 1.

14. Tracking the combined light point of the light emitted by at least one of the light-emitting elements and the light emitted by the light-emitting elements on the base, The further includes performing the target tracking based on the tracking of the composite light point, The method according to claim 2.

15. The aforementioned target tracking is The first device, and Tracking of at least one of the human bodies wearing the first device, The method according to claim 1.

16. A device for tracking targets, An optical configuration determination module configured to determine an optical configuration associated with the operating mode of a first device, wherein the first device comprises a motion sensor and at least one light-emitting element, and the optical configuration is used to control the light emission of the at least one light-emitting element, and the optical configuration determination module is configured to determine an optical configuration associated with the operating mode of a first device, the first device comprising a motion sensor and at least one light-emitting element, and the optical configuration determination module is used to control the light emission of the at least one light-emitting element. A device configuration module configured to configure the first device, at least based on the optical configuration, to perform target tracking by tracking light emitted by the at least one light-emitting element and / or based on sensing data from the motion sensor, Device.

17. At least one processing unit, An electronic device comprising: at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, wherein, when the instructions are executed by the at least one processing unit, the electronic device causes the electronic device to perform the method according to any one of claims 1 to 15. Electronic devices.

18. The first device is a tracker, the second device is a head-mounted display device, and the electronic device includes at least one of the tracker or the head-mounted display device. The electronic device according to claim 17.

19. A computer program is stored, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 15. Computer-readable storage medium.

Images