Controller, control method, and wearable tracking system

Abstract

The present disclosure provides a controller, a control method, and a wearable tracking system. The controller is adapted to determine a position conversion relationship between a predetermined position of a body part of a user and a tracked position of a wearing part of the user. The wearing part is a position where a wearable device is placed on the user. The controller is configured to: obtain camera data of a previous time instance from a camera, wherein the camera data comprises an image of the wearable device; obtain sensor data of the previous time instance from a sensor of the wearable device; determine the position conversion relationship based on the camera data of the previous time instance and the sensor data of the previous time instance; and output fused data of the wearing part of a current time instance based on the position conversion relationship.

Description

Technical Field

[0001] This disclosure relates to a controller; more specifically, this disclosure relates to a controller, a control method, and a wearable tracking system. Background Technology

[0002] To provide users with immersive experiences, various technologies such as augmented reality (AR) and virtual reality (VR) are constantly being developed. AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter an entire new virtual world to experience different lives. However, these types of devices are often bulky and difficult to use. Summary of the Invention

[0003] This disclosure relates to a controller, control method, and wearable tracking system to improve the tracking of the position of wearable devices.

[0004] This disclosure provides a controller. The controller is adapted to determine a positional transition relationship between a predetermined position of a user's body part and a tracked position of a wearable part of the user. The wearable part is the position of a wearable device placed on the user. The controller is configured to: acquire camera data from a camera at a previous moment, wherein the camera data includes an image of the wearable device; acquire sensor data from a sensor of the wearable device at a previous moment; determine the positional transition relationship based on the camera data and the sensor data at the previous moment; and output fused data of the wearable part at the current moment based on the positional transition relationship.

[0005] This disclosure provides a control method for a wearable device. The control method for a wearable device includes: acquiring camera data from a camera at a previous moment, wherein the camera data includes an image of the wearable device; acquiring sensor data from a sensor of the wearable device at a previous moment; determining a position transition relationship based on the camera data and the sensor data at the previous moment, wherein the position transition relationship indicates the relationship between a predetermined position of a user's body part and a tracking position of a wearable part of the user, and the wearable part is the position where the wearable device is placed on the user; and outputting fused data of the wearable part at the current moment based on the position transition relationship.

[0006] This disclosure provides a wearable tracking system. The wearable tracking system includes a camera, a wearable device, and a controller. The camera is configured to acquire camera data from a previous moment, wherein the camera data includes an image of the wearable device. The wearable device includes a sensor. The sensor is configured to acquire sensor data from a previous moment. The controller is configured to: determine a position transition relationship based on the camera data and the sensor data from the previous moment, wherein the position transition relationship indicates the relationship between a predetermined position of a user's body part and the tracking position of a wearable part of the user, and the wearable part is the position where the wearable device is placed on the user; and output fused data of the wearable part at the current moment based on the position transition relationship.

[0007] Based on the above, the controller, control method, and wearable tracking system disclosed herein, by determining the position transition relationship, the fused data more accurately reflects the user's operation and thereby improves the user experience.

[0008] To make the foregoing easier to understand, several embodiments accompanied by accompanying drawings are described in detail below. Attached Figure Description

[0009] The accompanying drawings are included to provide a further understanding of this disclosure and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of this disclosure and, together with the embodiments, serve to explain the principles of this disclosure.

[0010] Figure 1 This is a schematic diagram of a wearable tracking system according to an embodiment of the present disclosure.

[0011] Figure 2 This is a schematic diagram of a wearable device disposed on a user's wearing portion according to an embodiment of the present disclosure.

[0012] Figure 3 This is a schematic diagram of a wearable device disposed on a user's wearing portion according to an embodiment of the present disclosure.

[0013] Figure 4 This is a schematic flowchart of a control method according to an embodiment of the present disclosure.

[0014] Figure 5A This is a schematic diagram illustrating user tracking according to an embodiment of the present disclosure.

[0015] Figure 5B This is a schematic diagram illustrating user tracking according to an embodiment of the present disclosure.

[0016] Figure 6 This is a schematic flowchart of a control method according to an embodiment of the present disclosure.

[0017] Explanation of icon numbers

[0018] 100: Wearable tracking system;

[0019] 110: Controller;

[0020] 120: Camera;

[0021] 130: Sensor;

[0022] 200, 300, 500A, 500B: Situations;

[0023] 400, 600: Control methods;

[0024] 501A, 501B: Predicted locations;

[0025] C1-2, C2-3, C4-3, C5-1: Camera images;

[0026] CD: Camera data;

[0027] F1-1, F2-1, F2-2, F3-2, F4-3: finger joints;

[0028] FD: Fusion Data;

[0029] S410, S420, S430, S440, S450, S460, S610, S620, S630, S640: Steps;

[0030] SD: Sensor data;

[0031] T, T-1, T-2: Time;

[0032] WD: Wearable devices. Detailed Implementation

[0033] Reference will now be made in detail to exemplary embodiments of the present disclosure, with examples of the embodiments illustrated in the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and descriptions to refer to the same or similar components.

[0034] In this disclosure and the appended claims, certain terms are used to refer to specific components. Those skilled in the art will understand that electronic device manufacturers may use different names to refer to the same components. This document is not intended to distinguish between components that have the same function but different names. In the following description and claims, words such as “comprising” and “including” are open-ended terms and should be interpreted as “including but not limited to…”.

[0035] The term "coupled (or connected)" as used throughout the description of this application (including the appended claims) may refer to any direct or indirect connecting element. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device via other devices or certain connecting elements. The terms "first," "second," and similar terms used throughout the description of this application (including the appended claims) are used only to name discrete elements or to distinguish them in different embodiments or scopes. Therefore, the terms should not be considered as an upper or lower limit on the number of elements, nor should they be used to limit the order of arrangement of elements. Furthermore, where possible, elements / components / steps using the same reference numerals in the drawings and embodiments denote the same or similar parts. The use of the same reference numerals or the same terms in different embodiments may refer to the related descriptions of elements / components / steps.

[0036] It should be noted that in the following embodiments, technical features of several different embodiments may be replaced, rearranged, and combined to complete other embodiments without departing from the spirit of this disclosure. Features of each embodiment may be arbitrarily mixed and used together as long as they do not violate the spirit of this disclosure or conflict with each other.

[0037] To provide users with immersive experiences, various technologies such as Augmented Reality (AR) and Virtual Reality (VR) are constantly being developed. AR technology allows users to bring virtual elements into the real world. VR technology allows users to enter an entire new virtual world to experience different lives. However, these types of devices are often bulky and difficult to use.

[0038] For example, some devices require users to hold the controls in their hands to perform operations, which is not very intuitive. Additionally, some devices use cameras to capture user behavior, which requires the user to remain within the camera's field of view (FOV). Otherwise, when the user remains in the camera's blind spot, the device will lose tracking. Therefore, making it easier and more intuitive for users to interact with virtual elements has become a goal that those skilled in the art want to improve.

[0039] Figure 1 This is a schematic diagram of a wearable tracking system according to an embodiment of the present disclosure. See also... Figure 1 In one embodiment, the wearable tracking system 100 includes a controller 110, a camera 120, and a sensor 130. The sensor 130 is included in a wearable device WD configured to be mounted on a user's wear portion. In one embodiment, the wear portion may be at least one arm of the user. In another embodiment, the wear portion may be at least one finger of the user, and this disclosure is not limited thereto.

[0040] In one embodiment, controller 110 includes, for example, a microcontroller unit (MCU), a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a programmable controller, a programmable logic device (PLD), other similar devices, or combinations thereof. This disclosure is not limited thereto. Additionally, in embodiments, each of the functions of controller 110 may be implemented as multiple program codes. The program codes are stored in memory and executed by controller 110. Alternatively, in embodiments, each of the functions of controller 110 may be implemented as one or more circuits. This disclosure does not limit the use of software or hardware to implement the functions of controller 110. In one embodiment, controller 110 may be mounted on a head-mounted display (HMD), wearable glasses (e.g., AR / VR goggles), electronic devices, other similar devices, or combinations thereof. However, this disclosure is not limited thereto.

[0041] In one embodiment, camera 120 includes, for example, a complementary metal oxide semiconductor (CMOS) camera or a charge coupled device (CCD) camera. However, this disclosure is not limited thereto. In one embodiment, camera 120 may be mounted on a head-mounted display (HMD), wearable glasses (e.g., AR / VR goggles), electronic devices, other similar devices, or combinations thereof. This disclosure is not limited thereto.

[0042] In one embodiment, sensor 130 includes, for example, an inertial measurement unit (IMU), an accelerometer, a gyroscope, other similar devices, or combinations thereof. This disclosure is not limited thereto. That is, sensor data SD includes inertial measurements of the wearable portion of the wearable device WD. In one embodiment, the inertial measurements include changes in six degrees of freedom (DOF), and the six DOFs include three translational values ​​corresponding to the three vertical axes and three rotational values ​​corresponding to the three vertical axes. In other words, the inertial measurements include three linear accelerations and three angular accelerations. However, this disclosure is not limited thereto.

[0043] In one embodiment, the wearable device WD includes, for example, a ring device, a wristband device, a bracelet device, an anklet device, a belt device, other similar devices, or combinations of these devices. This disclosure is not limited thereto.

[0044] In one embodiment, controller 110 is configured to communicate with camera 120 and sensor 130 via wired or wireless communication. That is, controller 110 is operatively coupled to camera 120 and sensor 130. In one embodiment, controller 110, camera 120, and sensor 130 may be housed on a separate device. However, in one embodiment, controller 110 may be integrated with camera 120 or sensor 130. That is, this disclosure does not limit the controller to being separate from or integrated with camera 120 and / or sensor 130.

[0045] In one embodiment, controller 110 is adapted to determine a positional transformation relationship between a predetermined position of a user's body part and a tracking position of a user's worn part. That is, controller 110 is configured to convert a predetermined position into a tracking position or vice versa based on the positional transformation relationship. In other words, the positional transformation relationship indicates the relationship between a predetermined position of a user's body part and a tracking position of a user's worn part.

[0046] Additionally, camera 120 is configured to acquire camera data CD from the previous moment (e.g., time T-1). Camera data CD may contain images from the wearable device WD. Sensor 130 is configured to acquire sensor data SD from the previous moment. Controller 110 is configured to acquire both the camera data CD and the sensor data SD from the previous moment. Based on the camera data CD and the sensor data SD, controller 110 is configured to determine a position transition relationship. Furthermore, controller 110 is configured to output fused data FD of the wearable portion at the current moment (e.g., time T) based on the position transition relationship.

[0047] In this way, controller 110 can determine the user's actions (behaviors) based not only on camera data CD from camera 120 but also on sensor data SD from sensor 130. That is, even when the user remains in the blind spot of camera 120, controller 110 can still determine the user's actions based on fused data FD. Furthermore, the camera frequency of camera data CD can be slower than the sensor frequency of sensor data. In one embodiment, the camera frequency is 30 frames per second (FPS) and the sensor frequency is 500 fps, but this disclosure is not limited thereto. Because the sensor frequency is higher than the camera frequency, the fused data can have a higher fusion frequency than the camera frequency. Therefore, the fusion frequency can be increased, and the size of the wearable device WD can be reduced, thereby improving the convenience and comfort of wearing the wearable device WD.

[0048] Figure 2 This is a schematic diagram of a wearable device disposed on a user's wearing portion according to an embodiment of the present disclosure. See also Figures 1 to 2 , Figure 2 Scenario 200 illustrates that the wearable device can be placed on different knuckles or fingers of a user. Because the lengths of different users' fingers vary, the positional transformation relationship from a predetermined position on the body (e.g., the wrist) to the tracking position of the worn portion of the wearable device (WD) differs between individuals. In other words, although different users may request to perform the same operation (e.g., certain movements), the changes in displacement or angle can differ between individuals.

[0049] Furthermore, even if the same operation is performed by the same user, changes in displacement or angle still occur when the wearable device WD is placed on different parts of the body. For example, a user can wear the wearable device WD on at least one of the following finger joints: F1-1, F2-1, F2-2, F3-2, F4-3, and F5-3, thereby altering the data detected by the camera 120 or sensor 130. Moreover, since the user's wearing part is not a rigid body, the wearable device WD may slip when the user performs an operation. When the wearable device WD slips, the detection data of the camera 120 or sensor 130 also changes.

[0050] Therefore, to improve the accuracy of camera data CD and sensor data SD, it is necessary to determine the position of the wearable part of the wearable device WD. Once the wearable part of the wearable device WD is determined, the results will be the same when different users perform the same operation, thereby improving the user experience.

[0051] Figure 3 This is a schematic diagram of a wearable device disposed on a user's wearing portion according to an embodiment of the present disclosure. See also Figures 1 to 3 , Figure 3Scenario 300 illustrates that the wearable portion of the wearable device WD can be determined by analyzing each part of the user's body. In one embodiment, each part of the user's body can be analyzed based on camera data CD from camera 120. In another embodiment, each part of the user's body can be analyzed based on sensor data SD from sensor 130. In yet another embodiment, each part of the user's body can be analyzed based on both camera data CD from the camera and sensor data SD from sensor 130. That is, this disclosure does not limit how the position of the wearable portion is determined.

[0052] In this embodiment, each part of the user's body can be analyzed based on camera data CD from sensor 130. For example, camera 120 can acquire multiple camera images C1-2, C2-3, C4-3, and C5-1. Controller 110 is configured to analyze multiple camera images C1-2, C2-3, C4-3, and C5-1 based on image recognition technology to determine where the wearable device WD is placed. Furthermore, controller 110 is configured to analyze multiple camera images C1-2, C2-3, C4-3, and C5-1 based on image recognition technology to determine whether the wearable device WD has slipped. In this embodiment, based on camera image C2-3, controller 110 is configured to determine the (previous moment) camera recognition position of the worn part as the tracking position. In other words, the controller determines that the wearable device is placed on the third knuckle of the index finger. Based on the tracked position (i.e., the camera-identified position) and the predetermined position (e.g., the wrist), the controller 110 is further configured to determine the position transition relationship to output fused data FD that indicates (reflects / corresponds to) the user's operation.

[0053] In one embodiment, controller 110 is configured to notify the user to perform a (predetermined) gesture (e.g., making a "five" gesture with the hand). Camera 120 is configured to acquire a gesture image of the user with the predetermined gesture, and camera data CD contains the gesture image. Controller 110 is configured to determine the (previous moment) camera recognition position of the wearable part as the tracking position based on the gesture image. Furthermore, the controller is configured to determine a position transition relationship based on the predetermined position and the tracking position. That is, controller 110 may be configured to notify the user to perform the predetermined gesture before determining the position of the wearable part of the wearable device WD to increase recognition efficiency. However, in another embodiment, controller 110 may be configured to determine the position of the wearable part of the wearable device without notifying the user to perform a gesture. This disclosure is not limited thereto.

[0054] In one embodiment, controller 110 is configured to instruct a user to perform a sequence of predetermined poses (e.g., making gestures "one" through "five" with their hand). Camera 120 is configured to acquire multiple pose sequence images of the user performing the predetermined pose sequence. Based on the multiple pose sequence images, controller 110 is configured to determine a tracking position and then determine a position transition relationship to output fused data FD indicating (reflecting / corresponding to) the user's actions.

[0055] In addition to determining the tracking position based on camera data CD, in one embodiment, controller 110 may be configured to determine the (previous time-lapse) sensor-identified position of the wearable portion as the tracking position based on sensor data SD. Furthermore, controller 110 may be configured to notify the user to perform a predetermined posture before determining the tracking position based on sensor data SD to increase recognition efficiency. In one embodiment, the sensor data may include posture parameters of the user with the predetermined posture. When the user has the predetermined posture, the posture parameters may include inertial measurements of the wearable portion of the wearable device WD. Specific processing of the embodiment for determining the tracking position based on camera data CD can be viewed with reference, and further details are not described herein.

[0056] In addition to determining the tracking position solely based on camera data CD or sensor data SD, controller 110 can be configured to determine the (previous) camera recognition position of the wearable part based on camera data CD and simultaneously determine the (previous) sensor recognition position of the wearable part based on sensor data SD. Furthermore, controller 110 is configured to determine the tracking position of the wearable part based on the camera recognition position of camera data CD and the sensor recognition position of sensor data SD. In one embodiment, the tracking position is determined by fusing camera data CD and sensor data SD using a Kalman filter to obtain the (previous) fused position. Furthermore, controller 110 is configured to determine a position transition relationship based on a predetermined position and a tracking position, and then obtain fused data indicating (reflecting / corresponding to) the user's operation at the current moment based on the position transition relationship.

[0057] Similarly, controller 110 can be configured to notify the user to perform a predetermined pose or a sequence of predetermined poses before determining the tracking position based on camera data CD and sensor data SD, in order to increase the efficiency of recognition. Specific processing for determining the tracking position can be viewed with reference to the foregoing embodiments, and will not be described in further detail herein.

[0058] Therefore, by determining the tracking position of the wearable part of the wearable device (WD), the fused data (FD) will more accurately reflect the user's actions and thus improve the user experience.

[0059] Figure 4 This is a schematic flowchart of a control method according to an embodiment of the present disclosure. See also... Figures 1 to 4 In one embodiment, the control method 400 includes steps S410, S420, S430, S440, S450, and S460.

[0060] In step S410, camera data CD is acquired by camera 120. In step S420, the user's pose is identified by controller 110 based on camera data CD, and the user pose is obtained for the next step S430. In step S430, controller 110 is configured to determine whether to initialize the tracking position. That is, controller 110 determines whether there is an initial position for the wearable device WD.

[0061] In step S440, if the initial position of the wearable device WD is not found, the controller 110 further determines the tracking position of the wearable device WD based on the camera data CD and the user's posture. In other words, the controller 110 locates the position of the wearable device WD based on the camera data CD and the user's posture. The tracking position is determined as the initial data (or initial position), and the initial data is further output.

[0062] In step S450, sensor data SD is obtained by sensor 130. In step S460, if the initial position of the wearable device WD is found, controller 110 is further configured to fine-tune the tracking position by fusing camera data CD and sensor data SD based on a Kalman filter. Based on the fine-tuned tracking position and position transformation relationship, fused data FD is then obtained by controller 110. Therefore, the fused data FD will more accurately reflect the user's operation and thus improve the user experience.

[0063] Figure 5A This is a schematic diagram illustrating user tracking according to an embodiment of this disclosure. See also... Figures 1 to 5A , Figure 5A Scenario 500A illustrates the tracking of a user's hand by camera 120. In this embodiment, the user's hand can perform actions from two consecutive previous moments (e.g., time T-2 and time T-1) to the current moment (e.g., time T). Specifically, from a moment before the previous moment (e.g., time T-2) to the previous moment (e.g., time T-1), the user's hand moves from a first position to a second position. Then, from the previous moment (e.g., time T-1) to the current moment (time T), the user's hand moves from the second position to a third position.

[0064] In this embodiment, the controller 110 can be configured to obtain a third position by extrapolation based on a first position and a second position. In other words, the controller is configured to determine the predicted camera position 501A at the current moment by extrapolating camera data CD from two consecutive previous time points (e.g., time T-2 and time T-1). It should be noted that the predicted position 501A can be accurate if the user's hand moves in the same direction. However, the predicted position 501A may be inaccurate if the user's hand does not move in the same direction.

[0065] Figure 5B This is a schematic diagram illustrating user tracking according to an embodiment of this disclosure. See also... Figures 1 to 5A , Figure 5B Scenario 500B illustrates tracking a user's hand based on camera 120 and sensor 130. In this embodiment, a wearable device WD is mounted on the user's wearing portion, and sensor 130 is included in the wearable device WD. At a previous moment (e.g., time T-1), camera 120 obtains camera data CD, and sensor 130 obtains sensor data SD. Based on camera data CD, the controller is configured to extrapolate to determine the predicted camera position at the current moment. Sensor data SD may contain the acceleration or displacement value of the user's hand at the previous moment. Based on sensor data SD (e.g., acceleration or displacement value), controller 110 is configured to determine the predicted sensor position of the wearing portion at the current moment. Furthermore, controller 110 is configured to obtain a fused predicted position at the current moment as predicted position 501B by fusing the camera predicted position and the sensor predicted position based on a Kalman filter. For example, when the user's hand tends to stop moving or tends to change the direction of movement, deceleration (negative acceleration value) can be detected in sensor data SD. Based on fused camera data (CD) and sensor data (SD), controller 110 is configured to obtain a fused predicted position as predicted position 501B, which is more accurate than a predicted position 501A based solely on camera data. Furthermore, the controller is configured to obtain fused data for the current moment based on the fused predicted position and position transition relationships. Therefore, since the position transition relationships are determined not only based on camera data (CD) but also on sensor data, the fused data (FD) will more accurately reflect the user's actions, thereby improving the user experience.

[0066] Figure 6 This is a schematic flowchart of a control method according to an embodiment of the present disclosure. See also... Figure 1 and Figure 6 The control method 600 includes steps S610, S620, S630 and S640.

[0067] In step S610, camera data CD from the previous moment is obtained from camera 120. In step S620, sensor data SD from the previous moment is obtained from sensor 130. In step S630, the position transition relationship at the current moment is determined based on the camera data CD and the sensor data SD from the previous moment. In step S640, fused data FD from the wearable device is output based on the position transition relationship. In this way, the fused data FD will more accurately reflect the user's operation and thus improve the user experience.

[0068] In summary, according to the controller, control method, and wearable tracking system of this disclosure, by determining the tracking position of the wearable portion where the wearable device is placed, the fused data more accurately reflects the user's operations and thereby improves the user experience. Furthermore, the size of the wearable device can be reduced, thereby improving the convenience and comfort of wearing the wearable device.

[0069] It will be apparent to those skilled in the art that various modifications and alterations can be made to the disclosed embodiments without departing from the scope or spirit of this disclosure. In view of the foregoing, it is intended that this disclosure cover such modifications and alterations, provided that they fall within the scope of the appended claims and their equivalents.

Claims

1. A controller adapted to determine a positional transformation relationship between a predetermined position of a user's body portion and a tracked position of a wearable portion of the user, wherein the positional transformation relationship is configured to transform a point in three-dimensional space into another point in the three-dimensional space, the wearable portion being a position on the user where a wearable device is mounted, and the controller is configured to: The camera data from the previous moment is obtained from the camera, wherein the camera data includes images of the wearable device and the predetermined location; Sensor data of the tracking position at the previous moment is obtained from the sensors of the wearable device; Based on the camera data and sensor data from the previous moment, the position transition relationship is determined; as well as By fusing the camera data and the sensor data and based on the position transformation relationship, the fused data of the wearable part at the current moment is output.

2. The controller according to claim 1, wherein the controller is further configured to: Based on the camera data, the camera recognition position of the wearable part at the previous moment is determined as the tracking position; and The position transformation relationship is determined based on the predetermined position and the tracking position.

3. The controller according to claim 1, wherein the controller is further configured to: Based on the sensor data, the sensor recognition position of the wearable part at the previous moment is determined as the tracking position; and The position transformation relationship is determined based on the predetermined position and the tracking position.

4. The controller according to claim 1, wherein the controller is further configured to: Based on the camera data, determine the camera recognition position of the wearable part at the previous moment; Based on the sensor data, determine the sensor recognition position of the wearable part at the previous moment; Based on the camera's recognized position and the sensor's recognized position, the tracking position of the wearable part is determined; and The position transformation relationship is determined based on the predetermined position and the tracking position.

5. The controller of claim 1, wherein the camera data includes a pose image of the user having a predetermined pose, and the controller is further configured to: Based on the pose image, the camera recognition position of the wearable part at the previous moment is determined as the tracking position; and The position transformation relationship is determined based on the predetermined position and the tracking position.

6. The controller of claim 1, wherein the sensor data includes attitude parameters of the user having a predetermined posture, and the controller is further configured to: Based on the attitude parameters, the sensor-identified position of the wearable part at the previous moment is determined as the tracking position; and The position transformation relationship is determined based on the predetermined position and the tracking position.

7. The controller of claim 1, wherein the camera data includes a pose image of the user having a predetermined pose, the sensor data includes pose parameters of the user having a predetermined pose, and the controller is further configured to: Based on the posture image, determine the camera recognition position of the wearable part at the previous moment; Based on the posture parameters, determine the sensor recognition position of the wearable part at the previous moment; Based on the camera's recognized position and the sensor's recognized position, the tracking position of the wearable part is determined; and The position transformation relationship is determined based on the predetermined position and the tracking position.

8. The controller of claim 1, wherein the controller is further configured to: The tracking position is obtained by fusing the camera data and the sensor data based on a Kalman filter, with the fused position at the previous moment being the fused position. Based on the predetermined location and the tracked location, the position transition relationship is determined; and Based on the aforementioned position transformation relationship, the fused data at the current moment is obtained.

9. The controller of claim 1, wherein the controller is further configured to: The predicted camera position of the wearable part at the current moment is determined by extrapolation based on the camera data. Based on the acceleration or displacement values ​​of the sensor data, determine the sensor-predicted position of the wearable part at the current moment; The fused predicted position at the current moment is obtained by fusing the camera's predicted position and the sensor's predicted position based on a Kalman filter; and Based on the fusion prediction location and the location transformation relationship, the fusion data at the current moment is obtained.

10. The controller of claim 1, wherein the sensor includes an inertial measurement unit, and the sensor data includes inertial measurement values ​​of the wearable portion.

11. The controller of claim 10, wherein the inertial measurement value comprises changes in six degrees of freedom, and the six degrees of freedom comprise three translation values ​​corresponding to three vertical axes and three rotation values ​​corresponding to the three vertical axes.

12. The controller of claim 1, wherein the wearable portion includes at least one arm of the user.

13. The controller of claim 1, wherein the wearable portion includes at least one finger of the user.

14. A control method for a wearable device, comprising: The camera data from the previous moment is obtained from the camera, wherein the camera data includes the wearable device and an image of the predetermined location; Sensor data of the tracking position at the previous moment is obtained from the sensors of the wearable device; Based on the camera data and sensor data of the previous moment, a position transformation relationship is determined, wherein the position transformation relationship is configured to transform a point in three-dimensional space into another point in the three-dimensional space, and the position transformation relationship indicates the relationship between the predetermined position of the user's body part and the tracking position of the user's wearing part, and the wearing part is the position where the wearable device is placed on the user. as well as By fusing the camera data and the sensor data and based on the position transformation relationship, the fused data of the wearable part at the current moment is output.

15. The control method for a wearable device according to claim 14, further comprising: Based on the camera data, determine the camera recognition position of the wearable part at the previous moment; Based on the sensor data, determine the sensor recognition position of the wearable part at the previous moment; Based on the camera's recognized position and the sensor's recognized position, the tracking position of the wearable part is determined; and The position transformation relationship is determined based on the predetermined position and the tracking position.

16. The control method for a wearable device according to claim 14, further comprising: Obtain a pose image of the user with a predetermined pose from the camera data; Based on the pose image, the camera recognition position of the wearable part at the previous moment is determined as the tracking position; and The position transformation relationship is determined based on the predetermined position and the tracking position.

17. The control method for a wearable device according to claim 14, further comprising: The user's posture parameters, having a predetermined posture, are obtained from the sensor data; Based on the attitude parameters, the sensor-identified position of the wearable part at the previous moment is determined as the tracking position; and The position transformation relationship is determined based on the predetermined position and the tracking position.

18. The control method for a wearable device according to claim 14, further comprising: Obtain a pose image of the user with a predetermined pose from the camera data; The posture parameters of the user having the predetermined posture are obtained from the sensor data; Based on the posture image, determine the camera recognition position of the wearable part at the previous moment; Based on the posture parameters, determine the sensor recognition position of the wearable part at the previous moment; Based on the camera's recognized position and the sensor's recognized position, the tracking position of the wearable part is determined; and The position transformation relationship is determined based on the predetermined position and the tracking position.

19. The control method for a wearable device according to claim 14, further comprising: The tracking position is obtained by fusing the camera data and the sensor data based on a Kalman filter, with the fused position at the previous moment being the fused position. Based on the predetermined location and the tracked location, the location conversion relationship is determined; as well as Based on the aforementioned position transformation relationship, the fused data at the current moment is obtained.

20. A wearable tracking system, comprising: A camera configured to acquire camera data from a previous moment, wherein the camera data includes images of the wearable device and a predetermined location; The wearable device includes a sensor, wherein the sensor is configured to obtain sensor data from the previous moment. as well as Controller, wherein the controller is configured to: Based on the camera data of the previous moment and the sensor data of the tracking position of the previous moment, a position transformation relationship is determined, wherein the position transformation relationship is configured to transform a point in three-dimensional space into another point in the three-dimensional space, and the position transformation relationship indicates the relationship between the predetermined position of the user's body part and the tracking position of the user's wearing part, and the wearing part is the position where the wearable device is placed on the user. as well as By fusing the camera data and the sensor data and based on the position transformation relationship, the fused data of the wearable part at the current moment is output.

Images