Attitude calculation device and method

Abstract

The invention provides an attitude calculation device and method. The attitude calculation device receives a plurality of instant images and a plurality of inertial measurement parameters corresponding to at least one inertial sensor worn by a user. The posture calculation device determines a posture calculation mode corresponding to each of a plurality of body regions of the user based on the plurality of instant images and the plurality of inertial measurement parameters, where the posture calculation mode corresponds to a static mode or a motion mode. The posture calculation device calculates a posture corresponding to each of the plurality of body regions based on the posture calculation mode corresponding to each of the plurality of body regions. The posture calculation technology provided by the invention can correspondingly adjust the posture calculation modes of the plurality of body areas.

Description

Technical Field

[0001] The present invention relates to a posture calculation device and method. Specifically, the present invention relates to a posture calculation device and method capable of correspondingly adjusting posture calculation modes of multiple body regions. Background Art

[0002] In recent years, due to the rapid development of interactive technologies, various interactive applications have been proposed. In existing technologies, the calculation of user posture can be achieved by analyzing real-time images (for example, judging actions through computer vision) or by having the user wear inertial sensors (for example, judging actions through inertial measurement parameters).

[0003] However, since the detection frequency of real-time images is relatively low (e.g., 30 frames per second), it is not suitable for use when the user is exercising. In addition, since computer vision is calculated based on the content of real-time images, it is easy to cause posture calculation errors due to environmental problems such as occlusion of the user's body parts, insufficient ambient light, insufficient contrast, etc.

[0004] In addition, although the detection frequency of the inertial sensor is high (for example, 60 frames per second), the inertial sensor will produce drift values, which will cause errors and accumulate over time, so the accuracy of posture calculation will decrease over time. Furthermore, the process of resetting the inertial sensor is cumbersome, and the user's movement may cause the position of the inertial sensor to move, thereby reducing the accuracy of the inertial sensor.

[0005] In view of this, how to provide a posture calculation technology that can correspondingly adjust the posture calculation modes of multiple body regions is a goal that the industry urgently needs to work on. Summary of the invention

[0006] One object of the present disclosure is to provide a posture calculation device. The posture calculation device includes a transceiver interface and a processor. The transceiver interface is communicatively connected to an image capture device and at least one inertial sensor, and the processor is electrically connected to the transceiver interface. The processor receives a plurality of real-time images and a plurality of inertial measurement parameters corresponding to the at least one inertial sensor worn by a user. Based on the plurality of real-time images and the plurality of inertial measurement parameters, the processor determines a posture calculation mode corresponding to each of the plurality of body regions of the user, wherein the posture calculation mode corresponds to a standby mode or a motion mode. The processor calculates a posture corresponding to each of the plurality of body regions based on the posture calculation mode corresponding to each of the plurality of body regions.

[0007] Another object of the present disclosure is to provide a posture calculation method, which is used in an electronic device. The posture calculation method includes the following steps: receiving a plurality of real-time images and a plurality of inertial measurement parameters corresponding to at least one inertial sensor worn by a user; determining a posture calculation mode corresponding to each of a plurality of body regions of the user based on the plurality of real-time images and the plurality of inertial measurement parameters, wherein the posture calculation mode corresponds to a standby mode or a motion mode; and calculating a posture corresponding to each of the plurality of body regions based on the posture calculation mode corresponding to each of the plurality of body regions.

[0008] In one embodiment of the present invention, the processor further performs the following operations: in response to determining that a first body region among the plurality of body regions corresponds to the standby mode, calculating the posture corresponding to the first body region based on the plurality of real-time images.

[0009] In one embodiment of the present invention, the processor further performs the following operations: calculating at least one candidate posture corresponding to the first body region based on the multiple real-time images; and in response to determining that the number of candidate postures of the at least one candidate posture corresponding to the first body region is greater than one, determining the posture corresponding to the first body region based on the multiple inertial measurement parameters.

[0010] In one embodiment of the present invention, the processor further performs the following operations: calculating a confidence value of the posture corresponding to the first body region; and in response to determining that the confidence value of the posture corresponding to the first body region is lower than a standard value, switching the posture calculation mode of the first body region to the motion mode.

[0011] In one embodiment of the present invention, the processor further performs the following operations: in response to determining that a second body region among the plurality of body regions corresponds to the motion pattern, calculating the posture corresponding to the second body region based on the plurality of inertial measurement parameters.

[0012] In one embodiment of the present invention, each of the at least one inertial sensor corresponds to one of the multiple body regions, and the processor further performs the following operations: based on the multiple real-time images, determining whether a body part corresponding to the second body region is in a visible state from among the multiple body regions; in response to determining that the body part corresponding to the second body region is in the visible state, calculating a first drift correction value corresponding to the body part based on the multiple real-time images corresponding to the body part; and based on the first drift correction value, calibrating the multiple inertial measurement parameters generated by a first inertial sensor corresponding to the second body region.

[0013] In one embodiment of the present invention, each of the at least one inertial sensor corresponds to one of the multiple body regions, and the processor further performs the following operations: based on the multiple real-time images, determine whether a first inertial sensor corresponding to the second body region is in a visible state from among the multiple body regions; in response to determining that the first inertial sensor corresponding to the second body region is in the visible state, calculate a second drift correction value corresponding to the first inertial sensor based on the multiple real-time images corresponding to the second body region; and based on the second drift correction value, calibrate the multiple inertial measurement parameters generated by the first inertial sensor.

[0014] In one embodiment of the present invention, the processor further performs the following operations: calculating a motion intensity value of the first body region based on the multiple inertial measurement parameters; and switching the posture calculation mode of the first body region to the motion mode in response to the motion intensity value exceeding a critical value corresponding to the first body region.

[0015] In one embodiment of the present invention, the processor further performs the following operations: based on the multiple real-time images, determining whether an occlusion state occurs in the first body region; and in response to the occlusion state occurring in the first body region, switching the posture calculation mode of the first body region to the motion mode.

[0016] In one embodiment of the present invention, each of the at least one inertial sensor corresponds to one of the multiple body regions, and the processor further performs the following operations: based on a preset time interval, periodically performing the following operations for the multiple body regions corresponding to the motion pattern: based on the multiple real-time images, calculating a third drift correction value corresponding to each of the multiple body regions; and based on the third drift correction value, calibrating the multiple inertial measurement parameters generated by each of the at least one inertial sensor.

[0017] The posture calculation technology (at least including the device and the method) provided by the present disclosure determines the posture calculation mode of each of the multiple body regions corresponding to the user through the multiple real-time images and the multiple inertial measurement parameters. In addition, through the switching mechanism of the standby mode and the motion mode, a suitable posture calculation mode is selected to calculate the posture of each of the multiple body regions. The posture calculation technology provided by the present disclosure can adjust the posture calculation mode of each of the multiple body regions accordingly, thereby solving the disadvantage of decreased accuracy that may be faced when judging posture based on real-time images or inertial sensors in the conventional technology.

[0018] The detailed technology and implementation methods of the present disclosure are described below in conjunction with the accompanying drawings so that a person having ordinary knowledge in the technical field to which the present disclosure belongs can understand the technical features of the disclosure for which protection is requested. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram showing the architecture of a posture calculation device according to a first embodiment;

[0020] Figure 2 A schematic diagram illustrating an operating environment of certain embodiments;

[0021] Figure 3 Schematic diagrams of body regions illustrating certain embodiments;

[0022] Figure 4A A schematic diagram showing the operation of the standby mode of certain embodiments;

[0023] Figure 4B A schematic diagram illustrating the operation of a motion mode of certain embodiments; and

[0024] Figure 5 A partial flowchart showing the posture calculation method according to the second embodiment.

[0025] Explanation of symbols:

[0026] 1: Attitude calculation device

[0027] 11: Transceiver interface

[0028] 13: Processor

[0029] 2: Image capture device

[0030] S1, S2, S3, S4, S5, ..., Sn: Inertial sensors

[0031] C: User

[0032] G1, G2, G3, G4, G5, G6: Body area

[0033] 400A: Schematic diagram of standby mode

[0034] 400B: Schematic diagram of sports mode

[0035] VF: Live video frame

[0036] SF: Inertial measurement parameter frame

[0037] IN1, IN2: time interval

[0038] 500: Posture calculation method

[0039] S501, S503, S505: Steps DETAILED DESCRIPTION

[0040] The following will explain a posture calculation device and method provided by the present disclosure through an implementation method. However, the multiple implementation methods are not intended to limit the present disclosure to any environment, application or method as described in the multiple implementation methods. Therefore, the description of the implementation methods is only for the purpose of illustrating the present disclosure, and is not intended to limit the scope of the present disclosure. It should be understood that in the following implementation methods and drawings, elements that are not directly related to the present disclosure have been omitted and are not shown, and the size of each element and the size ratio between the elements are only for illustration, and are not intended to limit the scope of the present disclosure.

[0041] The first embodiment of the present disclosure is a posture calculation device 1, and its architecture diagram is shown in Figure 1 The posture calculation device 1 includes a transceiver interface 11 and a processor 13, and the processor 13 is electrically connected to the transceiver interface 11. In this embodiment, the transceiver interface 11 is communicatively connected to the image capture device 2 and at least one inertial sensor S1, S2, ..., Sn.

[0042] It should be noted that the processor 13 can be various processing units, a central processing unit (CPU), a microprocessor or other computing devices known to those with ordinary knowledge in the technical field of the present disclosure. The transceiver interface 11 is an interface that can receive and transmit data or other interfaces that can receive and transmit data known to those with ordinary knowledge in the technical field of the present disclosure.

[0043] In some implementations, the posture calculation device 1 may be disposed in an edge device (eg, a head mounted device HMD) or a device with computing capabilities.

[0044] For ease of understanding, Figure 2 A schematic diagram of an operating environment of the present disclosure is illustrated. In the operating environment of the present disclosure, an image capture device 2 may be set in a regional space (e.g., a living room), and a user C wears inertial sensors S1, S2, S3, S4, and S5 on the left wrist, right wrist, left ankle, right ankle, and waist, respectively. In this embodiment, the image capture device 2 is used to capture a real-time image corresponding to the user C (e.g., a real-time image including all body parts of the user C), and the inertial sensors S1, S2, S3, S4, and S5 may be used to detect inertial measurement parameters corresponding to each body part of the user C.

[0045] It should be understood that the present disclosure does not limit the number of inertial sensors worn by user C. A person having ordinary knowledge in the art should be able to understand the implementation method when user C wears multiple inertial sensors based on the content of the present disclosure, and no further explanation is given here.

[0046] It should be noted that the image capture device 2 can be any device with an image capture function. For example, the image capture device 2 can include multiple image capture units (e.g., multiple depth camera lenses) to generate multiple real-time images corresponding to a field of view (FOV), wherein the body part of the user C wearing the inertial sensor is located in the field of view. In addition, the multiple real-time images can be displayed in RGB color mode or grayscale mode.

[0047] It should be noted that the inertial sensor may include an inertial measurement unit, and the inertial sensors S1, S2, S3, S4 and S5 may continuously generate a sequence of multiple inertial measurement parameters (for example, a stream of inertial measurement parameters generated at a frequency of 10 times per second), and each of the multiple inertial measurement parameters may include an acceleration, a rotation amount and an angular acceleration. During operation, the attitude calculation device 1 may periodically receive the multiple inertial measurement parameters from the inertial sensors S1, S2, S3, S4 and S5. In some embodiments, the inertial sensor may also be disposed in a wearable device (for example, a smart bracelet).

[0048] First, in this embodiment, the processor 13 receives a plurality of real-time images and a plurality of inertial measurement parameters corresponding to the user C, wherein the plurality of inertial measurement parameters each correspond to the at least one inertial sensor worn by the user C (for example: Figure 2 Inertial sensors S1, S2, S3, S4 and S5).

[0049] Next, the processor 13 may divide the body of the user C into different regions, and determine a posture calculation mode suitable for the region according to the state of the different regions (e.g., exercise level, environmental conditions). Specifically, the processor 13 determines a posture calculation mode corresponding to each of the multiple body regions of the user C based on the multiple real-time images and the multiple inertial measurement parameters, wherein the posture calculation mode corresponds to a standby mode or a motion mode.

[0050] Next, the processor 13 calculates a posture corresponding to each of the plurality of body regions based on the posture calculation mode corresponding to each of the plurality of body regions.

[0051] For easier understanding, please refer to Figure 3 , which illustrates a body region diagram corresponding to user C. Figure 3In the embodiment, the processor 13 can divide the body of user C into different body areas (for example, groups) G1-G6, wherein body area G1 corresponds to the right arm range of user C, body area G2 corresponds to the left arm range of user C, body area G3 corresponds to the right foot range of user C, body area G4 corresponds to the left foot range of user C, body area G5 corresponds to the waist range of user C, and body area G6 corresponds to the head range of user C.

[0052] It should be noted that in the present disclosure, each body region may correspond to one or more inertial sensors (e.g.: Figure 3 body regions G1-G5 in the figure), or there may be no corresponding inertial sensors (for example: Figure 3 In other words, the present disclosure does not limit the number of inertial sensors corresponding to a body region. When the body region does not have a corresponding inertial sensor, the posture of the body region can be calculated by real-time images (for example, when performing interactive activities such as football, the user C only needs to wear inertial sensors on the left and right feet, and does not need to wear an inertial sensor on the head).

[0053] It should be noted that in the present disclosure, when the posture calculation mode of the body region is determined to correspond to the standby mode, the processor 13 will mainly calculate the posture of the body region using the real-time image method. In some embodiments, when certain conditions are met, the processor 13 will further assist in calculating the posture of the body region based on the multiple inertial measurement parameters of the body region.

[0054] In addition, when the posture calculation mode of the body region is determined to correspond to the motion mode, the processor 13 will mainly calculate the posture of the body region based on the multiple inertial measurement parameters. In some embodiments, when some conditions are met, the processor 13 will further assist in calculating the posture of the body region based on the real-time image method, or correct the multiple inertial measurement parameters based on the real-time image method.

[0055] The following will specifically describe, in certain embodiments, operations when the posture calculation mode of the body region is determined to correspond to the standby mode.

[0056] Specifically, in response to determining that a body region (hereinafter referred to as the first body region) among the plurality of body regions corresponds to the standby mode, the processor 13 calculates the posture corresponding to the first body region based on the plurality of real-time images.

[0057] For example, the processor 13 may analyze the multiple real-time images through a trained image neural network to generate a heat map corresponding to the first body region of the user C. It should be understood that the heat map may indicate the probability of action occurrence (e.g., confidence value) of each position. In some embodiments, the processor 13 may select the position with the highest probability of action occurrence as the posture of the first body region.

[0058] In some embodiments, due to environmental issues (e.g., body parts being shielded, insufficient light, etc.), the heat map of the first body region may have two or more locations with similar probability of occurrence of actions (i.e., multiple candidate postures), and the processor 13 may determine the posture of the first body region from the multiple candidate postures based on the inertial measurement parameters of the first body region. Specifically, the processor 13 calculates at least one candidate posture corresponding to the first body region based on the multiple real-time images. Then, in response to determining that the number of candidate postures corresponding to the at least one candidate posture of the first body region is greater than one, the processor 13 determines the posture corresponding to the first body region based on the multiple inertial measurement parameters.

[0059] In some embodiments, when the processor 13 determines that the posture confidence value calculated based on the real-time image is insufficient, the processor 13 can actively switch the posture calculation mode corresponding to the body region to the motion mode. Specifically, the processor 13 calculates a confidence value corresponding to the posture of the first body region. Then, in response to determining that the confidence value of the posture corresponding to the first body region is lower than a standard value, the processor 13 switches the posture calculation mode of the first body region to the motion mode.

[0060] The following will specifically describe, in certain embodiments, the operation when the posture calculation mode of the body region is determined to correspond to the motion mode.

[0061] Specifically, in response to determining that a body region (hereinafter referred to as a second body region) among the plurality of body regions corresponds to the motion mode, the processor 13 calculates the posture corresponding to the second body region based on the plurality of inertial measurement parameters.

[0062] It should be noted that, in the present disclosure, each of the at least one inertial sensor corresponds to one of the plurality of body regions (for example: Figure 3 Inertial sensors S1-S5 correspond to body regions G1-G5 respectively).

[0063] For example, the processor 13 may analyze the plurality of inertial sensing parameters through a trained neural network to generate a posture corresponding to the user C. In some embodiments, the processor 13 may represent the nodes of the body region (e.g., limb part) through an inverse tree, and the posture may be calculated through an inverse kinematics (IK) algorithm.

[0064] In certain embodiments, since the inertial measurement parameters may accumulate drift values ​​over time, the processor 13 may calibrate the inertial measurement parameters based on the body parts (e.g., the joint positions of the user C) captured in a visible state (i.e., not obscured) by the real-time images. Specifically, the processor 13 determines from the multiple body regions whether a body part corresponding to the second body region is in a visible state based on the multiple real-time images. Then, in response to determining that the body part corresponding to the second body region is in the visible state, the processor 13 calculates a first drift correction value corresponding to the body part based on the multiple real-time images corresponding to the body part. Finally, the processor 13 calibrates the multiple inertial measurement parameters generated by a first inertial sensor corresponding to the second body region based on the first drift correction value.

[0065] In some embodiments, the processor 13 may also directly calibrate the inertial measurement parameters based on an inertial sensor (e.g., an inertial sensor worn by user C) in a visible state (i.e., not obscured) captured by the real-time image. Specifically, the processor 13 determines from the multiple body regions whether a first inertial sensor corresponding to the second body region is in a visible state based on the multiple real-time images. Then, in response to determining that the first inertial sensor corresponding to the second body region is in the visible state, the processor 13 calculates a second drift correction value corresponding to the first inertial sensor based on the multiple real-time images corresponding to the second body region. Finally, the processor 13 calibrates the multiple inertial measurement parameters generated by the first inertial sensor based on the second drift correction value.

[0066] In some embodiments, the processor 13 may determine whether the body region is switched to the exercise mode by determining the exercise intensity of the body region. Specifically, the processor 13 calculates an exercise intensity value of the first body region based on the multiple inertial measurement parameters. Then, in response to the exercise intensity value exceeding a critical value corresponding to the first body region, the processor 13 switches the posture calculation mode of the first body region to the exercise mode.

[0067] It should be noted that the exercise intensity value may include one of an exercise amplitude and an exercise speed or a combination thereof. In addition, the processor 13 may set different corresponding critical values ​​for different body regions. For example, the body region G1 of the right hand has a higher critical value than the body region G5 of the waist. In addition, the processor 13 may also set different weight values ​​for different parts of the same body region, for example: the wrist of the right arm body region G1 has a higher weight value, while the elbow that also belongs to the right arm body region G1 has a lower weight value. Accordingly, when the processor 13 determines that the accumulated value of the same body region exceeds the critical value, the body region is switched to the exercise mode together.

[0068] In some embodiments, when the content of the body region cannot be identified (for example, due to environmental problems such as body parts being blocked, insufficient ambient light, insufficient contrast, etc.), the body region can also be actively switched to the motion mode. Specifically, the processor 13 determines whether the first body region is in an occlusion state based on the multiple real-time images. Then, in response to the occlusion state occurring in the first body region, the processor 13 switches the posture calculation mode of the first body region to the motion mode.

[0069] In some embodiments, the processor 13 may actively calibrate the multiple inertial measurement parameters for the body region of the motion pattern after a period of time to avoid the accumulation of error values. Specifically, the processor 13 periodically performs the following operations for the multiple body regions corresponding to the motion pattern based on a preset time interval: the processor 13 calculates a third drift correction value corresponding to each of the multiple body regions based on the multiple real-time images; and the processor 13 calibrates the multiple inertial measurement parameters generated by each of the at least one inertial sensor based on the third drift correction value.

[0070] For example, when the user C is waiting to swing (eg, baseball, tennis), serious drift values ​​may occur during the waiting time for the swing. Therefore, the processor 13 needs to perform active correction through real-time images.

[0071] To understand how to operate the standby mode and exercise mode, please refer to Figure 4A and Figure 4B , which respectively illustrate a schematic diagram 400A of the standby mode and a schematic diagram 400B of the motion mode.

[0072] In the schematic diagram 400A, the block represented by the solid line is the current main method for calculating the posture (i.e., the frame VF of the real-time image), and the block represented by the dotted line is the auxiliary frame when a specific condition is met (i.e., the frame SF of the inertial measurement parameter). In the time interval IN1 or IN2, the frequency of the detection of the frame SF of the inertial measurement parameter is higher than the frame VF of the real-time image. When the specific conditions as mentioned above are met, the processor 13 will perform auxiliary judgment through the frame SF of the inertial measurement parameter.

[0073] In addition, in the schematic diagram 400B, the block represented by the solid line is the current main method for calculating the posture (i.e., the frame SF of the inertial measurement parameter), and the block represented by the dotted line is the auxiliary frame when a specific condition is met (i.e., the frame VF of the real-time image). In the time interval IN1 or IN2, the frequency of the frame SF of the inertial measurement parameter detection is higher than the frame VF of the real-time image. When the specific conditions as mentioned above are met, the processor 13 will perform auxiliary judgment or correction through the frame VF of the real-time image.

[0074] In some implementations, the processor 13 may further combine the postures of the plurality of body regions to generate a whole body posture corresponding to the user C.

[0075] It can be seen from the above description that the posture calculation device 1 provided by the present disclosure determines the posture calculation mode of each of the multiple body regions corresponding to the user through the multiple real-time images and the multiple inertial measurement parameters. In addition, the posture calculation device 1 provided by the present disclosure can select a suitable posture calculation mode to calculate the posture of each of the multiple body regions through a switching mechanism between a standby mode and a motion mode. The posture calculation device 1 provided by the present disclosure can adjust the posture calculation mode of each of the multiple body regions accordingly, thereby solving the disadvantage of decreased accuracy that may be faced when judging posture based on real-time images or inertial sensors in the conventional technology.

[0076] The second embodiment of the present disclosure is a posture calculation method, and its flow chart is as follows: Figure 5 The posture calculation method 500 is applicable to an electronic device, such as the posture calculation device 1 described in the first embodiment. The electronic device is communicatively connected to an image capture device and at least one inertial sensor, such as the image capture device 2 and at least one inertial sensor S1, S2, ..., Sn described in the first embodiment. The posture calculation method 500 calculates the postures corresponding to the multiple body regions through steps S501 to S505.

[0077] In step S501 , a plurality of real-time images and a plurality of inertial measurement parameters corresponding to at least one inertial sensor worn by a user are received by an electronic device.

[0078] Next, in step S503 , the electronic device determines a posture calculation mode corresponding to each of the plurality of body regions of the user based on the plurality of real-time images and the plurality of inertial measurement parameters, wherein the posture calculation mode corresponds to a standby mode or a motion mode.

[0079] Finally, in step S505 , the electronic device calculates a posture corresponding to each of the plurality of body regions based on the posture calculation modes corresponding to each of the plurality of body regions.

[0080] In some embodiments, the posture calculation method 500 further comprises the following steps: in response to determining that a first body region among the plurality of body regions corresponds to the standby mode, calculating the posture corresponding to the first body region based on the plurality of real-time images.

[0081] In some embodiments, the posture calculation method 500 further includes the following steps: calculating at least one candidate posture corresponding to the first body region based on the multiple real-time images; and in response to determining that the number of candidate postures of the at least one candidate posture corresponding to the first body region is greater than one, determining the posture corresponding to the first body region based on the multiple inertial measurement parameters.

[0082] In some embodiments, the posture calculation method 500 further includes the following steps: calculating a confidence value of the posture corresponding to the first body region; and in response to determining that the confidence value of the posture corresponding to the first body region is lower than a standard value, switching the posture calculation mode of the first body region to the motion mode.

[0083] In some embodiments, the posture calculation method 500 further includes the following steps: in response to determining that a second body region among the plurality of body regions corresponds to the motion mode, calculating the posture corresponding to the second body region based on the plurality of inertial measurement parameters.

[0084] In certain embodiments, wherein the at least one inertial sensor each corresponds to one of the multiple body regions, the posture calculation method 500 further includes the following steps: based on the multiple real-time images, determining whether a body part corresponding to the second body region from the multiple body regions is in a visible state; in response to determining that the body part corresponding to the second body region is in the visible state, calculating a first drift correction value corresponding to the body part based on the multiple real-time images corresponding to the body part; and based on the first drift correction value, calibrating the multiple inertial measurement parameters generated by a first inertial sensor corresponding to the second body region.

[0085] In certain embodiments, wherein each of the at least one inertial sensor corresponds to one of the multiple body regions, the posture calculation method 500 further includes the following steps: based on the multiple real-time images, determining whether a first inertial sensor corresponding to the second body region is in a visible state from among the multiple body regions; in response to determining that the first inertial sensor corresponding to the second body region is in the visible state, calculating a second drift correction value corresponding to the first inertial sensor based on the multiple real-time images corresponding to the second body region; and based on the second drift correction value, calibrating the multiple inertial measurement parameters generated by the first inertial sensor.

[0086] In some embodiments, the posture calculation method 500 further includes the following steps: calculating a motion intensity value of the first body region based on the multiple inertial measurement parameters; and in response to the motion intensity value exceeding a critical value corresponding to the first body region, switching the posture calculation mode of the first body region to the motion mode.

[0087] In some embodiments, the posture calculation method 500 further includes the following steps: based on the multiple real-time images, determining whether an occlusion state occurs in the first body region; and in response to the occlusion state occurring in the first body region, switching the posture calculation mode of the first body region to the motion mode.

[0088] In certain embodiments, wherein each of the at least one inertial sensor corresponds to one of the multiple body regions, the posture calculation method 500 further includes the following steps: based on a preset time interval, periodically performing the following operations for the multiple body regions corresponding to the motion mode: based on the multiple real-time images, calculating a third drift correction value corresponding to each of the multiple body regions; and based on the third drift correction value, calibrating the multiple inertial measurement parameters generated by each of the at least one inertial sensor.

[0089] In addition to the above steps, the second embodiment can also perform all operations and steps of the posture calculation device 1 described in the first embodiment, has the same functions, and achieves the same technical effects. A person with ordinary knowledge in the technical field of the present disclosure can directly understand how the second embodiment performs these operations and steps based on the above-mentioned first embodiment, has the same functions, and achieves the same technical effects, so it will not be repeated.

[0090] The posture calculation method described in the second embodiment can be implemented by a computer program having multiple instructions. Each computer program can be a file that can be transmitted on a network, or can be stored in a non-transitory computer-readable storage medium. For each computer program, after the multiple instructions contained therein are loaded into an electronic device (e.g., posture calculation device 1), the computer program executes the posture calculation method described in the second embodiment. The non-transitory computer-readable storage medium can be an electronic product, such as a read only memory (ROM), a flash memory, a floppy disk, a hard disk, a compact disk (CD), a portable disk, a database accessible by a network, or any other storage medium known to a person of ordinary skill in the technical field to which the present disclosure belongs and having the same function.

[0091] It should be noted that in the patent specification and claims of this disclosure, certain terms (including: body region, drift correction value, inertial sensor, etc.) are preceded by "first", "second", or "third", and the multiple "first", "second", or "third" are only used to distinguish different terms. For example, the "first" and "second" in the first body region and the second body region are only used to indicate different body regions corresponding to different operations.

[0092] In summary, the posture calculation technology (at least including the device and the method) provided by the present disclosure determines the posture calculation mode of each of the multiple body regions corresponding to the user through the multiple real-time images and the multiple inertial measurement parameters. In addition, through the switching mechanism of the standby mode and the motion mode, a suitable posture calculation mode is selected to calculate the posture of each of the multiple body regions. The posture calculation technology provided by the present disclosure can adjust the posture calculation mode of each of the multiple body regions accordingly, thereby solving the disadvantage of decreased accuracy that may be faced when judging posture based on real-time images or inertial sensors in the conventional technology.

[0093] The above embodiments are only used to illustrate some embodiments of the present disclosure and to explain the technical features of the present disclosure, and are not used to limit the protection scope and range of the present disclosure. Any changes or equivalent arrangements that can be easily completed by a person with ordinary knowledge in the technical field to which the present disclosure belongs are within the scope claimed by the present disclosure, and the scope of protection of the present disclosure is subject to the claims.

Claims

1. A posture calculation device, characterized in that: Include: a transceiver interface, communicatively connected to an image capture device and at least one inertial sensor; A processor, electrically connected to the transceiver interface, configured to perform the following operations: Receiving a plurality of real-time images and a plurality of inertial measurement parameters corresponding to the at least one inertial sensor worn by a user; Based on the multiple real-time images and the multiple inertial measurement parameters, determine a posture calculation mode corresponding to each of the multiple body regions of the user, wherein the posture calculation mode corresponds to a standby mode or a motion mode; as well as Based on the posture calculation modes corresponding to the plurality of body regions, a posture corresponding to each of the plurality of body regions is calculated.

2. The posture calculation device according to claim 1, characterized in that: The processor further performs the following operations: In response to determining that a first body region among the plurality of body regions corresponds to the standby mode, the posture corresponding to the first body region is calculated based on the plurality of real-time images.

3. The posture calculation device according to claim 2, characterized in that: The processor further performs the following operations: Calculating at least one candidate posture corresponding to the first body region based on the multiple real-time images; as well as In response to determining that a number of candidate postures of the at least one candidate posture corresponding to the first body region is greater than one, determining the posture corresponding to the first body region based on the plurality of inertial measurement parameters.

4. The posture calculation device according to claim 3, characterized in that: The processor further performs the following operations: calculating a confidence value corresponding to the posture of the first body region; and In response to determining that the confidence value corresponding to the posture of the first body region is lower than a standard value, switching the posture calculation mode of the first body region to the motion mode.

5. The posture calculation device according to claim 1, characterized in that: The processor further performs the following operations: In response to determining that a second body region among the plurality of body regions corresponds to the motion pattern, the posture corresponding to the second body region is calculated based on the plurality of inertial measurement parameters.

6. The posture calculation device as claimed in claim 5, characterized in that: The at least one inertial sensor corresponds to one of the plurality of body regions, and the processor further performs the following operations: Based on the multiple real-time images, determining whether a body part corresponding to the second body area is in a visible state from among the multiple body areas; In response to determining that the body part corresponding to the second body region is in the visible state, calculating a first drift correction value corresponding to the body part based on the multiple real-time images corresponding to the body part; as well as The plurality of inertial measurement parameters generated by a first inertial sensor corresponding to the second body region are calibrated based on the first drift correction value.

7. The posture calculation device according to claim 5, characterized in that: The at least one inertial sensor corresponds to one of the plurality of body regions, and the processor further performs the following operations: Based on the multiple real-time images, determining from the multiple body regions whether a first inertial sensor corresponding to the second body region is in a visible state; In response to determining that the first inertial sensor corresponding to the second body region is in the visible state, calculating a second drift correction value corresponding to the first inertial sensor based on the multiple real-time images corresponding to the second body region; and The plurality of inertial measurement parameters generated by the first inertial sensor are corrected based on the second drift correction value.

8. The posture calculation device according to claim 2, characterized in that: The processor further performs the following operations: Calculating a motion intensity value of the first body region based on the plurality of inertial measurement parameters; and In response to the motion intensity value exceeding a critical value corresponding to the first body region, the posture calculation mode of the first body region is switched to the motion mode.

9. The posture calculation device according to claim 2, characterized in that: The processor further performs the following operations: Based on the multiple real-time images, determining whether the first body area is in an occlusion state; and In response to the occlusion state occurring in the first body region, the posture calculation mode of the first body region is switched to the motion mode.

10. The posture calculation device according to claim 1, characterized in that: The at least one inertial sensor corresponds to one of the plurality of body regions, and the processor further performs the following operations: Based on a preset time interval, regularly performing the following operations on the plurality of body regions corresponding to the exercise mode: Calculating a third drift correction value corresponding to each of the plurality of body regions based on the plurality of real-time images; and The plurality of inertial measurement parameters generated by each of the at least one inertial sensor are corrected based on the third drift correction value.

11. A posture calculation method, characterized in that: For an electronic device, the posture calculation method comprises the following steps: Receiving a plurality of real-time images and a plurality of inertial measurement parameters corresponding to at least one inertial sensor worn by a user; Based on the multiple real-time images and the multiple inertial measurement parameters, determine a posture calculation mode corresponding to each of the multiple body regions of the user, wherein the posture calculation mode corresponds to a standby mode or a motion mode; as well as Based on the posture calculation modes corresponding to the plurality of body regions, a posture corresponding to each of the plurality of body regions is calculated.

Images