Information processing device and information processing method

Abstract

To provide a technique capable of correcting data detected by a sensor without preparing a motion model in advance. [Solution] The information processing device includes a control unit. The control unit detects a first pause, in which the body part pauses exercise, and a second pause, in which the body part pauses exercise again after the first pause, based on sensor data detected by a sensor attached to the user's body part. The control unit corrects the sensor data detected by the sensor between the first and second pauses based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] Conventionally, there is known a technology for detecting a user's movements using a sensor attached to a body part of the user. In such a technology, the user often attaches the sensor to a body part of the user. When a user attaches a sensor to a body part of the user, the angle or position of the attached sensor may deviate from the expected angle or position.

[0003] Therefore, a technique is known in which the position information of the attachment site where the motion sensor is attached is corrected by referring to a first output obtained by performing a first process on the sensor data and a second output obtained by performing a second process on the sensor data (Patent Document 1).Patent Document 1 describes that the second process includes a regression estimation process that estimates the position information of the attachment site by regression. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 203188 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, in the regression estimation process described in Patent Document 1, position information is estimated by regression in which the attitude and acceleration of the sensor device in a global coordinate system are fitted to a previously prepared motion model. In other words, in Patent Document 1, unless the motion model is fitted to the previously prepared motion model, it is not possible to estimate the position information of the attachment site where the sensor is attached.

[0006] It would be useful if the data detected by a sensor could be corrected according to the sensor's orientation without the need to prepare a motion model in advance.

[0007] In view of the above, an object of the present disclosure is to provide a technology that can correct data detected by a sensor according to the attitude of the sensor without having to prepare a motion model in advance. [Means for solving the problem]

[0008] An information processing device according to an embodiment of the present disclosure includes: A control unit is provided, The control unit Detecting a first pause in which the body part of the user pauses exercise and a second pause in which the body part pauses exercise again after the first pause based on sensor data detected by a sensor attached to the body part of the user; The sensor data detected by the sensor between the first pause and the second pause is corrected based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

[0009] An information processing method according to an embodiment of the present disclosure includes: Detecting a first pause in which the body part of the user pauses exercise and a second pause in which the body part pauses exercise again after the first pause based on sensor data detected by a sensor attached to the body part of the user; and correcting the sensor data detected by the sensor between the first pause and the second pause based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause. [Effects of the Invention]

[0010] According to an embodiment of the present disclosure, it is possible to provide a technology that can correct data detected by a sensor according to the attitude of the sensor without preparing a motion model in advance. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a configuration example of an information processing system according to a first embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram of the information processing system shown in FIG. [Figure 3] FIG. 10 is a diagram for explaining a user's operation. [Figure 4] 10 is a graph for explaining correction data. [Figure 5] FIG. 10 is a diagram showing the experimental results when the subject took one step. [Figure 6] FIG. 10 is a diagram showing experimental results when subjects walked with their legs together. [Figure 7] FIG. 10 is a diagram showing the experimental results when the subject walked backwards. [Figure 8] FIG. 10 is a diagram showing experimental results when a subject took small steps. [Figure 9] FIG. 10 is a diagram illustrating an example of a walking index. [Figure 10] FIG. 10 is a diagram showing experimental results of foot direction angles when subjects walked with their legs turned inward. [Figure 11] FIG. 10 is a diagram showing experimental results of stride length when subjects walked backwards. [Figure 12] FIG. 10 is a diagram showing experimental results of stride length when subjects took short steps. [Figure 13] FIG. 10 is a diagram showing experimental results in the sagittal plane when the subject walked with his legs together. [Figure 14] FIG. 10 shows experimental results in the sagittal plane when the subject walked backwards. [Figure 15] FIG. 10 is a diagram showing experimental results in the sagittal plane when the subject took small steps. [Figure 16] 5 is a flowchart showing an example of an operation of the information processing device according to the first embodiment of the present disclosure. [Figure 17] FIG. 2 is a diagram for explaining an application example to which the first embodiment of the present disclosure can be applied. [Figure 18] FIG. 10 is a diagram for explaining an example of how a sensor is attached in the second embodiment of the present disclosure. [Figure 19] 10 is a flowchart showing an example of an operation of the information processing device according to the second embodiment of the present disclosure. [Figure 20] FIG. 10 is a diagram for explaining an experiment on the estimation error of the yaw angle. [Figure 21] FIG. 10 is a diagram for explaining an experiment on the estimation error of the yaw angle. [Figure 22] FIG. 10 is a diagram for explaining an experiment on the estimation error of the yaw angle. [Figure 23] FIG. 10 is a table showing experimental results of yaw angle estimation error. [Figure 24] FIG. 10 is a graph showing experimental results of yaw angle estimation error. [Figure 25] FIG. 11 is a diagram for explaining a user's action according to the third embodiment of the present disclosure. [Figure 26] 10 is a graph showing the angle and angular velocity of a user's head when the user looks down at their feet. [Figure 27] 10A and 10B are diagrams for explaining angular velocity vectors and the like when a user looks at his or her feet. [Figure 28] 10 is a flowchart showing an example of an operation of the information processing device according to the third embodiment of the present disclosure. [Figure 29] FIG. 10 is a diagram illustrating a pause during continuous walking. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] (First embodiment) 1 includes a sensor 10A, a sensor 10B, a sensor 10C, sensors 10D-1 and 10D-2, sensors 10E-1 and 10E-2, sensors 10F-1 and 10F-2, and an information processing device 20. However, the information processing system 1 does not need to include all of the sensors 10A, 10B, 10C, 10D-1, 10D-2, 10E-1, 10E-2, 10F-1, and 10F-1. The information processing system 1 only needs to include at least one of the sensors 10A, 10B, 10C, 10D-1, 10D-2, 10E-1, 10E-2, 10F-1, and 10F-1.

[0014] Hereinafter, when sensors 10D-1 and 10D-2 are not particularly distinguished from one another, they will be referred to as "sensor 10D." When sensors 10E-1 and 10E-2 are not particularly distinguished from one another, they will be referred to as "sensor 10E." When sensors 10F-1 and 10F-2 are not particularly distinguished from one another, they will be referred to as "sensor 10F." When sensors 10A to 10D are not particularly distinguished from one another, they will be referred to as "sensor 10."

[0015] The sensor 10 and the information processing device 20 can communicate with each other via a communication line. The communication line may be wired or wireless.

[0016] In the present disclosure, the global coordinate system is a coordinate system based on a position in the space in which the user walks. The global coordinate system is composed of, for example, an X-axis, a Y-axis, and a Z-axis. The X-axis, the Y-axis, and the Z-axis are perpendicular to each other. The X-axis is parallel to the left-right direction as seen from the user. In this embodiment, the positive direction of the X-axis is the direction from the left side to the right side of the user. The negative direction of the X-axis is the direction from the right side to the left side of the user. The Y-axis is parallel to the front-to-back direction as seen from the user. In this embodiment, the positive direction of the Y-axis is the direction from the rear side of the user to the front side of the user. The negative direction of the Y-axis is the direction from the front side of the user to the rear side of the user. The Z-axis is parallel to the up-down direction as seen from the user. In this embodiment, the positive direction of the Z-axis is the direction from the bottom side of the user to the top side of the user. The negative direction of the Z-axis is the direction from the top side of the user to the bottom side of the user. However, the positive and negative directions of the X-axis, Y-axis, and Z-axis may be set according to the configuration of the information processing system 1, etc.

[0017] In the present disclosure, a roll direction, a pitch direction, and a yaw direction are set in the global coordinate system. The roll direction is the direction of rotation around the Y axis. The pitch direction is the direction of rotation around the X axis. The yaw direction is the direction of rotation around the Z axis. The positive and negative directions of the roll direction, pitch direction, and yaw direction may be set depending on the configuration of the information processing system 1, etc.

[0018] In the present disclosure, the local coordinate system is a coordinate system based on the position of the sensor 10. FIG. 1 shows the local coordinate system of the sensor 10B as an example of a local coordinate system. The local coordinate system is composed of, for example, an x-axis, a y-axis, and a z-axis. The x-axis, the y-axis, and the z-axis are perpendicular to one another. The x-axis is parallel to the left-right direction as seen from the sensor 10. The y-axis is parallel to the front-rear direction as seen from the sensor 10. The z-axis is parallel to the up-down direction as seen from the sensor 10. The positive and negative directions of the x-axis, y-axis, and z-axis may be set according to the configuration of the information processing system 1, etc.

[0019] The sensor 10 is worn by a user on a body part of the user. The sensor 10 detects local sensor data that indicates the movement of the body part on which the sensor 10 is worn. The local sensor data is data in a local coordinate system. Hereinafter, sensor data obtained by converting the local sensor data into a global coordinate system by the process described below will be referred to as "global sensor data." When there is no particular distinction between local sensor data and global sensor data, they will be referred to as "sensor data."

[0020] The sensor data detected by the sensor 10 is data indicating the movement of the body part to which the sensor 10 is attached. By attaching such a sensor 10 to the body part of the user, the information processing system 1 can detect the movement of the user.

[0021] Here, the sensor 10 is attached to a body part of the user by the user. When the user attaches the sensor 10 to a body part, the orientation of the attached sensor 10 may deviate from an expected orientation. If the orientation of the sensor 10 deviates from the expected orientation, the information processing system 1 may not be able to accurately detect the user's movements. Even if the orientation of the sensor 10 deviates from the expected orientation, the information processing system 1 according to this embodiment generates correction data for correcting the sensor data in accordance with the orientation of the sensor 10, as will be described later. With this configuration, the information processing system 1 according to this embodiment can accurately detect the user's movements even if the orientation of the sensor 10 deviates from the expected orientation.

[0022] [Sensor configuration] The sensor 10A is worn on the user's head. For example, the sensor 10A is worn on the user's ear. The sensor 10A may be a wearable device. The sensor 10A may be an earphone or may be included in an earphone. Alternatively, the sensor 10A may be a device that can be retrofitted to existing glasses, earphones, or the like.

[0023] The sensor 10A detects local sensor data indicating the movement of the user's head. The local sensor data detected by the sensor 10A includes, for example, at least one of the acceleration, velocity, angular velocity, and angle of the user's head.

[0024] The sensor 10B is worn on the user's forearm. For example, the sensor 10B is worn on the user's wrist. The sensor 10B may be worn on the user's left forearm or on the user's right forearm. The sensor 10B may be a wristwatch-type wearable device.

[0025] The sensor 10B detects local sensor data indicating the movement of the user's forearm, e.g., wrist. The local sensor data detected by the sensor 10B includes, for example, at least one of the acceleration, velocity, angular velocity, and angle data of the user's forearm.

[0026] The sensor 10C is attached to the user's waist. The sensor 10C may be a wearable device. The sensor 10C may be attached to the user's waist by a belt, a clip, or the like.

[0027] The sensor 10C detects local sensor data indicating the movement of the user's waist. The local sensor data detected by the sensor 10C includes, for example, at least one of the acceleration, velocity, angular velocity, and angle data of the user's waist.

[0028] The sensor 10D-1 is worn on the user's left thigh. The sensor 10D-2 is worn on the user's right thigh. The sensor 10D may be a wearable device. The sensor 10D may be attached to the user's thigh by a belt, a clip, or the like. The sensor 10D may be attached to the thigh by being placed in a pocket near the thigh of pants worn by the user.

[0029] The sensor 10D-1 detects local sensor data indicating the movement of the user's left thigh. The sensor 10D-2 detects local sensor data indicating the movement of the user's right thigh. The local sensor data detected by the sensor 10D includes, for example, at least one of the acceleration, velocity, angular velocity, and angle data of the user's thigh.

[0030] The sensor 10E-1 is worn on the user's left ankle. The sensor 10E-2 is worn on the user's right ankle. The sensor 10E may be a wearable device. The sensor 10E may be attached to the user's ankle by a belt, a clip, or the like.

[0031] Sensor 10E-1 detects local sensor data indicating the movement of the user's left ankle. Sensor 10E-2 detects local sensor data indicating the movement of the user's right ankle. The local sensor data detected by sensor 10E includes, for example, at least one of the acceleration, velocity, angular velocity, and angle data of the user's ankle.

[0032] The sensor 10F-1 is attached to the user's left foot. The sensor 10F-2 is attached to the user's right foot. In this embodiment, the foot is the part of the user from the ankle to the toes. The sensor 10F may be a shoe-shaped wearable device. The sensor 10F may be provided in the shoe.

[0033] The sensor 10F-1 detects local sensor data indicating the movement of the user's left foot. The sensor 10F-2 detects local sensor data indicating the movement of the user's right ankle. The local sensor data detected by the sensor 10F includes, for example, at least one of the acceleration, velocity, angular velocity, and angle data of the user's foot.

[0034] As shown in FIG. 2, the sensor 10 includes a communication unit 11, a sensor unit 12, a storage unit 13, and a control unit 14.

[0035] The communication unit 11 includes at least one communication module capable of communicating with the information processing device 20 via a communication line. The communication module is a communication module that complies with the standard of the communication line. The standard of the communication line is, for example, a short-range wireless communication standard including Bluetooth (registered trademark), infrared, and NFC (Near Field Communication), or a wired communication standard.

[0036] The sensor unit 12 is configured to include any sensor corresponding to the local sensor data to be detected by the sensor 10. The sensor unit 12 is configured to include at least one of, for example, an inertial measurement unit (IMU), a three-axis motion sensor, a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis geomagnetic sensor.

[0037] The storage unit 13 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The storage unit 13 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 13 stores data used in the operation of the sensor 10 and data obtained by the operation of the sensor 13. The storage unit 13 may also store a program executed by the control unit 14.

[0038] The control unit 14 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 14 controls each part of the sensor 10 and executes processes related to the operation of the sensor 10.

[0039] The control unit 14 receives an instruction signal from the information processing device 20 via the communication unit 11. This instruction signal is a signal instructing the sensor unit 12 to start detecting local sensor data. Upon receiving this instruction signal, the control unit 14 causes the sensor unit 12 to start detecting local sensor data. The control unit 14 acquires local sensor data from the sensor unit 12 at a preset time interval and transmits the acquired local sensor data to the information processing device 20. This time interval may be set based on the walking speed of a typical user, etc.

[0040] [Configuration of information processing device] The information processing device 20 is carried by a user, for example. The information processing device 20 is a mobile device such as a smartphone or a tablet, for example. However, the information processing device 20 may be any device.

[0041] As shown in FIG. 2, the information processing device 20 includes a communication unit 21, an input unit 22, an output unit 23, a storage unit 24, and a control unit 25.

[0042] The communication unit 21 includes at least one communication module capable of communicating with the sensor 10 via a communication line. The communication module is at least one communication module compatible with the standard of the communication line. The standard of the communication line is, for example, a short-range wireless communication standard including Bluetooth (registered trademark), infrared, and NFC (Near Field Communication), or a wired communication standard.

[0043] The input unit 22 can accept input from a user. The input unit 22 includes at least one input interface that can accept input from a user. The input interface is, for example, a physical key, a capacitance key, a pointing device, a touch screen that is integrated with a display, a microphone, or the like.

[0044] The output unit 23 is capable of outputting data. The output unit 23 includes at least one output interface capable of outputting data. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electronic Luminescence) display.

[0045] The storage unit 24 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The storage unit 24 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 24 stores data used in the operation of the information processing device 20 and data obtained by the operation of the information processing device 20. The storage unit 24 may also store a program executed by the control unit 25.

[0046] The control unit 25 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA or an ASIC. The control unit 25 executes processes related to the operation of the information processing device 20 while controlling each unit of the information processing device 20.

[0047] The control unit 25 receives an execution instruction input from the user via the input unit 22. This input is for causing the information processing device 20 to execute a process for detecting the user's movement. The user inputs this input via the input unit 22 after wearing the sensor 10 on themselves. When the control unit 25 receives this input, it transmits an instruction signal to at least one sensor 10 via the communication unit 21. When this instruction signal is transmitted to the sensor 10, local sensor data is transmitted from the sensor 10 worn by the user to the information processing device 20.

[0048] Here, when the user inputs the execution instruction from the input unit 22, the operations [1], [2], and [3] are performed as described below with reference to FIG.

[0049] [1] After wearing the sensor 10, the user pauses from exercising. In this embodiment, this pause in the user's exercise is considered to be a first pause. In the present disclosure, a pause in the user's exercise refers to a state in which the user's body sway is below a predetermined level. The duration of the first pause may be set in advance according to the specifications of the information processing system 1. When the first pause is a pause in the user's exercise as in this embodiment, the duration of the first pause is, for example, two or three seconds. In this case, the user pauses from exercising by standing upright for two or three seconds. As described below, the first pause may be any pause in the exercise of the user's body part to which the sensor 10 is attached. In other words, the user does not necessarily have to pause from exercising. As described below, the user may be exercising continuously as long as the first pause is a pause in the exercise of the body part to which the sensor 10 is attached.

[0050] [2] After the first pause, the user exercises. The user may do any exercise. Examples of such exercise include walking, running, or dancing. The section in which the user exercises is referred to as the "exercise section."

[0051] [3] After exercising, the user pauses again. In this embodiment, this second pause is considered to be the second pause. The duration of the second pause may be preset according to the specifications of the information processing system 1. When the second pause is a second pause in the user's exercise, as in this embodiment, the duration of the second pause is, for example, two or three seconds. In this case, the user may pause the exercise by standing upright for two or three seconds, in the same or similar manner as in [1] above. Here, as described below, the second pause may be a pause in the body part to which the sensor 10 attached, where the first pause was detected, again. In other words, the user does not necessarily have to pause the exercise again. As described below, if the second pause is a pause in the body part to which the sensor 10 attached, the user may continue exercising.

[0052] The control unit 25 detects the first pause and the second pause from the local sensor data received from the sensor 10. An example of this process will be described later in the pause detection process. When the control unit 25 detects the first pause and the second pause, it corrects the sensor data detected by the sensor 10 during the movement interval based on the sensor data detected by the sensor 10 during the first pause and the sensor data detected by the sensor 10 during the second pause. In this embodiment, the control unit 25 generates correction data for correcting the sensor data detected by the sensor 10 during the movement interval. An example of this process will be described later in the correction data generation process.

[0053] <Pause detection process> The control unit 25 receives local sensor data from the sensor 10 via the communication unit 21. The control unit 25 determines whether the shaking of the sensor 10 has stopped for a first period of time based on the local sensor data. If the control unit 25 determines that the shaking of the sensor 10 has stopped for the first period of time, it determines that the body part to which the sensor 10 is attached has stopped exercising. In this embodiment, if the control unit 25 determines that the shaking of the sensor device 10 has stopped for the first period of time, it determines that the user has stopped exercising. The first period of time may be set taking into consideration the time the user stops exercising in [1] and [3] above. The first period of time is, for example, 2 or 3 seconds.

[0054] Here, the user's pause in exercise refers to a state in which the user's body sway is below a predetermined level, as described above. Whether the user's body sway is below a predetermined level can be evaluated based on acceleration or angular velocity data detected by the sensor 10 attached to the user's body part.

[0055] Therefore, when the local sensor data includes data on the acceleration of a body part of the user, the control unit 25 may evaluate the shaking of the sensor 10 that detected the local sensor data using the mean absolute error (MAE) of the acceleration data of the body part. However, the control unit 25 is not limited to the mean absolute error, and may use any method to determine whether the shaking of the sensor 10 has stopped for the first time. Below, a process for determining whether the shaking of the sensor 10 has stopped for the first time using the mean absolute error will be described.

[0056] <<Mean Absolute Error>> First, the control unit 25 calculates the absolute value A of the acceleration included in the local sensor data by the formula (1).

[0057]

number

[0058] Next, the control unit 25 sequentially calculates the average value of the absolute value A of the acceleration and the mean absolute error of the absolute value A of the acceleration by using the formulas (2) and (3). n+1 The average value and mean absolute error of the absolute value of acceleration A at time T n The mean absolute error of the acceleration A is calculated sequentially.

[0059]

number

number

[0060] If the time period during which the mean absolute error of the absolute value A of the acceleration is equal to or less than the error threshold continues for a first time, the control unit 25 determines that the shaking of the sensor 10 has stopped for a first time. In other words, if the time period during which the mean absolute error of the absolute value A of the acceleration is equal to or less than the error threshold continues for a first time, the control unit 25 determines that the body part to which the sensor 10 is attached has stopped exercising, and determines that the user has stopped exercising.

[0061] The error threshold may be set according to the body part on which the sensor 10 that evaluates the sway, i.e., the sensor 10 that detected the local sensor data used to calculate the mean absolute error, is worn. For example, when a user stops exercising with their feet on the ground, the sensor 10 worn on a body part closer to the user's feet will experience less sway. In contrast, the sensor 10 worn on a body part farther from the user's feet will experience more sway. Therefore, the error threshold may be set smaller for sensors 10 worn on body parts closer to the user's feet.

[0062] <<Another example of mean absolute error>> As described above, whether the shaking of the user's body is below a predetermined level can be evaluated based on acceleration or angular velocity data detected by the sensor 10 attached to the user's body part. Therefore, instead of calculating the mean absolute error of the absolute value A of the acceleration, the control unit 25 may calculate the mean absolute error of the absolute value of the angular velocity included in the local sensor data. In this case, Equations (1), (2), and (3) may be appropriately modified according to the angular velocity. The error threshold used to determine the acceleration may be modified according to the angular velocity. Alternatively, the control unit 25 may determine whether the shaking of the sensor 10 has stopped for a first period based on time-series changes in the absolute value, standard deviation, variance, amplitude, or the like of the sensor data. The absolute value A of the angular velocity calculated using the modified Equation (1) is also referred to as a composite angular velocity, which is a composite of the absolute values ​​of the angular velocities of the x-axis, y-axis, and z-axis.

[0063] Alternatively, the control unit 25 may simply determine that the body part to which the sensor 10 is attached has stopped exercising if the absolute value of the acceleration or angular velocity detected by the sensor 10 attached to the body part of the user is equal to or less than a threshold. With this configuration, it is possible to determine whether the swaying of the user's body is equal to or less than a predetermined level. The threshold may be set according to the body part to which the sensor 10 for evaluating the swaying is attached.

[0064] <Correction processing> When the control unit 25 detects the first pause and the second pause, it corrects the sensor data detected by the sensor 10 during the movement interval based on the sensor data detected by the sensor 10 during each of the first pause and the second pause. In this embodiment, the control unit 25 generates correction data for correcting the sensor data detected by the sensor 10 during the movement interval based on the sensor data. The following describes a method in which the control unit 25 generates correction data for correcting global sensor data. However, the control unit 25 may also generate correction data for correcting local sensor data.

[0065] <<Acceleration correction data>> The following describes a process for generating correction data for correcting acceleration data included in the global sensor data. When the duration of the first pause and the second pause is 2 seconds or 3 seconds or more, the control unit 25 may generate correction data for correcting the acceleration data.

[0066] First, the control unit 25 converts the local sensor data in the local coordinate system into global sensor data in the global coordinate system. As an example of this process, the control unit 25 detects the direction of gravitational acceleration acting on the sensor 10 based on acceleration or angular velocity data included in the local sensor data. The control unit 25 detects the attitude of the sensor 10 based on the detected direction of gravitational acceleration acting on the sensor 10. The control unit 25 calculates a rotation matrix that converts the data in the local coordinate system into data in the global coordinate system based on the detected attitude of the sensor 10. The control unit 27 converts the local sensor data in the local coordinate system into global sensor data in the global coordinate system based on the calculated rotation matrix.

[0067] Next, the control unit 25 acquires acceleration data at the end point of the first pause from the global sensor data, and acquires acceleration data at the start point of the second pause from the global sensor data.

[0068] The control unit 25 generates correction data for correcting the acceleration data in the exercise section based on the acceleration data at the end of the first pause and the acceleration data at the start of the second pause. Here, at the end of the first pause, the exercise of the body part of the user to which the sensor 10 is attached is stopped. Similarly, at the start of the second pause, the exercise of the body part of the user to which the sensor 10 is attached is stopped. Therefore, if the orientation of the sensor 10 attached to the body part is the same as the expected orientation, the acceleration detected by the sensor 10 at the end of the first pause and the start of the second pause will be zero. However, if the orientation of the sensor 10 attached to the body part deviates from the expected orientation, the gravitational acceleration acting on the sensor 10 may affect the acceleration data in at least one of the X-axis and Y-axis directions. In this case, at the end of the first pause and the start of the second pause, the acceleration in at least one of the X-axis and Y-axis directions detected by the sensor 10 will not be zero. Therefore, correction data for correcting the acceleration data can be generated based on the acceleration data at the end point of the first pause and the acceleration data at the start point of the second pause.

[0069] As an example of the correction data, the control unit 25 may generate the correction data based on linear correction using the acceleration data at the end point of the first pause and the acceleration data at the start point of the second pause. Linear correction is correcting the data between two points using correction data generated by linearly connecting the two points. For example, the control unit 25 generates the correction data Da for the acceleration data at time t by generating the formula (4): i The subscript i indicates whether the data is for correction of acceleration data in the X-axis direction (i=X) or acceleration data in the Y-axis direction (i=Y).

[0070] Da i (t)=a iT1 +(a iT2 -a iT1 ) / (T2-T1)×(t-T1) Formula (4) In equation (4), time T1 is the end time of the first pause. Time T2 is the start time of the second pause. Acceleration a iT1 is the acceleration at time T1. Acceleration a iT2 is the acceleration at time T2.

[0071] The control unit 25 derives correction data Da from the acceleration data included in the global sensor data. i The acceleration data included in the global sensor data is corrected by subtracting (t). The control unit 25 can calculate the displacement of the body part to which the sensor 10 is attached during the motion interval by integrating the corrected acceleration data twice. The control unit 25 may obtain data on the position trajectory of the body part to which the sensor 10 is attached during the motion interval by calculating the displacement of the body part to which the sensor 10 is attached during the motion interval.

[0072] For example, the upper part of FIG. 4 shows a graph of the acceleration of the user's head in the Y-axis direction. In this graph, the vertical axis represents the acceleration [G] of the user's head in the Y-axis direction. The horizontal axis represents time [s]. The dashed line represents the data of the head's acceleration in the Y-axis direction before correction. The solid line represents the data of the head's acceleration in the Y-axis direction after correction. These acceleration data are detected by sensor 10A.

[0073] In the graph at the top of FIG. 4, the time after T2 is a second pause, during which the body part of the user wearing the sensor 10 stops moving. Therefore, after time T2, the gravitational acceleration of the head in the Y direction ideally becomes zero. However, as shown by the dashed line, the pre-correction head acceleration in the Y direction does not become zero after time T2. This is because the orientation of the sensor 10A deviates from the expected orientation, causing the gravitational acceleration acting on the sensor 10A to appear in the Y direction. The control unit 25 generates correction data of Equation (4) using the pre-correction head acceleration data at times T1 and T2. The control unit 25 generates corrected head acceleration data, shown by the solid line, by subtracting the correction data of Equation (4) from the pre-correction head acceleration data. The control unit 25 calculates the displacement of the user's head by integrating the corrected head acceleration data twice.

[0074] The bottom of Figure 4 shows a graph of the displacement of the user's head in the Y-axis direction. In this graph, the vertical axis represents the displacement [m] of the user's head in the Y-axis direction. The horizontal axis represents time [s]. The solid line represents the displacement of the user's head calculated from the corrected head acceleration data. The dashed line represents the displacement of the user's head calculated from the pre-correction head acceleration data. As described above, from time T2 onwards, the user's body part to which sensor 10 is attached enters a second rest period in which it stops moving. Therefore, it is desirable that the displacement of the user's head remains constant from time T2 onwards. As shown by the solid line in the graph at the bottom of Figure 4, the displacement of the user's head calculated from the corrected head acceleration data remains constant from time T2 onwards.

[0075] <<Speed ​​correction data>> The process of generating correction data for correcting the speed data included in the global sensor data will be described below.

[0076] First, the control unit 25 converts the local sensor data in the local coordinate system into global sensor data in the global coordinate system in the same or similar manner as the above-mentioned process.

[0077] Next, the control unit 25 acquires speed data at the end point of the first pause from the global sensor data. The control unit 25 also acquires speed data at the start point of the second pause from the global sensor data. Here, if the global sensor data does not include speed data, the control unit 25 may acquire the speed data at the end point of the first pause by integrating once the acceleration of the global sensor data at the end point of the first pause. The control unit 25 may also acquire the speed data at the start point of the second pause by integrating once the acceleration of the global sensor data at the start point of the second pause.

[0078] The control unit 25 generates correction data for correcting the velocity data based on the velocity data at the end of the first pause and the velocity data at the start of the second pause. In the same manner as or similar to the acceleration described above, if the orientation of the sensor 10 attached to the body part is the same as the expected orientation, the velocity detected by the sensor 10 at the end of the first pause and the start of the second pause will be zero. However, if the orientation of the sensor 10 attached to the body part deviates from the expected orientation, gravitational acceleration acting on the sensor 10 may affect the acceleration data in at least one of the X-axis and Y-axis directions. If the gravitational acceleration acting on the sensor 10 affects the acceleration data in at least one of the X-axis and Y-axis directions, it will also affect the velocity data in at least one of the X-axis and Y-axis directions detected by the sensor 10. In this case, at the end of the first pause and the start of the second pause, the velocity in at least one of the X-axis and Y-axis directions detected by the sensor 10 will not be zero. Therefore, the control unit 25 can generate correction data for correcting the speed data based on the speed data at the end point of the first pause and the speed data at the start point of the second pause.

[0079] As an example of the correction data, the control unit 25 may generate the correction data by linear correction using the speed data at the end point of the first pause and the speed data at the start point of the second pause. Linear correction is correcting data between two points using correction data generated by linearly connecting the two points. For example, the control unit 25 generates the correction data Dv of the speed data at time t by generating the formula (5). i The subscript i indicates whether the data is for correction of acceleration data in the X-axis direction (i=X) or acceleration data in the Y-axis direction.

[0080] Dv i (t)=v iT1 +(v iT2 -v iT1 ) / (T2-T1)×(t-T1) Formula (5) In equation (5), time T1 is the end time of the first pause. Time T2 is the start time of the second pause. The velocity v iT1 is the velocity at time T1. Velocity v iT2 is the acceleration at time T2.

[0081] The control unit 25 calculates the correction data Dv from the speed data included in the global sensor data. i The velocity data included in the global sensor data is corrected by subtracting (t). The control unit 25 can calculate the displacement of the body part to which the sensor 10 is attached by integrating the corrected velocity data once. The control unit 25 may obtain data on the trajectory of the position of the body part to which the sensor 10 is attached during the motion section by calculating the displacement of the body part to which the sensor 10 is attached during the motion section.

[0082] For example, the center of FIG. 4 shows a graph of the velocity of the user's head in the Y-axis direction. In this graph, the vertical axis represents the velocity of the user's head in the Y-axis direction [m / s]. The horizontal axis represents time [s]. The dashed line represents the data of the head velocity in the Y-axis direction before correction. The solid line represents the data of the head velocity in the Y-axis direction after correction. In the graph in the center of FIG. 4, the control unit 25 generates the correction data of equation (5) using the data represented by the dashed line. The control unit 25 generates the corrected data represented by the solid line by subtracting the correction data of equation (5) from the data represented by the dashed line. The control unit 25 calculates the displacement of the user's head by integrating the corrected head velocity data once. The head displacement calculated from the corrected head velocity data becomes constant after time T2, as shown by the solid line in the graph at the bottom of FIG. 4.

[0083] <Experiment 1> Figure 5 shows the experimental results when the subject took one step. In Figure 5, the vertical axis represents the subject's displacement [m] in the Y-axis direction, and the horizontal axis represents the subject's displacement [m] in the X-axis direction.

[0084] The estimated trajectory is trajectory data obtained by correcting sensor data acquired by an inertial measurement unit (IMU) attached to the ankle of the subject. In other words, the estimated trajectory is data on the trajectory of the subject's position detected (estimated) by processing according to this embodiment. This estimated trajectory was obtained by integrating the velocity of the subject's ankle once. The correction data was generated from the velocity data. In other words, the estimated trajectory was generated by correcting the velocity data.

[0085] The ground truth trajectory is data on the trajectory of the subject's position obtained by motion capture (Mocap).

[0086] The estimated trajectory has a small deviation from the correct trajectory. Therefore, it can be seen that the present embodiment allows the user's movement to be detected with high accuracy.

[0087] <Experiment 2> Figure 6 shows the experimental results when the subject walked with their legs together. Figure 7 shows the experimental results when the subject walked backwards. Figure 8 shows the experimental results when the subject walked with small steps. In Figures 6 to 8, the vertical axis shows the subject's displacement [m] in the Y-axis direction. The horizontal axis shows the subject's displacement [m] in the X-axis direction. The results in Figures 6 to 8 are the experimental results when the subject took one step.

[0088] The estimated trajectory is trajectory data obtained by correcting sensor data acquired by an inertial measurement unit (IMU) attached to the ankle of the subject, as in Fig. 5. In other words, the estimated trajectory is trajectory data of the subject's position detected (estimated) by processing according to this embodiment. As in Fig. 5, the estimated trajectory is generated by correcting velocity data.

[0089] The ground truth trajectory is the data of the subject's position trajectory obtained by motion capture (Mocap), as in Figure 5.

[0090] 6 to 8, even when the subject walks in a variety of ways, the estimated trajectory has a small deviation from the correct trajectory. Therefore, according to this embodiment, it is clear that the user's movement can be detected with high accuracy even when the user walks in a variety of ways.

[0091] Here, gait indices used in walking rehabilitation can be calculated from the walker's posture and data on the ankle position trajectories as shown in Figures 6 to 8. Gait indices are indices for evaluating the walking condition of a walker. Examples of gait indices include the foot direction angle and stride length as shown in Figure 9. The foot direction angle is the angle between the direction of the foot during the first rest and the user's forward direction. The direction of the foot during the first rest is along the y-axis of the local coordinate system. The stride length is the length of one walking cycle relative to the user's forward direction.

[0092] The inventors verified whether gait indices can be calculated with high accuracy from data on the trajectory of the ankle position of a subject detected (estimated) by processing according to this embodiment. The verification results are shown in Figs. 10 to 12. In Figs. 10 to 12, the estimated values ​​are values ​​calculated based on estimated trajectory data obtained in a manner identical to or similar to the estimated trajectory data shown in Fig. 5. The correct values ​​are values ​​calculated based on correct trajectory data obtained in a manner identical to or similar to the correct trajectory data shown in Fig. 5. In Figs. 10 to 12, the horizontal axis indicates the number of walking steps (steps).

[0093] FIG. 10 shows data on the foot direction angle when the subject walked with their legs together. FIG. 10 also shows data on the foot direction angle of the subject's left foot. In FIG. 10, the direction in which the ankle rotates clockwise is the positive direction, and the opposite is the negative direction. When the subject's left foot faces the subject's knee-to-knee walking, the subject's ankle rotates clockwise, and the data shown in FIG. 10 becomes negative.

[0094] When subjects walk with their feet turned inward, they turn their feet inward. Therefore, it is believed that the foot direction angle is a good indicator of the walking state when walking with their feet turned inward. Therefore, the inventors selected the foot direction angle as a gait indicator when walking with their feet turned inward. In Figure 10, the vertical axis indicates the foot direction angle [deg]. The average error from the estimated value to the correct value was 0.8 degrees. This result shows that the foot direction angle when the user walks with their feet turned inward can be calculated with high accuracy from the data of the foot position trajectory detected by the processing according to this embodiment.

[0095] FIG. 11 shows stride length data when the subject walked backwards. When the subject walked backwards, the stride length was a negative value. This is because the direction of the ankle velocity was the negative Y-axis of the global coordinate system when the subject walked backwards. Therefore, it is believed that stride length effectively represents the characteristics of the walking state when walking backwards. Therefore, the inventors selected stride length as the gait index when walking backwards. In FIG. 11, the vertical axis represents stride length [m]. The average error from the correct value of the estimated value was 3 cm. This result shows that the stride length when the user walked backwards can be calculated with high accuracy from the data of the ankle position trajectory detected by the processing according to this embodiment.

[0096] FIG. 12 shows data on stride length when the subject took small steps. When the subject took small steps, their step length became narrower than when they walked normally. Therefore, it is believed that stride length better represents the characteristics of the walking state when walking with small steps. Therefore, the inventors selected stride length as the gait index for small steps. In FIG. 12, the vertical axis represents stride length [m]. The average error from the correct value of the estimated value was 1.6 cm. This result shows that the stride length when the user took small steps can be calculated with high accuracy from the data of the ankle position trajectory detected by the processing according to this embodiment.

[0097] Here, the processing according to this embodiment can correct any sensor data in the X-axis, Y-axis, and Z-axis directions. In other words, the processing according to this embodiment can obtain data on the three-dimensional trajectory of the user's body parts. This experimental example is shown in Figures 13 to 15.

[0098] Figure 13 shows the experimental results when the subject walked with their legs together. Figure 14 shows the experimental results when the subject walked backwards. Figure 15 shows the experimental results when the subject walked with small steps. In Figures 13 to 15, the horizontal axis represents the subject's displacement [m] in the Y-axis direction. The vertical axis represents the subject's displacement [m] in the Z-axis direction. In other words, the data shown in Figures 13 to 15 is data on the subject's displacement in the sagittal plane.

[0099] 13 to 15, the estimated trajectory is trajectory data obtained by correcting sensor data acquired by an inertial measurement unit (IMU) attached to the ankle of the subject, as in Fig. 5. The ground truth trajectory is trajectory data of the subject's position obtained by motion capture (Mocap), as in Fig. 5.

[0100] 13 to 15, the deviation of the estimated trajectory from the correct trajectory is small even in the sagittal plane, which shows that the three-dimensional movement of the user can be detected with high accuracy according to this embodiment.

[0101] [Operation of information processing device] 16 is a flowchart showing an example of the operation of the information processing device 20 according to the first embodiment of the present disclosure. This operation of the information processing device 20 corresponds to the information processing method according to the present embodiment. When the control unit 25 receives an execution instruction input via the input unit 22, it starts the processing of step S1.

[0102] The control unit 25 transmits an instruction signal to the sensor 10 attached to a body part of the user via the communication unit 21 (step S1). When this instruction signal is transmitted to the sensor 10, local sensor data is transmitted from the sensor 10 to the information processing device 20.

[0103] The control unit 25 receives the local sensor data from the sensor 10 via the communication unit 21 (step S2).

[0104] The control unit 25 determines whether the user has stopped exercising based on the local sensor data received in the process of step S1 (step S3). As described above in the process of detecting a pause, the control unit 25 determines whether the user has stopped exercising by determining based on the local sensor data whether the vibration of the sensor 10 has stopped for a first period of time.

[0105] If the control unit 25 determines that the user has stopped exercising (step S3: YES), the control unit 25 proceeds to the process of step S4. If the control unit 25 does not determine that the user has stopped exercising (step S3: NO), the control unit 25 returns to the process of step S2.

[0106] In the process of step S4, the control unit 25 determines whether two pauses have been detected. That is, the control unit 25 determines whether a first pause and a second pause have been detected. If the control unit 25 determines that two pauses have been detected (step S4: YES), the control unit 25 proceeds to the process of step S5. If the control unit 25 does not determine that two pauses have been detected (step S4: NO), the control unit 25 returns to the process of step S2.

[0107] In the process of step S5, the control unit 25 determines whether the length of the exercise section between the first pause and the second pause is equal to or less than a second time. The second time may be set based on the expected exercise the user will perform. For example, if the user is expected to take a few steps, the second time is 2 or 3 seconds.

[0108] If the control unit 25 determines that the duration of the motion section is equal to or shorter than the second time (step S5: YES), the control unit 25 proceeds to the processing of step S6. If the control unit 25 determines that the duration of the motion section exceeds the second time (step S5: NO), the control unit 25 ends the processing as shown in FIG. 16. By performing the processing of step S5, it is possible to eliminate movements other than those expected in the information processing system 1. For example, if the expected movement is walking a few steps in the above [2], it is possible to eliminate movements other than walking a few steps by performing the processing of step S5.

[0109] In the process of step S6, the control unit 25 converts the local sensor data in the local coordinate system into global sensor data in the global coordinate system.

[0110] In the process of step S7, the control unit 25 generates correction data for correcting the global sensor data of the motion section based on the global sensor data at the end point of the first pause and the global sensor data at the start point of the second pause. For example, the control unit 25 generates the correction data of equation (4) or equation (5).

[0111] In the process of step S8, the control unit 25 uses the correction data generated in step S7 to calculate the displacement of the body part to which the sensor 10 is attached during the exercise section.

[0112] Here, in the processing of step S3, when a pause in the first exercise is detected, that is, when a first pause is detected, control unit 25 may cause output unit 23 to output a message indicating that the first pause has been detected. For example, control unit 25 may cause a speaker of output unit 23 to output a sound indicating that the first pause has been detected. By outputting the message indicating that the first pause has been detected, the user knows that the first pause has been detected, and can continue exercising as described in [2] above.

[0113] As described above, in the information processing device 20 according to the first embodiment, the control unit 25 detects a first pause and a second pause. Furthermore, the control unit 25 corrects the sensor data detected by the sensor 10 during the exercise interval based on the sensor data detected by the sensor 10 during the first pause and the sensor data detected by the sensor 10 during the second pause. Here, during the first and second pauses, the user's body part to which the sensor 10 is attached stops exercising. Therefore, if the orientation of the sensor 10 attached to the body part is the same as the expected orientation, the acceleration or velocity detected by the sensor 10 during the first and second pauses will be zero. However, if the orientation of the sensor 10 attached to the body part deviates from the expected orientation, the gravitational acceleration acting on the sensor 10 may affect at least one of the accelerations in the X-axis and Y-axis directions. Therefore, if the orientation of the sensor 10 attached to the body part deviates from the expected orientation, the acceleration or velocity detected by the sensor 10 during the first and second pauses will not be zero. Therefore, the control unit 25 can correct the sensor data detected by the sensor 10 during the motion interval based on the sensor data detected by the sensor 10 during the first pause and the sensor data detected by the sensor 10 during the second pause. For example, the control unit 25 can generate correction data for correcting the sensor data by making the acceleration or velocity detected by the sensor 10 zero during the first pause and the second pause. With this configuration, in this embodiment, the sensor data detected by the sensor 10 can be corrected according to the posture of the sensor 10 without having to prepare an exercise model in advance.

[0114] Furthermore, in this embodiment, the control unit 25 can correct the sensor data detected by the sensor 10 even without an exercise model. Therefore, the processing according to this embodiment can be applied to users who cannot perform certain movements. For example, a user undergoing rehabilitation from an injury may have difficulty walking. Even for such a user, if the sensor 10 is attached to the user, the information processing device 20 according to this embodiment can accurately detect the user's movements by correcting the sensor data. Furthermore, the processing according to this embodiment can also be applied to a user in a wheelchair, as will be described later with reference to FIG. 17.

[0115] As a comparative example, consider a method in which markers or the like are attached to the user's body parts and the user's movements are detected by motion capture. This method requires the preparation of a geometric model, such as a link structure, that mimics the user's body structure.

[0116] In contrast, in this embodiment, the control unit 25 can accurately detect the user's movements by correcting the sensor data, without having to prepare a geometric model in advance, as long as it can detect the first and second pauses.

[0117] As another comparative example, consider a method of correcting the sensor data by detecting the posture angle of the sensor 10 from a captured image generated by capturing an image of a user wearing the sensor 10. In this comparative example, if the user is wearing a skirt, kimono, or the like, the body part where the sensor 10 is attached may not appear in the captured image, making it impossible to correct the sensor data.

[0118] In contrast, in this embodiment, the control unit 25 corrects the sensor data detected by the sensor 10 during the exercise interval based on the sensor data detected by the sensor 10 during the first and second pauses, rather than using captured images of the user wearing the sensor 10. With this configuration, the control unit 25 can correct the sensor data and detect the displacement of the body part to which the sensor 10 is attached, even if the body part is not visible from the outside due to clothing such as a skirt or kimono. Furthermore, in this embodiment, regardless of the body part to which the sensor 10 is attached, the control unit 25 can correct the sensor data and detect the displacement of the body part by detecting the first and second pauses.

[0119] Furthermore, in this embodiment, the control unit 25 may correct acceleration or velocity data included in the sensor data using correction data. For example, the control unit 25 may correct the acceleration data of the global sensor data using equation (4), or may correct the velocity data of the global sensor data using equation (5). The control unit 25 may obtain data on the trajectory of the position of the body part to which the sensor 10 is attached by integrating the corrected acceleration data twice or integrating the corrected velocity data once. Correcting the acceleration data or velocity data before integration allows for more accurate calculation of the displacement of the body part to which the sensor 10 is attached than correcting the displacement data of the body part to which the sensor 10 is attached.

[0120] <Application example> The process according to the first embodiment can be applied to a user who cannot walk, as long as the first pause and the second pause can be detected. For example, the process according to the first embodiment can be applied to a user who is in a wheelchair, as shown in FIG.

[0121] In FIG. 17, sensors 10A, 10B, and 10E are attached to a user. The user equipped with sensors 10A and the like is stationary in a wheelchair. In FIG. 17, an assistant operates the wheelchair. The assistant may carry an information processing device 20. In FIG. 17, in the first pause of [1] above, the assistant stops the wheelchair. In [2] above, the assistant pushes the wheelchair carrying the user, moving the user forward. In the second pause of [3] above, the assistant stops the wheelchair. Here, when the process shown in FIG. 16 is executed in the configuration shown in FIG. 17, the control unit 25 may, when the first pause is detected in the process of step S3, output a message from the output unit 23 prompting the user to start moving forward. For example, the control unit 25 may output a voice message saying, "Please move the wheelchair forward" from the speaker of the output unit 23. By outputting such a message from the output unit 23, the assistant can smoothly move the wheelchair forward after the first pause.

[0122] (Second embodiment) The information processing system according to the second embodiment can have the same or similar configuration as the information processing system 1 shown in Figures 1 and 2. Therefore, the information processing system 1 according to the second embodiment will be described below with reference to Figures 1 and 2.

[0123] As shown in FIG. 1, a roll direction, a pitch direction, and a yaw direction are set in the global coordinate system.

[0124] The roll direction is the direction of rotation within the XZ plane. The XZ plane includes the direction of gravitational acceleration. Here, the control unit 25 can detect the direction of gravitational acceleration acting on the sensor 10 based on the acceleration or angular velocity data included in the local sensor data. Since the XZ plane includes the direction of gravitational acceleration, the control unit 25 can estimate the roll direction, which is the direction of rotation within the XZ plane, based on the detected direction of gravitational acceleration acting on the sensor 10.

[0125] The pitch direction is the rotation direction within the YZ plane. The YZ plane includes the direction of gravitational acceleration. Since the YZ plane includes the direction of gravitational acceleration, the control unit 25 can estimate the pitch direction, which is the rotation direction within the YZ plane, based on the detected direction of gravitational acceleration acting on the sensor 10.

[0126] The yaw direction is the direction of rotation within the XY plane. The XY plane does not include the direction of gravitational acceleration. Therefore, even if the control unit 25 detects the direction of gravitational acceleration acting on the sensor 10, it cannot estimate the yaw direction from the detected direction of gravitational acceleration.

[0127] Here, the XY plane including the yaw direction includes the user's moving direction when the user moves forward. By the processing described above in the first embodiment, the control unit 25 can accurately detect the displacement of the body part to which the sensor 10 is attached during the motion section. Therefore, when the user moves forward, if the trajectory of the position of the body part during the motion section is taken as the user's moving direction, the yaw direction, which is the rotation direction within the XY plane, can be estimated based on the user's moving direction.

[0128] For example, as shown in FIG. 18, assume that sensors 10A-1 and 10A-2 are attached to the user's head as sensor 10A. The orientation of sensor 10A-1 is different from the orientation of sensor 10A-2. However, when the user moves forward, control unit 25 can accurately detect the trajectory of the head position, i.e., the user's traveling direction, by the processing according to the first embodiment, regardless of whether the local sensor data is detected by sensor 10A-1 or 10A-2. Furthermore, control unit 25 can estimate the yaw direction based on the user's traveling direction and the direction in which the y-axis direction is projected onto the ground.

[0129] [Operation of information processing device] 19 is a flowchart showing an example of the operation of the information processing device 20 according to the second embodiment of the present disclosure. This operation of the information processing device 20 corresponds to the information processing method according to this embodiment. After processing step S5 shown in FIG. 16, the control unit 25 proceeds to processing step S11. That is, when the control unit 25 determines that the duration of the exercise section is equal to or shorter than the second time (step S5: YES), it proceeds to processing step S11.

[0130] In the process of step S11, the control unit 25 detects the direction of gravitational acceleration acting on the sensor 10 based on the acceleration or angular velocity data of the local sensor data received in the process of step S2. The control unit 25 estimates the roll direction and pitch direction based on the detected direction of gravitational acceleration acting on the sensor 10.

[0131] In the process of step S12, the control unit 25 calculates a rotation matrix for converting the local sensor data in the local coordinate system into global sensor data in the global coordinate system, based on the roll direction and pitch direction estimated in the process of step S11.

[0132] In the process of step S13, the control unit 25 converts the local sensor data received in the process of step S2 into global sensor data using the rotation matrix calculated in the process of step S12.

[0133] In the process of step S14, the control unit 25 generates correction data for correcting the global sensor data in the movement section in the same or similar manner as in the process of step S7. For example, the control unit 25 generates the correction data of equation (4) or equation (5).

[0134] In the process of step S15, the control unit 25 uses the correction data generated in step S14 to calculate the trajectory of the position of the body part to which the sensor 10 is attached during the exercise section. The control unit 25 detects the trajectory of the position of the body part to which the sensor 10 is attached as the user's moving direction.

[0135] In the process of step S16, the control unit 25 estimates the yaw direction based on the user's traveling direction calculated in the process of step S15 and the direction in which the y-axis direction is projected onto the ground. The control unit 25 may estimate the ground based on the x-axis direction and the y-axis direction. The control unit 25 may project the y-axis direction onto the estimated ground.

[0136] <Experiment> The inventors conducted an experiment to confirm the estimation error of the yaw angle estimated by the above-described process.

[0137] (1) Motion capture In the experiment, the reference yaw angle was detected by motion capture. Motion capture is a system that detects the movement of an object by using a camera to detect the positions of markers attached to the object. In the experiment, a unit including a sensor 10 and four markers placed at four locations around the sensor 10 was used, as shown in Figure 20. In the experiment, the position and orientation of the sensor 10 were detected by detecting the positions of these four markers using a motion capture camera. Twelve cameras were used in the experiment. The units were worn on the head, wrists, and ankles of the subject.

[0138] (2) Subjects The experiment was conducted on three subjects. After the subjects were fitted with the unit shown in Figure 20, they first performed the first rest period described above in [1]. After that, the subjects took one step in the exercise section described above in [2], and then performed the second rest period described above in [3].

[0139] (3) Sensor posture detection method using motion capture In Figure 21, the starting point is the point at which the subject took the first pause described above in [1]. The end point is the point at which the subject took the second pause described above in [3]. The positions of the markers attached to the subject as units were detected by motion capture. The positions of the markers were detected as positions in a global coordinate system. This global coordinate system is not a global coordinate system detected by the sensor 10, but a global coordinate system fixed in space. The center position of the sensor 10 was set to the center of the positions of the four markers detected by motion capture. The attitude angle Ψ1 of the sensor 10 is the yaw angle when the position of the sensor 10 is at the starting point. The attitude angle Ψ1 of the sensor 10 was obtained from the assumed angles of the positions of the four markers. The moving direction of the sensor 10 (moving direction of the subject) is the direction from the starting point to the end point. The moving direction of the sensor 10 was obtained from the displacement of the center position of the sensor 10 from the starting point to the end point. The tilt angle of the moving direction of the sensor 10 from the Y-axis direction is described as "angle Ψ2." The yaw angle Ψ3 obtained by motion capture, i.e., the yaw angle Ψ3 that is the basis for the estimation error, was calculated as "Ψ3 = Ψ1 + Ψ2".

[0140] (4) Calculation method for yaw angle estimation error 22, the X', Y', and Z' axes are the X, Y, and Z axes of the global coordinate system detected by the sensor 10. The dashed lines indicate the trajectories of the subject's movements. The estimation error of the yaw angle was calculated as "E = Ψ4 - Ψ3", where E is the estimation error of the yaw angle. The angle Ψ4 is the mounting angle of the sensor 10. The angle Ψ4 is the tilt angle of the Y' axis relative to the traveling direction of the sensor 10. The accuracy of the traveling direction detected by the sensor 10 affects the estimation error of the yaw angle. For example, if the detection accuracy of the traveling direction of the sensor 10 deteriorates, the estimation error of the yaw angle will increase.

[0141] (5) Experimental results FIG. 23 shows a table of experimental results of yaw angle estimation error. FIG. 24 shows a graph of experimental results of yaw angle estimation error. The vertical axis of FIG. 24 corresponds to the yaw angle estimation error [degrees] in FIG. 23. The horizontal axis of FIG. 24 corresponds to the list number in the table of FIG. 23. Experiments were conducted on three subjects. However, inappropriate data has been removed from the experimental results shown in FIG. 23 and FIG. 24. Inappropriate data is, for example, data in which the subject's body swaying while resting is unnaturally large or the subject's walking speed is significantly slow.

[0142] As shown in Figure 23, the stride length of the subjects was classified as either "normal" or "short." A "normal" stride length is the stride length that the subject normally takes when walking. A "short" stride length is a stride length that is approximately 20 cm shorter than a "normal" stride length.

[0143] As shown in Figure 23, the walking speed of the subjects was categorized as either "average" or "slow." "Average" walking speed refers to the speed at which the subject normally walks. "Slow" walking speed refers to a speed about half the "average" walking speed.

[0144] As shown in Figure 23, the subject's body swaying during rest was categorized as "none," "stiff," or "swaying." "None" refers to the subject standing without swaying during rest periods. "Stiff" refers to the subject standing with slight swaying during rest periods. "Swaying" refers to the subject standing with swaying throughout the body during rest periods.

[0145] As shown in FIG. 23, the number of steps in the above-mentioned [2] movement section was set to one step. However, it was predicted that the estimation accuracy of the user's traveling direction would deteriorate if the stride length was "short" and the walking speed was "slow." As described above with reference to FIG. 22, when the detection accuracy of the user's traveling direction deteriorates, the estimation error of the yaw angle increases. In FIG. 23, list numbers 4, 5, and 6 correspond to the conditions that the stride length is "short" and the walking speed is "slow." Therefore, the number of steps was set to one step for list number 4, two steps for list number 5, and three steps for list number 6. In other words, it was verified whether the estimation error of the yaw angle improves depending on the number of steps.

[0146] As shown in Figures 23 and 24, for List No. 1, whose stride length and walking speed were both "average," the average yaw angle estimation error was 6 degrees. The worst value was 19 degrees. The worst value is the yaw angle estimation error of the subject with the largest yaw angle estimation error among the three subjects. The results for List No. 1 show that, for everyday walking, the yaw angle can be estimated accurately within about one step.

[0147] Furthermore, if the subject's stride is "short," the data of the trajectory of the body part position detected by the sensor 10 will be short. This is thought to result in a deterioration in the accuracy of the detection of the user's direction of travel, and a deterioration in the accuracy of the estimation error in the yaw direction. Furthermore, if the walking speed is "slow," the accuracy of the detection of the user's direction of travel will be poor, and a deterioration in the accuracy of the estimation error in the yaw direction will be poor. Furthermore, if the subject's body sways during rest, it becomes difficult to perform the first and second rests. This is thought to result in a deterioration in the accuracy of the detection of the user's direction of travel, and a deterioration in the accuracy of the estimation error in the yaw direction. In summary, the subject's "short" stride, "slow" walking speed, and "swaying" body sway during rest are thought to be error factors that worsen the estimation error in the yaw direction. The results shown in Figures 23 and 24 confirm that the addition of these error factors increases the estimation error and worst value in the yaw direction.

[0148] As described above, in the second embodiment, the control unit 25 determines the trajectory of the positions of the body parts to which the sensors 10 are attached as the user's moving direction, and estimates the yaw direction of the global coordinate system based on the user's moving direction. By being able to estimate the yaw direction of the global coordinate system in this way, the sensor 10 can detect the user's movement in the yaw direction. With this configuration, it is possible to detect a wider variety of user movements.

[0149] <Application example> The process according to the second embodiment can be applied to a user who cannot walk, as long as the first pause and the second pause can be detected in the same or similar manner as in the first embodiment. For example, the process according to the second embodiment can be applied to a user who is in a wheelchair, as described above with reference to FIG.

[0150] (Third embodiment) The information processing system according to the third embodiment may have the same or similar configuration as the information processing system 1 shown in Figures 1 and 2. Therefore, the information processing system 1 according to the third embodiment will be described below with reference to Figures 1 and 2.

[0151] The control unit 25 may not detect a pause in the user's exercise. For example, if the user is unfamiliar with using the information processing system 1, the user may pause for a short period of time. In this case, if the vibration of the sensor 10 does not stop for the first period of time, the control unit 25 cannot detect a pause in the user's exercise. If the control unit 25 cannot detect the first pause and the second pause, it cannot accurately detect the user's direction of travel. If the control unit 25 cannot accurately detect the user's direction of travel, it cannot accurately estimate the yaw direction by the processing according to the second embodiment.

[0152] Here, as shown in FIG. 25, when a user looks down at their feet while walking in the motion section [2], the amount of change in the angle of the user's head increases. For example, for a user with a typical build, the amount of change in the pitch and roll angles of the user's head is 15 degrees or more. Furthermore, when a user looks down at their feet while walking in the motion section [2], the absolute value of the angular velocity of the user's head changes. For example, for a user with a typical build, the maximum absolute value of the angular velocity of the user's head is 50 dps or more.

[0153] Furthermore, as shown in FIG. 25 , when a walking user looks down at their feet during the motion section [2], the timing at which the amount of change in the user's head angle increases and the timing at which the absolute value of the user's head angular velocity reaches a maximum may be the same. For example, the upper part of FIG. 26 shows a graph of the user's head angle when the user looks down at their feet. In this graph, the vertical axis represents the angle [deg], and the horizontal axis represents time [s]. The lower part of FIG. 26 shows a graph of the user's head angular velocity when the user looks down at their feet. In this graph, the vertical axis represents the angular velocity [deg / s], and the horizontal axis represents time [s]. As can be seen from the upper graph of FIG. 26 , at time 36 s, the amount of change in the pitch angle and the roll angle increases. As can be seen from the lower graph of FIG. 26 , at time 36 s, the pitch angular velocity reaches a maximum value, and the roll angular velocity reaches a minimum value. In other words, at time 36 s, the absolute value of the user's head angular velocity reaches a maximum. Therefore, in FIG. 26, the timing at which the amount of change in the angle of the user's head increases and the timing at which the absolute value of the angular velocity of the user's head reaches a maximum are both 36 seconds.

[0154] Also, as shown in Figure 27, the angular velocity vector (V X ,V Y ) can be the mounting direction of the sensor 10A. X ,V Y) is the maximum value of the angular velocity vector consisting of components on the XY plane. The mounting direction of the sensor 10A is, for example, the direction of the local coordinate system relative to the global coordinate system. Such an angular velocity vector (V X ,V Y The rotation direction ψ of the X-axis direction of the yaw axis θ can be the same as the yaw direction. The angle of the rotation direction ψ is given by equation (6).

number

[0155] Therefore, if the first pause and the second pause are not detected, the control unit 25 determines whether the user looks at their feet based on data from the sensor 10A attached to the user's head. As described above with reference to FIGS. 25 and 26, when a walking user looks at their feet, the timing at which the amount of change in the user's head angle increases may coincide with the timing at which the absolute value of the user's head angular velocity becomes maximum. Therefore, the control unit 25 determines whether the first timing at which the pitch direction angle of the global coordinate system of the sensor 10A becomes equal to or greater than a first threshold coincides with the second timing at which the absolute value of the angular velocity of the sensor 10A in the global coordinate system becomes equal to or greater than a second threshold. The first threshold may be set based on the assumed angle of the user's head when the walking user looks at their feet. The first threshold may be, for example, 15 degrees. The second threshold may be set based on the assumed absolute value of the user's head angular velocity when the walking user looks at their feet. The second threshold may be, for example, 50 dps. If the control unit 25 determines that the first timing and the second timing are the same, it determines that the user looks at their feet.

[0156] When it is determined that the user looks at his / her feet, the control unit 25 calculates an angular velocity vector (V X ,V Y ) with respect to the X-axis direction. X ,V Y) and calculates the rotation direction ψ of the user. The control unit 25 provisionally adopts the rotation direction ψ as the yaw direction and detects the user's traveling direction in the motion section [2] by the process described above in the first embodiment. Here, for example, even when the user looks straight ahead from their feet, the control unit 25 may erroneously determine that the user is looking at their feet. Therefore, if the angle between the user's traveling direction and the Y-axis direction is below an angle threshold, the control unit 25 adopts the rotation direction ψ as the yaw direction. The angle threshold may be set assuming an angle that can distinguish between when the user looks at their feet and other cases. The angle threshold is, for example, 90 degrees. If the angle between the user's traveling direction and the Y-axis direction is equal to or greater than the angle threshold, the control unit 25 corrects the rotation direction ψ and adopts the corrected rotation direction ψ as the yaw direction. As an example of the correction, the control unit 25 may invert the rotation direction ψ and use the inverted rotation direction ψ as the corrected rotation direction ψ.

[0157] [Operation of information processing device] 28 is a flowchart showing an example of the operation of the information processing device 20 according to the third embodiment of the present disclosure. This operation of the information processing device 20 corresponds to the information processing method according to the present embodiment. For example, the control unit 25 may start the process of step S21 when repeatedly executing the processes of steps S1 to S4 shown in FIG. 16 for a predetermined time, that is, when the first pause and the second pause cannot be detected for a predetermined time. The predetermined time may be set in consideration of the specifications of the information processing system 1.

[0158] The control unit 25 executes the processes of steps S21, S22, and S23 in the same manner as or similar to the processes of steps S11, S12, and S13.

[0159] The control unit 25 applies a low-pass filter to the angular velocity components in the XY plane of the global sensor data (step S24). This low-pass filter is a filter that passes frequencies equal to or lower than a set cutoff frequency. The cutoff frequency may be set according to the specifications of the information processing system 1. The cutoff frequency is, for example, 6 Hz.

[0160] The control unit 25 determines whether the user has looked at their feet based on the global sensor data of the sensor 10A (step S25). The global sensor data of the sensor 10A is sensor data in the global coordinate system of the sensor 10A. As described above, the control unit 25 determines whether the first timing at which the angle in the pitch direction of the global coordinate system of the sensor 10A becomes equal to or greater than the first threshold is the same as the second timing at which the absolute value of the angular velocity of the global coordinate system of the sensor 10A becomes equal to or greater than the second threshold.

[0161] If the control unit 25 determines that the user has looked at their feet (step S25: YES), the process proceeds to step S26. If the control unit 25 does not determine that the user has looked at their feet (step S25: NO), the process shown in FIG. 28 ends.

[0162] In the process of step S26, the control unit 25 calculates the vector (V X ,V Y ) to obtain the angular velocity vector (V X ,V Y ) is the maximum value of the angular velocity vector consisting of components in the XY plane, as described above.

[0163] In the process of step S27, the control unit 25 calculates the angular velocity vector (V X ,V Y The control unit 25 calculates the rotation direction ψ of the yaw direction (step S28).

[0164] In the process of step S29, the control unit 25 detects the user's traveling direction in the motion section [2] based on the provisionally adopted yaw direction. The control unit 25 detects the user's traveling direction by executing the processes of steps S7 and S8 as shown in FIG.

[0165] In the process of step S30, the control unit 25 determines whether the angle formed between the user's traveling direction and the Y-axis direction detected in the process of step S29 is below the angle threshold. If the control unit 25 determines that the angle formed between the user's traveling direction and the Y-axis direction is below the angle threshold (step S30: YES), the control unit 25 proceeds to the process of step S31. If the control unit 25 determines that the angle formed between the user's traveling direction and the Y-axis direction is equal to or greater than the angle threshold (step S30: NO), the control unit 25 proceeds to the process of step S32.

[0166] In the process of step S31, the control unit 25 adopts the rotation direction ψ as the yaw direction.

[0167] In the process of step S32, the control unit 25 corrects the rotation direction ψ and adopts the corrected rotation direction ψ as the yaw direction.

[0168] As described above, in the third embodiment, when the first pause and the second pause are not detected, the control unit 25 determines whether or not the user looks at their feet based on the sensor data of the sensor 10A. When the control unit 25 determines that the user looks at their feet, the control unit 25 calculates the angular velocity vector (V X ,V Y ) and calculates the rotation direction ψ of the angular velocity vector with respect to the X-axis direction. The control unit 25 estimates the yaw direction based on the calculated rotation direction ψ. With this configuration, it is possible to estimate the yaw direction even when the first pause and the second pause are not detected.

[0169] <Application example> The process according to the third embodiment can be applied to a user who cannot walk, in the same way as or similar to the process according to the first embodiment. For example, the process according to the third embodiment can be applied to a user who is in a wheelchair, as described above with reference to FIG.

[0170] While the present disclosure has been described based on various drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present disclosure. For example, the functions included in each functional unit can be rearranged so as not to cause logical inconsistencies. Multiple functional units can be combined into one or separated. The above-described embodiments of the present disclosure are not limited to faithful implementation of each of the described embodiments, but can be implemented by combining features or omitting some features as appropriate. In other words, those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, these modifications and alterations are within the scope of the present disclosure. For example, in each embodiment, each functional unit, means, or step can be added to other embodiments so as not to cause logical inconsistencies, or can be replaced with each functional unit, means, or step of other embodiments. Furthermore, in each embodiment, multiple functional units, means, or steps can be combined into one or separated. Furthermore, each of the above-described embodiments of the present disclosure is not limited to being implemented faithfully according to each of the described embodiments, but can also be implemented by combining each feature or omitting some of them as appropriate.

[0171] For example, in the first embodiment, the control unit 25 of the information processing device 20 is described as detecting the first pause and the second pause. However, an information processing device other than the information processing device 20 may detect the first pause and the second pause. As another example, the sensor 10 may detect the first pause and the second pause as the information processing device. In this case, the control unit 14 of the sensor 10 detects the first pause and the second pause based on the sensor data detected by the sensor unit 12 in the same or similar manner as the process described above in the first embodiment. Furthermore, the control unit 14 transmits the timings of the first pause and the second pause together with the sensor data detected by the sensor unit 12 to the information processing device 20 via the communication unit 11. In the information processing device 20, the control unit 25 receives the timings of the first pause and the second pause together with the sensor data from the sensor 10 via the communication unit 21. The control unit 25 corrects the sensor data between the first pause and the second pause based on the received sensor data and the timings of the first pause and the second pause.

[0172] For example, in the first embodiment, the control unit 25 of the information processing device 20 is described as detecting the first pause and the second pause. Furthermore, the control unit 25 is described as correcting the sensor data detected by the sensor 10 between the first pause and the second pause based on the sensor data detected by the sensor 10 during the first pause and the sensor data detected by the sensor 10 during the second pause. The control unit 25 is also described as generating correction data. Additionally, the control unit 25 is described as detecting data on the trajectory of the position of the body part to which the sensor 10 is attached based on the corrected sensor data. However, these processes may be performed by an information processing device other than the information processing device 20. As another example, the sensor 10 may perform these processes as an information processing device. That is, the control unit 14 of the sensor 10 may detect the first pause and the second pause, and correct the sensor data detected by the sensor 10 between the first pause and the second pause based on the sensor data detected by the sensor 10 during the first pause and the sensor data detected by the sensor 10 during the second pause. Furthermore, the control unit 14 may generate the correction data. Furthermore, the control unit 14 may detect data on the trajectory of the position of the body part to which the sensor 10 is attached, based on the corrected sensor data.

[0173] For example, in the first embodiment, it has been described that control unit 14 of sensor 10 transmits sensor data detected by sensor unit 12 to information processing device 20 via communication unit 11. However, control unit 14 may store the sensor data detected by sensor unit 12 in memory unit 13. In this case, control unit 14 may transmit the sensor data stored in memory unit 13 to information processing device 20 via communication unit 11 at any timing. Alternatively, control unit 14 may detect first pauses and second pauses, correct sensor data, generate correction data, or detect data on the trajectory of the position of the body part to which sensor 10 is attached, based on the sensor data stored in memory unit 13, as described above.

[0174] For example, in the first embodiment, as described above with reference to FIG. 3, [1] the first pause is a pause in the user's exercise, and [3] the second pause is a second pause by the user. However, as long as the first pause is a pause of the user's body part to which the sensor 10 is attached and the second pause is a second pause of the user's body part to which the sensor 10 is attached, the user may perform any movement. That is, the control unit 25 may detect the first and second pauses of the body part to which the sensor 10 is attached from the user's arbitrary movement, and generate correction data for correcting the sensor data detected by the sensor 10 in the movement section between the first and second pauses. As another example, the control unit 25 may detect the first and second pauses of the body part to which the sensor 10 is attached from the user's continuous movement, and generate correction data for correcting the sensor data detected by the sensor 10 in the movement section between the first and second pauses. The user may perform continuous walking, rhythmic acting, dancing, etc. as continuous movement. In this case, the control unit 25 may detect a momentary pause in the movement of the body part to which the sensor 10 is attached during continuous walking, rhythmic acting, dancing, or the like, as the first pause or the second pause. With this configuration, the information processing system 1 can be used for coaching, self-study, or the like. For example, FIG. 29 shows examples of the first pause and the second pause during continuous walking. In FIG. 29, at least one of the sensors 10D-2, 10E-2, and 10F-2 is attached to the user. During continuous walking, a gait cycle including a stance phase and a swing phase is repeated one or more times. In FIG. 29, the moment when the user's right foot lands on the ground is detected as a pause. The detected pause is the first pause or the second pause. As an example, the control unit 25 may detect a momentary pause of the ankle to which the sensor 10E-2 is attached, at the moment when the user's right foot lands on the ground, as the first pause or the second pause. When the processing of the present disclosure is applied to continuous movement, a pause detected again after a second pause is considered to be a first pause. If a momentary pause in the movement of the body part to which the sensor 10 is attached is detected as a first or second pause, the acceleration data may not be zero. In this case, the control unit 25 may generate correction data for correcting the velocity data.

[0175] For example, in the first embodiment, the control unit 25 is described as generating correction data by formulating Equation (4) or Equation (5). However, the control unit 25 may also correct the sensor data according to the attitude of the sensor 10 by applying a high-pass filter to the acceleration or velocity of the sensor data. This high-pass filter is a filter that passes data with frequencies equal to or higher than a set cutoff frequency. The cutoff frequency may be set based on an offset value of a gyro sensor or the like included in the sensor 10. The cutoff frequency is, for example, 1 Hz. Applying the high-pass filter to the sensor data can reduce drift of the sensor data in the yaw direction of the attitude of the sensor 10. Furthermore, the high-pass filter can be considered as correction data. As an example, the control unit 25 may use the high-pass filter as correction data and correct acceleration data using the high-pass filter.

[0176] For example, in the first to third embodiments, the sensor 10 has been described as being attached to a user. However, the sensor 10 is not limited to being attached to a human. As another example, the sensor 10 may be attached to a body part of any moving object. The body part of the moving object may be, for example, the arm of a robot or a body part of an animal such as livestock.

[0177] For example, in the second embodiment, a learning model may be used. In this case, the learning model may be one that is machine-learned so that when the components of the XY plane of the global sensor data are input, the yaw direction is estimated from the combination of the data and the characteristics of the waveform.

[0178] In one embodiment, (1) an information processing device includes: A control unit is provided, The control unit Detecting a first pause in which the body part of the user pauses exercise and a second pause in which the body part pauses exercise again after the first pause based on sensor data detected by a sensor attached to the body part of the user; The sensor data detected by the sensor between the first pause and the second pause is corrected based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

[0179] (2) In the information processing device described in (1), the sensor data includes data on acceleration or angular velocity of the body part in x-axis, y-axis, and z-axis directions of a local coordinate system; The control unit may determine that the body part has stopped moving when the absolute value of the acceleration or the angular velocity is equal to or less than a threshold value.

[0180] (3) In the information processing device according to (1) or (2), the sensor data includes data on acceleration or angular velocity of the body part in x-axis, y-axis, and z-axis directions of a local coordinate system; The control unit may determine that the body part has stopped moving when a mean absolute error of a resultant acceleration obtained by combining the absolute values ​​of accelerations in the x-axis, y-axis, and z-axis directions of the local coordinate system or a resultant angular velocity obtained by combining angular velocities in the x-axis, y-axis, and z-axis directions of the local coordinate system is equal to or less than an error threshold.

[0181] (4) The information processing device according to any one of (1) to (3) above, further comprising an output unit; When detecting the first pause, the control unit may cause the output unit to output information indicating that the first pause has been detected.

[0182] (5) In the information processing device according to any one of (1) to (4), the sensor data detected by the sensor during the first pause is the sensor data at an end point of the first pause; The sensor data detected by the sensor during the second pause may be the sensor data at the start point of the second pause.

[0183] (6) In the information processing device described in (5) above, The control unit may correct the sensor data detected by the sensor between the first pause and the second pause based on linear correction using the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

[0184] (7) In the information processing device described in (6), The control unit may acquire data on a trajectory of the position of the body part in an exercise section in which the user exercises between the first pause and the second pause.

[0185] (8) In the information processing device described in (7), the sensor data includes data on acceleration or angular velocity of the body part in x-axis, y-axis, and z-axis directions of a local coordinate system; The control unit may obtain data on a trajectory of a position of the body part based on the corrected acceleration or the corrected angular velocity.

[0186] (9) In the information processing device according to (7) or (8), The control unit may estimate a yaw direction of a global coordinate system based on data of a trajectory of the position of the body part.

[0187] (10) In the information processing device described in (9), the control unit defines data of the trajectory of the positions of the body parts as a moving direction of the user, and estimates the yaw direction based on the moving direction of the user and a y-axis direction of a local coordinate system; The local coordinate system may be a coordinate system based on the position of the sensor.

[0188] (11) In the information processing device according to (9) or (10), The sensor is attached to the user's head, The control unit If the first pause and the second pause are not detected, it is determined whether the user looks at their feet based on sensor data from the sensor attached to the user's head; When it is determined that the user looks at his / her feet, a maximum value of an angular velocity vector consisting of components of an XY plane of a global coordinate system is obtained; calculating a rotation direction of the angular velocity vector relative to an X-axis direction of the global coordinate system; The yaw direction may be estimated based on the calculated rotation direction.

[0189] (12) In the information processing device described in (11), The control unit may determine that the user has looked at their feet when it determines that a first timing at which the angle in the pitch direction of the global coordinate system of the sensor becomes equal to or greater than a first threshold is the same as a second timing at which the absolute value of the angular velocity of the global coordinate system of the sensor becomes equal to or greater than a second threshold.

[0190] In one embodiment, (13) an information processing method includes: Detecting a first pause in which the body part of the user pauses exercise and a second pause in which the body part pauses exercise again after the first pause based on sensor data detected by a sensor attached to the body part of the user; and correcting the sensor data detected by the sensor between the first pause and the second pause based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

[0191] In this disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configuration. In this disclosure, configurations distinguished by descriptions such as "first" and "second" can have their numbers exchanged. For example, the first pause can exchange the identifiers "first" and "second" with the second pause. The exchange of identifiers is performed simultaneously. The configurations remain distinguished even after the exchange of identifiers. Identifiers may be deleted. Configurations from which identifiers have been deleted are distinguished by symbols. The descriptions of identifiers such as "first" and "second" in this disclosure should not be used solely to interpret the order of the configurations or to justify the existence of identifiers with smaller numbers. [Explanation of symbols]

[0192] 1: information processing system, 10, 10A, 10A-1, 10A-2, 10B, 10C, 10D, 10D-1, 10D-2, 10E, 10E-1, 10E-2, 10F, 10F-1, 10F-2: sensor, 11: communication unit, 12: sensor unit, 13: memory unit, 14: control unit, 20: information processing device, 21: communication unit, 22: input unit, 23: output unit, 24: memory unit, 25: control unit

Claims

1. A control unit is provided, The control unit Detecting a first pause in which the body part of the user pauses exercise and a second pause in which the body part pauses exercise again after the first pause based on sensor data detected by a sensor attached to the body part of the user; an information processing device that corrects the sensor data detected by the sensor between the first pause and the second pause based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

2. the sensor data includes data on acceleration or angular velocity of the body part in x-axis, y-axis, and z-axis directions of a local coordinate system; The information processing device according to claim 1 , wherein the control unit determines that the body part has stopped exercising when an absolute value of the acceleration or the angular velocity is equal to or less than a threshold value.

3. the sensor data includes data on acceleration or angular velocity of the body part in x-axis, y-axis, and z-axis directions of a local coordinate system; 2. The information processing device according to claim 1, wherein the control unit determines that the body part has stopped moving when a mean absolute error of a resultant acceleration obtained by combining absolute values ​​of accelerations in the x-axis, y-axis, and z-axis directions of a local coordinate system or a resultant angular velocity obtained by combining angular velocities in the x-axis, y-axis, and z-axis directions of the local coordinate system is equal to or less than an error threshold.

4. further comprising an output unit; The information processing device according to claim 1 , wherein, when the control unit detects the first pause, the control unit causes the output unit to output a signal indicating that the first pause has been detected.

5. the sensor data detected by the sensor during the first pause is the sensor data at an end point of the first pause; The information processing device according to claim 1 , wherein the sensor data detected by the sensor during the second pause is the sensor data at a start point of the second pause.

6. 6. The information processing device according to claim 5, wherein the control unit corrects the sensor data detected by the sensor between the first pause and the second pause based on linear correction using the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

7. The information processing device according to claim 6 , wherein the control unit acquires data on a trajectory of the position of the body part in an exercise section in which the user exercises between the first pause and the second pause.

8. the sensor data includes data on acceleration or angular velocity of the body part in x-axis, y-axis, and z-axis directions of a local coordinate system; The information processing device according to claim 7 , wherein the control unit acquires data on a trajectory of the position of the body part based on the corrected acceleration or the corrected angular velocity.

9. The information processing device according to claim 7 , wherein the control unit estimates a yaw direction of a global coordinate system based on data of a trajectory of the position of the body part.

10. the control unit defines data of the trajectory of the positions of the body parts as a moving direction of the user, and estimates the yaw direction based on the moving direction of the user and a y-axis direction of a local coordinate system; The information processing apparatus according to claim 9 , wherein the local coordinate system is a coordinate system based on the position of the sensor.

11. The sensor is attached to the user's head, The control unit If the first pause and the second pause are not detected, it is determined whether the user looks at their feet based on sensor data from the sensor attached to the user's head; When it is determined that the user looks at his / her feet, a maximum value of an angular velocity vector consisting of components on an XY plane of a global coordinate system is obtained; calculating a rotation direction of the angular velocity vector relative to an X-axis direction of the global coordinate system; The information processing device according to claim 9 , wherein the yaw direction is estimated based on the calculated rotation direction.

12. 12. The information processing device according to claim 11, wherein the control unit determines that the user has looked at their feet when it determines that a first timing at which an angle in the pitch direction of the global coordinate system of the sensor becomes equal to or greater than a first threshold is the same as a second timing at which an absolute value of the angular velocity of the global coordinate system of the sensor becomes equal to or greater than a second threshold.

13. Detecting a first pause in which the body part of the user pauses exercise and a second pause in which the body part pauses exercise again after the first pause based on sensor data detected by a sensor attached to the body part of the user; correcting the sensor data detected by the sensor between the first pause and the second pause based on the sensor data detected by the sensor during the first pause and the sensor data detected by the sensor during the second pause.

Images