Information processing device, information processing system, storage medium and information processing method

Abstract

The present invention provides an information processing device, an information processing system, a non-transitory storage medium, and an information processing method, which can more accurately determine the individual motion state of at least one of an arm and a leg. The information processing device includes a processing unit, which obtains first information related to a first motion state of at least one of an arm and a leg of a user superimposed with the motion state of a body of a user who is exercising, obtains second information related to a second motion state of the body of the user who is exercising, and generates third information related to a third motion state of at least one of an arm and a leg of the user based on the first information and the second information.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing system, a storage medium and an information processing method. Background Art

[0002] In the past, the following technology is known: a detection unit such as an acceleration sensor and an angular velocity sensor is provided in an electronic device that is equipped on the body of a user for use, and the user's motion state is analyzed based on the detection result of the detection unit when the user is exercising. For example, in the Japanese Patent Publication No. 2013-143996, a technology is disclosed for analyzing the posture of arm swinging based on the detection result of the detection unit when the user is running in a state where the electronic device is equipped on the arm. In addition, by equipping the electronic device on the leg, the posture of leg swinging can also be analyzed.

[0003] The analysis of the arm and leg swinging posture is usually performed to understand how the arms and legs are moving based on the trunk. However, since the movement of the arms and legs is linked to the movement of the trunk, the movement state of the arms or legs determined by the electronic equipment equipped on the arms or legs overlaps the movement state of the trunk. Therefore, in the above-mentioned prior art, there is a problem that the independent movement state of the arms and legs based on the trunk cannot be correctly determined. Summary of the invention

[0004] An object of the present invention is to provide an information processing device, an information processing system, a program, and an information processing method that can more accurately determine the individual motion state of at least one of an arm and a leg.

[0005] In order to solve the above-mentioned problems, the information processing device involved in the present invention has a processing unit, which performs the following processing: obtaining first information related to a first movement state of at least one of the arm and the leg of the user that overlaps the movement state of the body of the user who is performing the exercise, obtaining second information related to a second movement state of the body of the user who is performing the exercise, and generating third information related to a third movement state of at least one of the arm and the leg of the user based on the first information and the second information.

[0006] In addition, in order to solve the above-mentioned problems, the information processing system involved in the present invention comprises: a first device, which is equipped for use on at least one of the arm and leg of the user, and has a first detection unit for detecting the movement state of the self-device; a second device, which is equipped for use on the body of the user, and has a second detection unit for detecting the movement state of the self-device; and a processing unit, which performs the following processing: obtaining first information related to the first movement state of at least one of the arm and leg of the user, which is generated based on the detection result of the first detection unit and overlaps the movement state of the body of the user who is exercising, obtaining second information related to the second movement state of the body of the user who is exercising, which is generated based on the detection result of the second detection unit, and generating third information related to the third movement state of at least one of the arm and leg of the user based on the first information and the second information.

[0007] In addition, in order to solve the above-mentioned problems, the non-temporary storage medium involved in the present invention stores a program, and the program enables a computer to implement the following functions: obtaining first information related to a first movement state of at least one of the arm and the leg of the user that overlaps the movement state of the body of the user who is exercising; obtaining second information related to a second movement state of the body of the user who is exercising; and generating third information related to a third movement state of at least one of the arm and the leg of the user based on the first information and the second information.

[0008] In addition, in order to solve the above-mentioned problems, the information processing method involved in the present invention is an information processing method performed by a computer, comprising the following steps: obtaining first information related to a first movement state of at least one of the arms and legs of the user that overlaps the movement state of the body of the user who is performing the exercise; obtaining second information related to a second movement state of the body of the user who is performing the exercise; and generating third information related to a third movement state of at least one of the arms and legs of the user based on the first information and the second information.

[0009] Effects of the Invention

[0010] According to the present invention, the individual movement state of at least one of the arm and the leg can be determined more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a diagram showing an information processing system.

[0012] Figure 2 This is a block diagram showing the functional structure of the first device.

[0013] Figure 3 It is a block diagram showing the functional structure of the second device.

[0014] Figure 4 The diagram schematically shows an example of the movement state of the trunk and arms during running.

[0015] Figure 5 2 is a diagram showing an example of an index indicating the motion state of the arm.

[0016] Figure 6 FIG. 1 is a diagram showing an example of a plurality of sections in a motion involving movement.

[0017] Figure 7 Detailed description of the process of controlling the body movement detection process.

[0018] Figure 8 Detailed description of the invention is given in the following figure: FIG.

[0019] Fig. 9 Detailed description of the invention is given in the following figure: FIG. DETAILED DESCRIPTION

[0020] Embodiments of the present invention will be described below with reference to the drawings.

[0021] <Overview of Information Processing System>

[0022] Figure 1 1 is a diagram showing an information processing system 1 according to the present embodiment. The information processing system 1 includes a first device 10 (information processing device) and a second device 20 .

[0023] The first device 10 is provided on the arm A for use in order to obtain information related to the motion state of the arm A of the user U. Figure 1 In the embodiment, the first device 10 is equipped on the wrist of the user U. The first device 10 in this embodiment is a watch. However, it is not limited to this. The first device 10 can be any wearable device other than a watch (for example, a health care device such as an activity meter), or an electronic device such as a smart phone that can be mounted on the arm A for fixing. The installation position of the first device 10 is not limited to the wrist, and can be any part where the exercise status is desired to be understood, for example, it can be installed on the elbow or upper arm.

[0024] The second device 20 is used to obtain information related to the movement state of the body B of the user U and is equipped on the body B. The body B is the body except the arm (the part in front of the shoulder joint) and the leg (the part in front of the hip joint). Figure 1In the embodiment, the second device 20 is provided on the waist of the back side of the user U. The second device 20 is preferably provided near the trunk of the body B, that is, near the line passing through the spine. This is because the position change due to the twisting of the body is small near the trunk, and it is easy to present the overall position and movement of the body B. The second device 20 can be a dedicated wearable device for obtaining information related to the movement state of the body B, or an electronic device such as a smartphone that can be mounted on the body B and fixed.

[0025] The user U performs exercise while being equipped with the first device 10 and the second device 20. The type of exercise is not particularly limited, and may be an exercise accompanying the movement of the user U, such as walking or running. In the present embodiment, the case where the user U performs running is taken as an example for explanation. Hereinafter, the direction of the user U who is running is recorded as the "front direction", the direction opposite to the direction of travel is recorded as the "rear direction", the direction of gravity (vertical downward direction) is recorded as the "downward direction", the direction opposite to the direction of gravity (vertical upward direction) is recorded as the "upward direction", the direction toward the right hand of the user U and perpendicular to both the front-back direction and the up-down direction is recorded as the "right direction", and the direction toward the left hand of the user U and perpendicular to both the front-back direction and the up-down direction is recorded as the "left direction". In addition, a coordinate system with the front-back direction, the up-down direction, and the left-right direction as three axes is recorded as the "world coordinate system".

[0026] The first device 10 and the second device 20 are connected to each other by short-range wireless communication, and data can be sent and received. As an example of short-range wireless communication, Bluetooth (registered trademark) can be cited, but it is not limited to this. Hereinafter, the connection between the first device 10 and the second device 20 by short-range wireless communication is also referred to as "pairing".

[0027] The first device 10 includes a first sensor unit 13 (first detection unit) for detecting the motion state of the device (see Figure 2 Since the movement of the arm A equipped with the first device 10 is linked to the movement of the trunk B, the movement state of the arm A (first movement state) superimposed on the movement state of the trunk B is reflected in the detection result of the first sensor unit 13. The first device 10 generates the composite arm movement data 122 based on the detection result of the first sensor unit 13 (refer to Figure 2 ), as the first information related to the first motion state of the arm A superimposed on the motion state of the trunk B. Hereinafter, the first motion state of the arm A superimposed on the motion state of the trunk B is also referred to as a "compound motion state".

[0028] The second device 20 includes a second sensor unit 23 (second detection unit) for detecting the motion state of the device (see Figure 3The detection result of the second sensor unit 23 reflects the movement state (second movement state) of the body B of the user U. The second device 20 generates the body movement data 123 (refer to Figure 2 ) as the second information related to the second movement state of the body B of the user U, and is sent to the first device 10.

[0029] The first device 10 generates arm motion data 124 (third information) related to the individual motion state (third motion state) of arm A based on the user's body B based on the difference between the composite arm motion data 122 and the body motion data 123 received from the second device 20. Hereinafter, the individual third motion state of arm A based on body B will also be referred to as "individual motion state". The arm motion data 124 can be used to understand the motion state of arm A itself excluding the motion state of body B. The first device 10 displays information related to the motion state of arm A, such as information related to the determination result of the arm swing state, based on the arm motion data 124. The method for determining the individual motion state of arm A performed by the first device 10 will be described in detail later.

[0030] In addition to the movement state of the arm A, the first device 10 can also display various information related to the movement state of the user U. For example, the first device 10 can display information such as the running distance, running pace, number of steps (number of steps per minute), stride length, size of up and down movement, contact time of each step, and inclination of the trunk.

[0031] <Structure of the first device>

[0032] Figure 2 It is a block diagram showing the functional structure of the first device 10.

[0033] The first device 10 includes a CPU 11 (Central Processing Unit), a memory 12 (storage unit), a first sensor unit 13, a wireless communication unit 14, a satellite radio wave reception processing unit 15, a display unit 16, an operation receiving unit 17, a timing unit 18, and the like.

[0034] The CPU 11 reads and executes the program 121 stored in the memory 12 to perform various calculations and thereby control the operation of each unit of the first device 10. In the present embodiment, the CPU 11 corresponds to a "processing unit." In addition, the processing unit may include two or more circuit elements such as CPUs that perform calculations.

[0035] The memory 12 is a non-temporary recording medium that can be read by the CPU 11 as a computer, and provides a memory space for the CPU 11 to use for operation and store various data. The memory 12 includes, for example, a RAM (Random Access Memory) and a non-volatile memory. The RAM is used in the calculation processing of the CPU 11, and also stores temporary data. The non-volatile memory is, for example, a flash memory, and stores various data in addition to the program 121. The program 121 is stored in the memory 12 in the form of a computer-readable program code. As data stored in the memory 12, there are compound arm motion data 122 (first information), body motion data 123 (second information), arm motion data 124 (third information), and body motion history record data 125 (second information).

[0036] The composite arm motion data 122 is data generated based on the detection result of the first sensor unit 13, and is data representing the composite motion state of the arm A superimposed with the motion state of the body B of the user U. The body motion data 123 is data representing the motion state of the body B of the user U received from the second device 20. The arm motion data 124 is data generated based on the difference between the composite arm motion data 122 and the body motion data 123, and is data representing the individual motion state of the arm A based on the body B of the user. The body motion history data 125 is the body motion data 123 generated during the past motion of the user U. The arm motion data 124 may also be generated based on the difference between the composite arm motion data 122 and the body motion history data 125.

[0037] The first sensor unit 13 includes a three-axis acceleration sensor 131 , a three-axis gyro sensor 132 , and a three-axis geomagnetic sensor 133 .

[0038] The three-axis acceleration sensor 131 detects the acceleration in each axis direction of the first device 10 at a given sampling frequency in response to the movement of the user U, and outputs acceleration data as the detection result. The acceleration data output from the three-axis acceleration sensor 131 includes signal components about three axes (x-axis, y-axis, and z-axis) that are orthogonal to each other. The directions of the three axes are not particularly limited, and can be, for example, Figure 1 As shown in FIG. 1 , two perpendicular axes in a plane parallel to the display surface of the first device 10 are defined as the x-axis and the y-axis, and a direction perpendicular to the x-axis and the y-axis is defined as the z-axis. Hereinafter, a coordinate system having three axes, the x-axis, the y-axis, and the z-axis, and having a fixed positional relationship with the first device 10, is referred to as a “sensor coordinate system”.

[0039] The three-axis gyro sensor 132 detects the angular velocity around each axis applied to the first device 10 at a given sampling frequency in response to the movement of the user U, and outputs angular velocity data as a detection result. The angular velocity data output from the three-axis gyro sensor 132 includes signal components about the x-axis, y-axis, and z-axis.

[0040] The three-axis geomagnetic sensor 133 detects the direction of geomagnetism passing through the first device 10 at a given sampling frequency and outputs geomagnetic data as a detection result. The geomagnetic data output from the three-axis geomagnetic sensor 133 includes signal components for the x-axis, y-axis, and z-axis.

[0041] The sampling frequency of the 3-axis acceleration sensor 131, the 3-axis gyro sensor 132, and the 3-axis geomagnetic sensor 133 is determined so as to have a resolution capable of determining the movement of the swing arm, for example, within a range of 50 Hz to 200 Hz. The detection timing of the 3-axis acceleration sensor 131, the 3-axis gyro sensor 132, and the 3-axis geomagnetic sensor 133 is hereinafter referred to as "first detection timing". A first timestamp representing the detection time is recorded at each of the plurality of first detection timings, and the timestamps are stored in correspondence with each measurement data of the first sensor unit 13.

[0042] In addition, the first sensor unit 13 includes: an amplifier (not shown) that amplifies analog signals output from the 3-axis acceleration sensor 131, the 3-axis gyro sensor 132, and the 3-axis geomagnetic sensor 133; and an AD converter (not shown) that converts the amplified analog signals into digital data and outputs them to the CPU 11.

[0043] The wireless communication unit 14 performs wireless communication, that is, transmits and receives data using radio waves, with the second device 20. In the present embodiment, the wireless communication unit 14 performs short-range wireless communication based on Bluetooth with the second device 20 as a pairing target.

[0044] The satellite radio wave receiving processing unit 15 is a module that receives the transmission radio waves from the positioning satellite of the global navigation satellite system (GNSS: Global Navigation Satellite System) to obtain GPS data, and calculates the current position and date and time based on the GPS data. The global positioning satellite system used is not particularly limited, and for example, it can be GPS (Global Positioning System), GLONASS, or a quasi-zenith satellite system using the quasi-zenith satellite "MITIBIKI" and the like. The satellite radio wave receiving processing unit 15 calculates the current position and date and time under the control of the CPU 11, and outputs the result to the CPU 11.

[0045] The display unit 16 displays various information related to the exercise state, time, etc. under the control of the CPU 11. As the display unit 16, for example, a liquid crystal display device that displays in a dot matrix system can be used, but the present invention is not limited to this.

[0046] The operation receiving unit 17 includes a plurality of operation buttons, receives an input operation (e.g., a pressing operation) from a user on the operation buttons, and outputs the received input signal to the CPU 11. The CPU 11 performs a process corresponding to the function of the operation button to which the input operation was performed. The operation receiving unit 17 may include a touch panel superimposed on the display screen of the display unit 16.

[0047] The timer unit 18 includes an oscillation circuit, a frequency division circuit, a timer circuit, etc. The frequency division circuit divides the frequency of the clock signal generated by the oscillation circuit, and the timer circuit counts the divided signal, so that the timer unit 18 counts and stores the current date and time.

[0048] <Structure of the Second Device>

[0049] Figure 3 It is a block diagram showing the functional structure of the second device 20.

[0050] The second device 20 includes a CPU 21 , a memory 22 (storage unit), a second sensor unit 23 , a wireless communication unit 24 , a satellite radio wave reception processing unit 25 , and the like.

[0051] The CPU 21 reads out and executes a program 221 stored in the memory 22 to perform various calculations and thereby control the operation of each unit of the second device 20 .

[0052] The memory 22 is a non-temporary recording medium that can be read by the CPU 21 of the computer, and provides a memory space for the CPU 21 to work, and stores various data. The memory 22 includes, for example, a RAM and a non-volatile memory. The RAM is used in the calculation processing of the CPU 21, and also stores temporary data. The non-volatile memory is, for example, a flash memory, and stores various data in addition to the program 221. The program 221 is stored in the memory 22 in the form of a computer-readable program code.

[0053] The second sensor unit 23 includes a 3-axis acceleration sensor 231, a 3-axis gyro sensor 232, a 3-axis geomagnetic sensor 233, an amplifier (not shown), and an AD converter (not shown). The structures and operations of the 3-axis acceleration sensor 231, the 3-axis gyro sensor 232, and the 3-axis geomagnetic sensor 233 are the same as those of the 3-axis acceleration sensor 131, the 3-axis gyro sensor 132, and the 3-axis geomagnetic sensor 133 of the first device 10, respectively, and thus description thereof is omitted. The sensor coordinate system of the 3-axis acceleration sensor 231, the 3-axis gyro sensor 232, and the 3-axis geomagnetic sensor 233 of the second device 20 is separate and independent from the sensor coordinate system of the 3-axis acceleration sensor 131, the 3-axis gyro sensor 132, and the 3-axis geomagnetic sensor 133 of the first device 10. The sampling frequencies of the three-axis acceleration sensor 231, the three-axis gyro sensor 232, and the three-axis geomagnetic sensor 233 of the second device 20 may be different from those of the three-axis acceleration sensor 131, the three-axis gyro sensor 132, and the three-axis geomagnetic sensor 133 of the first device 10, but by making the sampling frequencies the same, the time resolution involved in the analysis of the motion state of the arm A and the body B can be made consistent. Furthermore, it is preferable that the detection timing (hereinafter referred to as "second detection timing") of the three-axis acceleration sensor 231, the three-axis gyro sensor 232, and the three-axis geomagnetic sensor 233 of the second device 20 is synchronized with the first detection timing of the three-axis acceleration sensor 131, the three-axis gyro sensor 132, and the three-axis geomagnetic sensor 133 of the first device 10. A second time stamp representing the detection time is recorded in each of the plurality of second detection timings, and is stored in association with each measurement data of the second sensor unit 23.

[0054] The wireless communication unit 24 performs wireless communication, that is, transmits and receives data using radio waves, with the first device 10. In the present embodiment, the wireless communication unit 24 performs short-range wireless communication based on Bluetooth with the first device 10 as a pairing target.

[0055] The satellite wave receiving processing unit 25 is a module that receives the transmission wave from the positioning satellite of the global positioning satellite system to obtain GPS data and calculates the current position and date and time based on the GPS data. The satellite wave receiving processing unit 25 calculates the current position and date and time under the control of the CPU 21 and outputs the result to the CPU 21. In addition, since the satellite wave receiving processing unit 15 of the first device 10 and the satellite wave receiving processing unit 25 of the second device 20 have overlapping functions, either one of them can be omitted.

[0056] <Operation of Information Processing System>

[0057] Next, the operation of the information processing system 1 will be described, focusing on the operation that is actually related to the motion state of the arm.

[0058] As described above, the motion state of the arm A determined by the first device 10 is not a single motion state based on the trunk B (i.e., fixed to the coordinate axis of the trunk B), but a composite motion state superimposed with the motion state of the trunk B. The movement of the arm A is linked to the movement of the trunk B because the position, velocity, and acceleration of the arm A are affected by the position, velocity, and acceleration of the trunk B. For example, when the user U is running, the trunk B is in a motion state corresponding to the movement of the running, and the position in the up-and-down direction changes periodically, or shakes in the left-right direction.

[0059] Figure 4 1 is a diagram schematically showing an example of the movement state of the trunk B and the arm A during running.

[0060] Figure 4 The upper layer is a schematic diagram showing the body motion of the user U at each time point of the exercise cycle T during running. Figure 4 The middle layer is a graph showing the change in height in the up-down direction of one side of the motion state of the trunk B (here, the waist equipped with the second device 20) during running. Here, the change in the height of the trunk B is schematically represented by a sine curve. As can be seen from the schematic diagram of the upper layer and the graph of the middle layer, the knee bends after the right leg touches the ground, and thus, at time t1, the position of the waist drops to the lowest point. Thereafter, the right leg kicks out and leaves the ground, and at time t2, while floating in the air, the height of the waist rises to the highest point. Next, the knee bends after the left leg touches the ground, and at time t3, the position of the waist drops to the lowest point again. Next, the left leg kicks out and leaves the ground, and at time t4, while floating in the air, the height of the waist rises to the highest point again. Next, the right leg touches the ground, the knee bends, and at time t5, the position of the waist drops to the lowest point. Time t1 to time t5 corresponds to the motion cycle T during running. During running, the actions of this motion cycle T are repeated. As shown in Figure 4 As shown in the middle layer, the height of the waist in the vertical direction changes in two round trips between the lowest point and the highest point in synchronization with the movement of the legs in one movement cycle T.

[0061] In this way, the body B is in a motion state while running, and the motion state of the body B is superimposed on the complex motion state of the arm A. Figure 4 The solid line L1 in the lower layer of is a graph showing the change in height in the vertical direction as one side of the compound motion state of the arm A (here, the wrist equipped with the first device 10) during running. In addition, Figure 4The dotted line L2 in the lower layer is a graph showing the change in the height in the up-down direction of one side of the arm A in a single motion state with the trunk B as a reference. As shown by the dotted line L2, the height of the arm A in the single motion state increases from time t1 to time t2 corresponding to the action of swinging the arm A in the forward direction, and decreases from time t2 to time t4 corresponding to the action of pulling the arm A in the backward direction. When the arm A is paused near time t4, a small peak value that becomes slightly higher is formed, and then at time t5, it becomes higher corresponding to the action of returning to the forward direction again. Even if the change in the height of the arm A in the single motion state is as shown by the dotted line L2, the change in the height of the arm A in the compound motion state is as shown by the solid line L1, because the change in the height of the trunk B in motion is superimposed, and the change in the height of the arm A in the compound motion state depicts a waveform different from the dotted line L2. The motion state of arm A determined by the first device 10 equipped on arm A is the composite motion state shown by the solid line L1. The single motion state of arm A shown by the dotted line L2 cannot be determined and analyzed based only on the detection result of the first sensor unit 13 of the first device 10.

[0062] As a method of extracting the individual motion state from the composite motion state of arm A, it is conceivable to determine the average motion state of the trunk of many runners when running, and then subtract it from the composite motion state of arm A. However, since such an average motion state of the trunk is not necessarily similar to the motion state of the trunk B of the user U, it is difficult to determine the individual motion state of arm A with good accuracy.

[0063] To this end, in the information processing system 1 of the present embodiment, the second device 20 determines the motion state of the body B of the user U in real time, and determines the individual motion state of the arm A based on the difference between the composite motion state of the arm A determined by the first device 10 and the motion state of the body B. Specifically, in the second device 20, body motion data 123 related to the motion state of the body B of the user U is generated based on the detection result of the second sensor unit 23, and is sent to the first device 10. In addition, in the first device 10, composite arm motion data 122 related to the composite motion state of the arm A of the user U is generated based on the detection result of the first sensor unit 13. Then, in the first device 10, arm motion data 124 related to the individual motion state of the arm A is generated based on the difference between the composite arm motion data 122 and the body motion data 123. According to this method, since the motion state of the body B of the user U can be deducted from the composite motion state of the arm A, the individual motion state of the arm A can be determined with high accuracy.

[0064] For example, the body movement data 123 includes Figure 4 The data representing the change in waist height shown in the middle layer and the composite arm motion data 122 are included in Figure 4In the case of data representing the change in wrist height in a compound motion state as shown by the solid line L1 in the lower layer, by deducting the waist height at each time point of the body motion data 123 from the wrist height at each time point of the compound arm motion data 122, the value of Figure 4 The lower layer of the arm motion data 124, which represents the change in the height of the wrist in the single motion state, is shown by the dotted line L2. When the composite arm motion data 122 and the body motion data 123 include time series data, the above-mentioned first time stamp representing the time point is associated with the data of each time point of the composite arm motion data 122, and the above-mentioned second time stamp representing the time point is associated with the data of each time point of the body motion data 123. Then, the difference between the corresponding timing data in the composite arm motion data 122 and the body motion data 123 is calculated with reference to the first time stamp and the second time stamp, thereby generating the arm motion data 124.

[0065] exist Figure 4 Although the change in height is illustrated in FIG. 1 , the information contained in the composite arm motion data 122 and the arm motion data 124 is not particularly limited as long as it represents the motion state of the arm A. For example, Figure 5 The information related to the swing amplitude W1 of the arm A in the front-to-back direction or the trajectory of the swing arm in the front-to-back direction may also be the swing amplitude W2 in the up-down direction or the trajectory of the swing arm in the up-down direction (equivalent to Figure 4 In addition, the information may include information on the swing amplitude in the left and right directions or the trajectory of the swing arm in the left and right directions. In addition, the information is not limited to the information related to the position of the arm A (the first device 10), and may also include information related to the speed or acceleration of the arm A.

[0066] Similarly, the information contained in the body motion data 123 is not particularly limited as long as it represents the motion state of the body B. For example, it can also be the change trajectory of the position of the body B in the up and down direction (equivalent to Figure 4 The information may include information on the position of the body B (the middle layer of the body B) or the range of change, or information on the change trajectory or the range of change of the position in the left and right directions. In addition, the information is not limited to the information on the position of the body B (the second device 20), and may include information on the speed or acceleration of the body B.

[0067] In order to cope with the case where the real-time body motion data 123 cannot be received from the second device 20 due to various factors, the body motion data 123 received by the first device 10 may be stored in the memory 12 as the body motion history data 125. Examples of the case where the real-time body motion data 123 cannot be received include the following cases: the pairing of the first device 10 and the second device 20 is cancelled or the detection of the second sensor unit 23 is no longer possible due to a failure of the second device 20 due to some reason (e.g., a short circuit of the circuit), or the battery of the second device 20 is exhausted (the output voltage of the battery becomes less than a threshold value due to discharge), and the user does not equip (or cannot equip) the second device 20. By storing the body motion data 123 as the body motion history data 125 in the memory 12, even when the real-time body motion data 123 cannot be received from the second device 20, the arm motion data 124 can be generated by using the body motion history data 125 instead of the body motion data 123. That is, the arm motion data 124 can be generated based on the difference between the composite arm motion data 122 related to the motion state of the arm A of the user U in a first period and the body motion history record data 125 related to the motion state of the body B of the user U in a second period before the first period. Although the body motion history record data 125 is not data reflecting the real-time motion state of the body B, since it represents the motion state of the body B of the user U himself, the individual motion state of the arm A can be determined with high accuracy compared with the direction of deducting the average value of many runners.

[0068] Furthermore, in the second period (e.g., during the running that was performed a few days ago), the body motion data 123 related to the motion state of the body B may be obtained in each of the plurality of intervals that distinguish the movement path after the start of the movement and stored as the body motion history data 125. For example, in Figure 6 As shown in the figure, in the case of running for 15 km in the second period, the moving path is divided into sections S1 to S3 every 5 km, and the body movement history record data 125 is stored in each of the sections S1 to S3. The moving distance of the user U can be calculated based on the positioning result of the satellite radio wave receiving processing unit 15 of the first device 10 or the satellite radio wave receiving processing unit 25 of the second device 20. The length of the section is not limited to 5 km, and can be appropriately determined according to the purpose of the analysis, etc. In addition, instead of the moving distance from the start of the movement, the section can be distinguished according to the elapsed time from the start of the movement.

[0069] When the body motion history record data 125 is stored for each interval, it is determined to which interval among the multiple intervals the moving distance of the user U after starting to move in the first period (for example, the current running) belongs, and the arm motion data 124 is generated based on the difference between the composite arm motion data 122 and the body motion history record data 125 corresponding to the determined interval. Figure 6 In the example of , in the first period, the moving distance until time ta is 8 km, which belongs to interval S2. Therefore, at time ta, arm motion data 124 is generated based on the difference between the composite arm motion data 122 and the body motion history data 125 corresponding to interval S2. According to this method, even if the running posture (motion state of body B) changes due to fatigue or the like depending on the running distance, the motion state of body B reflecting the change can be deducted from the composite motion state of arm A. Therefore, the individual motion state of arm A can be determined more accurately.

[0070] Furthermore, when the movement distance of the user U in the first period is longer than the total of the multiple intervals storing the body movement history data 125, the arm movement data 124 may be generated based on the difference between the composite arm movement data 122 and the body movement history data 125 corresponding to the last interval among the multiple intervals. Figure 6 In the example of , in the first period, the moving distance until time tb is 18 km, which is longer than the entire interval (15 km) storing the body movement history data 125. Therefore, the arm movement data 124 can be generated using the body movement history data 125 of the last interval, that is, interval S3.

[0071] In addition, the walking information related to the walking state of the user U running can be recorded instead of the interval or together with the interval, and the body movement history record data 125 can be stored for each part of the walking information that has different contents. As the walking state represented by the walking information, for example, the slope of the walking path (uphill road, downhill road), the curvature of the walking path (straight road, curved road), the surface state of the walking path (asphalt, soil, degree of unevenness, whether it is wet, whether it is frozen), wind direction, and wind speed can be cited. According to these walking states (external factors), the movement state of the trunk B and the relationship between the activities of the arm A and the trunk B will also change. For this reason, by determining the walking state in the first period, using the body movement history record data 125 stored in correspondence with the determined state to generate the arm movement data 124, the independent movement state of the arm A can be determined more accurately.

[0072] Next, the body motion detection process executed in the second device 20 and the arm motion detection process executed in the first device 10 in order to identify the individual motion state of the arm A as described above will be described.

[0073] Figure 7 2 is a flowchart showing a control procedure of the CPU 21 in the body movement detection process. The body movement detection process is started when a predetermined process start signal is received from the paired first device 10, for example.

[0074] When the body movement detection process is started, the CPU 21 of the second device 20 obtains the measurement data (detection result) of the second sensor unit 23 (step S101). Here, the CPU 21 uses the three-axis acceleration sensor 231, the three-axis gyro sensor 232, and the three-axis geomagnetic sensor 233 to obtain the measurement data detected at a given sampling frequency and at the second detection timing synchronized with the first detection timing of the first sensor unit 13. In addition, the CPU 21 stores the second timestamp of the second detection timing representing the measurement data in the memory 22 in correspondence with the obtained measurement data.

[0075] CPU 21 determines whether measurement data of the amount of motion cycle T has been accumulated (step S102). The determination method in step S102 is not particularly limited, and for example, a method for determination based on the peak value contained in the detection result of the 3-axis acceleration sensor 231 can be used. Since a peak corresponding to the contact of the leg is generated in the acceleration detected by the 3-axis acceleration sensor 231, when two peaks corresponding to the contact of 2 steps are detected, it can be determined that measurement data of the amount of motion cycle T has been accumulated. That is, the measurement data from the peak corresponding to the contact to the next peak below the peak can be determined as measurement data of the amount of motion cycle T. When it is determined that measurement data of the amount of motion cycle T has not been accumulated (step S102 "No"), CPU 21 returns the processing to step S101.

[0076] When it is determined that the measurement data of the amount of motion cycle T has been accumulated (step S102 "Yes"), CPU21 determines the gravity direction and the travel direction based on the measurement data of the second sensor unit 23, and transforms the measurement data into data of the up-down direction, left-right direction, and front-back direction (step S103). Here, CPU21 determines the gravity direction (downward direction) based on the gravity direction detected by the 3-axis acceleration sensor 231 and / or the orientation of the geomagnetic field detected by the 3-axis geomagnetic sensor 233. In addition, CPU21 determines the travel direction (forward direction) of the user U based on the determined gravity direction and the magnitude of the acceleration of each axis of the 3-axis acceleration sensor 231. In addition, CPU21 transforms (separates) the measurement data of the sensor coordinate system of the second sensor unit 23 into measurement data of the world coordinate system with the up-down direction, left-right direction, and front-back direction as coordinate axes.

[0077] Based on the measurement data of the motion cycle T of the second sensor unit 23, the CPU 21 generates the body motion data 123 related to each motion state of the body B in the up-down direction, left-right direction, and front-back direction (step S104). For example, the CPU 21 calculates data related to a given index representing the motion state of the body B in the up-down direction, left-right direction, and front-back direction. Here, the index representing the motion state can be appropriately determined corresponding to the purpose of analyzing the motion state. As the index, for example, in addition to Figure 4 In addition to the vertical change trajectory of the trunk B shown in the middle layer of , the change amplitude of the vertical position, the change trajectory or change amplitude of the left-right position, the change of the rotation angle centered on the trunk, and the change of the inclination angle of the trunk can also be cited. In addition, it is not limited to the index represented by the position and angle of the trunk B, and it can also be an index represented by the speed or acceleration of the trunk B. The data of the position of the trunk B can be obtained by integrating the measurement data of the acceleration twice, and the data of the speed can be obtained by integrating the measurement data of the acceleration once. The rotation angle around each axis can be obtained by integrating the angular velocity. In addition, when the measurement data of the second sensor unit 23 includes a twisting (rotation) action centered on the trunk (rotation axis) of the trunk B, the trunk motion data 123 can also be generated based on the data obtained by pre-deducting the twisting action. In this way, the trunk motion data 123 representing the motion state of a part (such as the trunk) represented by the whole body of the trunk B can be obtained. The second time stamp representing the time point is associated with the data at each time point of the body movement data 123 and stored.

[0078] The CPU 21 determines whether pairing with the first device 10 is ongoing (i.e., whether communication is ongoing) (step S105). If it is determined that pairing is ongoing (step S105: "Yes"), the CPU 21 sends the body motion data 123 (including the second time stamp) generated in step S104 to the first device 10 (step S106).

[0079] When step S106 is completed, or when it is determined in step S105 that the pairing with the first device 10 is not continued (the communication connection is disconnected) (step S105 "No"), CPU21 determines whether an instruction to end the detection of the movement state of body B has been given (step S107). The instruction may be an operation performed by user U to indicate the end of detection, or an operation to indicate power off, etc. Alternatively, when the end of the movement of user U is detected based on the detection result of the second sensor unit 23, it can be regarded as an instruction to end the detection. When it is determined that an instruction to end the detection of the movement state of body B has not been given (step S107 "No"), CPU21 returns the processing to step S101. When it is determined that an instruction to end the detection of the movement state of body B has been given (step S107 "Yes"), the body movement detection processing ends.

[0080] Figure 8 as well as Fig. 9 1 is a flowchart showing the control procedure of the CPU 11 in the arm movement detection process.

[0081] The arm movement detection process is started, for example, when the user U performs an operation to instruct the start of movement measurement.

[0082] When the arm movement detection process is started, the CPU 11 of the first device 10 obtains the current position calculated by the satellite radio wave reception processing unit 15, and determines the interval to which the moving distance from the movement start position to the current position belongs (for example, Figure 6 (Step S201) Alternatively, the current position may be determined by receiving the current position acquired by the satellite radio wave reception processing unit 25 of the second device 20.

[0083] The CPU 11 obtains the measurement data (detection result) of the first sensor unit 13 (step S202). Here, the CPU 11 uses the three-axis acceleration sensor 131, the three-axis gyro sensor 132, and the three-axis geomagnetic sensor 133 to obtain the measurement data detected at the same sampling frequency as the second sensor unit 23 of the second device 20 and at the first detection timing synchronized with the second detection timing of the second sensor unit 23. In addition, the CPU 11 stores the first timestamp of the first detection timing representing the measurement data in the memory 12 in correspondence with the obtained measurement data.

[0084] CPU 11 determines whether measurement data of the amount of motion cycle T has been accumulated (step S203). The determination in step S203 can be performed in the same way as the determination in step S102 of the body motion detection process. Here, measurement data from the peak value of the acceleration corresponding to the ground contact to the next peak value of the peak value can also be determined as measurement data of the amount of motion cycle T. In the case of determining that measurement data of the amount of motion cycle T has not been accumulated (step S203 "No"), CPU 11 returns the process to step S202.

[0085] When it is determined that the measurement data of the amount of the motion cycle T is accumulated (step S203 "Yes"), the CPU 11 determines the gravity direction and the travel direction based on the measurement data of the first sensor unit 13, and converts the measurement data into data of the up-down direction, the left-right direction, and the front-back direction (step S204). Here, the CPU 11 determines the gravity direction and the travel direction by the same method as step S103 of the body motion detection process, and converts them into measurement data of the world coordinate system.

[0086] Based on the measurement data of the motion cycle T of the first sensor unit 13, the CPU 11 generates composite arm motion data 122 (step S205) related to the composite motion state of the arm A superimposed with the motion state of the trunk B in each of the up-down direction, the left-right direction, and the front-back direction of the arm A. For example, the CPU 11 calculates data related to a given index characterizing the composite motion state of the arm A in the up-down direction, the left-right direction, and the front-back direction. Here, the index characterizing the composite motion state can be appropriately determined corresponding to the purpose of analyzing the motion state. As such an index, for example, in addition to Figure 4 In addition to the vertical change trajectory of arm A shown by the solid line L1 in the lower layer, the change amplitude of the position in the vertical direction, the change trajectory or change amplitude of the position in the left and right direction, the change trajectory or change amplitude of the position in the front and back direction, and the shift of the rotation angle centered on the shoulder joint can also be cited. In addition, it is not limited to indicators represented by the position and angle of arm A. It can be an indicator represented by the speed or acceleration of arm A. The position data of arm A can be obtained by integrating the measured data of acceleration twice, and the speed data can be obtained by integrating the measured data of acceleration once. The rotation angle around each axis can be obtained by integrating the angular velocity. The first timestamp representing the time point is stored in correspondence with the data of each time point of the composite arm motion data 122.

[0087] The CPU 11 receives the body motion data 123 (including the second time stamp) corresponding to the motion cycle T of the compound arm motion data 122 generated in step S205 from the second device 20 (step S206 ).

[0088] The CPU 11 determines whether the reception of the body motion data 123 is successful (step S207). If the reception is successful (step S207: "Yes"), it determines whether it is the timing to register the body motion history record data 125 corresponding to the interval (step S208). The timing is determined so that the body motion history record data 125 is registered at least once in each interval. For example, when the interval determined in step S101 is different from the interval determined in the previous step S101 (that is, when the current interval is switched), it can be determined that it is the timing to register the body motion history record data 125.

[0089] When it is determined that it is time to register the body motion history data 125 corresponding to the interval (step S208 "Yes"), the CPU 11 registers (stores) the body motion history data 123 received in step S206 in correspondence with the current interval as the body motion history data 125 in the memory 12 (step S209). In addition, when the body motion history data 125 corresponding to the current interval is already stored in the memory 12, the acquired body motion history data 123 may be overwritten on the existing body motion history data 125, or the existing body motion history data 125 may be retained without being overwritten. Alternatively, the body motion history data 125 may be newly generated based on the existing body motion history data 125 stored in correspondence with the current interval and the body motion data 123 acquired in step S206. The newly generated body motion history data 125 may be, for example, data of a representative value (e.g., an average value) of the value of the existing body motion history data 125 and the value of the acquired body motion data 123.

[0090] When step S209 is finished, or when it is determined in step S208 that the body motion history record data 125 corresponding to the interval is not registered at a timing (step S208 "No"), CPU 11 generates arm motion data 124 related to the individual motion state based on the difference between the compound arm motion data 122 generated in step S205 and the body motion data 123 obtained in step S206 for each direction of the up-down direction, the left-right direction, and the front-back direction (step S210). Here, CPU 11 refers to the first timestamp of the compound arm motion data 122 and the second timestamp of the body motion data 123 to calculate the difference between the corresponding timing data in the compound arm motion data 122 and the body motion data 123, thereby generating the arm motion data 124. For example, the body motion data 123 contains the data representing Figure 4 The data representing the change in waist height shown in the middle layer and the composite arm motion data 122 include Figure 4In the case of data representing the change in the height of the wrist in the compound motion state as shown by the solid line L1 in the lower layer of , the arm motion data 124 including data of the waist height at each time point of the body motion data 123 deducted from the wrist height at each time point from the compound arm motion data 122 (data representing the change in the height of the wrist in the single motion state as shown by the dotted line L2) is generated. In the case where the compound arm motion data 122 includes data related to a plurality of different indices representing the compound motion state of arm A, and the body motion data 123 includes data related to a plurality of different indices representing the motion state of body B, data representing the single motion state of arm A is generated based on the difference between the compound arm motion data 122 and the body motion data 123 for each indices, and the arm motion data 124 including the data is generated. Here, as multiple different indices representing the complex motion state of arm A, for example, the change trajectory or change range of arm A in the up-down direction, the change trajectory or change range of the position in the left-right direction, the change trajectory or change range of the position in the front-back direction, the transition of the rotation angle centered on the shoulder joint, the transition of the speed change of arm A, and the transition of the acceleration of arm A can be cited. In addition, as multiple different indices representing the motion state of trunk B, for example, the change trajectory or change range of the position in the up-down direction of trunk B, the change trajectory or change range of the position in the left-right direction, the transition of the rotation angle centered on the trunk, the transition of the inclination angle from the vertical direction of the trunk, the transition of the speed change of trunk B, and the transition of the acceleration change of trunk B can be cited.

[0091] In step S207, when it is determined that the reception of the body motion data 123 has failed (step S207: No), the CPU 11 determines whether pairing with the second device 20 is ongoing (i.e., whether the communication connection is ongoing) (step S211). When it is determined that pairing is ongoing (step S211: Yes), the CPU 11 returns the process to step S206 and tries to receive the body motion data 123 again.

[0092] When it is determined that the pairing with the second device 20 is not continued (the communication connection is disconnected) (step S211 "No"), the CPU 11 determines whether the body movement history record data 125 corresponding to the current interval is registered in the memory 12 (step S212). When it is determined that the body movement history record data 125 corresponding to the current interval is stored in the memory 12 (step S212 "Yes"), the CPU 11 obtains the body movement history record data 125 corresponding to the current interval from the memory 12 (step S213). On the other hand, when it is determined that the body movement history record data 125 corresponding to the current interval is not stored in the memory 12 (step S212 "No"), the CPU 11 obtains the body movement history record data 125 corresponding to the final interval ( Figure 6 In the example of the interval S3), the body movement history data 125 corresponding to the interval S3 is obtained (step S214). In addition, when step S209 has not been executed in the past and the body movement history data 125 is not stored in the memory 12, the CPU 11 obtains the predetermined body movement data (body movement data of a given value) from the memory 12 as the body movement history data 125.

[0093] When step S213 or step S214 is completed, CPU 11 generates arm motion data 124 related to the individual motion state based on the difference between the compound arm motion data 122 generated in step S205 and the body motion history record data 125 obtained in step S213 or step S214 for each direction of the up-down direction, the left-right direction, and the front-back direction (step S215). Here, although the first timestamp of the compound arm motion data 122 and the second timestamp of the body motion history record data 125 represent different moments, for example, the first timestamp and the second timestamp may be converted into timestamps representing the elapsed time from the start time point of the motion cycle T for comparison. The processing of step S215 is the same as the processing of step S210, except that the body motion history record data 125 is used instead of the body motion data 123.

[0094] When the processing of step S210 or step S215 is completed, CPU 11 generates arm swing determination data based on arm motion data 124 and makes a determination related to the arm swing state (step S216). The data items contained in the arm swing determination data are appropriately set according to the items to be determined. Figure 5In order to determine whether the swing amplitude W2 of the arm A in the up-down direction shown is greater than a given reference value, arm swing determination data representing the swing amplitude W2 of the arm A is generated based on the arm motion data 124. Then, it is determined whether the swing amplitude W2 represented by the arm swing determination data is greater than the reference value. Since the arm swing determination data generated here is generated based on the arm motion data 124 related to the independent motion state of the arm A, it is possible to determine the correct swing amplitude W2 of the arm A that does not overlap the motion state of the trunk B. In addition, the object of determination related to the arm swing state is not limited to Figure 5 The swing amplitude W2 of the arm A in the up-down direction shown may also be the trajectory of the arm swing in the up-down direction, the swing amplitude W1 of the arm A in the front-to-back direction or the trajectory of the arm swing, the swing amplitude W1 of the arm A in the left-to-right direction or the trajectory of the arm swing, the speed of the arm A, and the acceleration of the arm A. In addition, the determination involving the arm swing state is not limited to the determination of whether the determination criterion is met. For example, it can be determined which evaluation level among multiple evaluation levels the swing amplitude W1, W2, etc. of the arm A meets, or the numerical value itself that characterizes the swing amplitude W1, W2, etc. of the arm A can be represented. In addition, the category of the arm swing posture can also be determined based on the trajectory of the arm swing in each direction, etc. As the category of the arm swing posture, there are, for example, a linear arm swing posture that makes the arm A move back and forth relatively straight, and a rotational arm swing posture that performs an activity such as rotating the arm A during the arm swing action.

[0095] CPU11 displays information including the determination result of the arm swing state on the display unit 16 (step S217). Here, various additional information may be displayed in addition to the determination result itself, such as information suggesting how to improve the arm swing posture. In addition, the determination result may be notified by other methods other than the display of information on the display unit 16 (or in place of the display). For example, a vibration unit may be provided in the first device 10, and the determination result may be notified by the vibration mode, the magnitude of the vibration, etc. of the vibration unit. In addition, a notification sound output unit may be provided in the first device 10, and the determination result may be notified by the mode of the notification sound, etc. In addition, a light-emitting unit may be provided in the first device 10, and the determination result may be notified by the light-emitting mode of the light-emitting unit, etc.

[0096] CPU11 determines whether an instruction to end the detection of the motion state of arm A has been given (step S218). The instruction may be an operation performed by user U to indicate the end of detection, or an operation to indicate power off, etc. Alternatively, when the end of the motion of user U is detected based on the detection result of the first sensor unit 13, it may be regarded as an instruction to end the detection. When CPU11 determines that an instruction to end the detection of the motion state of arm A has not been given (step S218 "No"), the processing returns to step S201. When it is determined that an instruction to end the detection of the motion state of arm A has been given (step S218 "Yes"), the arm motion detection processing ends.

[0097] Figure 7 The body motion detection process shown and Figure 8 , Fig. 9 The arm motion detection process shown can be performed in each of a plurality of continuous motion cycles T, or can be performed once for a given number of motion cycles T of 2 or more. In addition, the arm swing state can be determined based on a representative value (e.g., an average value) of a plurality of arm motion data 124 generated in a plurality of arm motion detection processes. Fig. 9 After the information including the determination result of the arm swinging state is displayed on the display unit 16 in step S217, since the user U swings his arm disorderly in order to confirm the display unit 16 of the first device 10 equipped on the arm, the body movement detection processing and the arm movement detection processing may not be performed during this period.

[0098] In addition, arm motion detection processing Fig. 9 Steps S210, S215, S216, and S217 may not necessarily be performed during the user's exercise, but may be performed after the exercise. In addition, in the second device 20, the acquisition of the measurement data performed by the second sensor unit 23 and the generation of the body movement data 123 are performed, but in the case where the pairing with the first device 10 is not performed and the body movement data 123 cannot be sent to the first device 10 ( Figure 7 In step S105 "No"), the body motion data 123 may be sent to the first device 10 when pairing is performed later. In this case, the smartphone 10 may execute the body motion data 123 when receiving the sent body motion data 123. Fig. 9 Steps S210, S216, and S217.

[0099] In addition, it can also be performed during exercise Fig. 9 Steps S210 and S215 are performed, and after the exercise (for example, for subsequent analysis), the processing related to the determination of the arm swing state of steps S216 and S217 is performed.

[0100] <Modification>

[0101] The first device 10 can also be equipped on the legs (e.g., ankles, thighs) of the user U. When the first device 10 is equipped on the legs, the first device 10 obtains compound leg motion data (first information) related to the compound motion state of the legs (first state) superimposed with the motion state of the trunk B. In this case, the same processing as in the above-mentioned embodiment can be used to generate leg motion data (third information) related to the individual motion state of the legs based on the trunk B based on the difference between the compound leg motion data and the trunk motion data 123. In addition, the individual motion state of the arm and the individual motion state of the leg can be determined by equipping the first device 10 on the arm and the leg, respectively.

[0102] <Effect>

[0103] As described above, the first device 10 as the information processing device involved in the present embodiment includes a CPU 11 as a processing unit. The CPU 11 obtains compound arm motion data 122 (first information) related to the compound motion state (first motion state) of at least one of the arm and leg of the user U superimposed with the motion state of the body of the user U who is exercising, obtains body motion data 123 (second information) related to the motion state (second motion state) of the body of the user U who is exercising, and generates arm motion data 124 (third information) related to the individual motion state (third motion state) of at least one of the arm and leg of the user U based on the compound arm motion data 122 and the body motion data 123 (or the body motion history record data 125).

[0104] According to this, compared with the prior art that only uses electronic devices equipped on the arms to determine the compound motion state of the arms or legs, the individual motion state of at least one of the arms and legs can be determined more accurately. In addition, since the motion state of the body of the user U can be deducted from the compound motion state of the arms, the individual motion state of the arms can be determined with high accuracy compared with the method of deducting the motion state of the body of others. Therefore, for example, the arm swing posture and leg swing posture during running can be analyzed more accurately, which can contribute to the improvement of more effective postures.

[0105] Furthermore, the composite arm motion data 122 may be data related to the composite motion state of at least one of the arm and the leg of the user U in the first period, and the body motion data 123 may be data related to the motion state of the body of the user U in the first period. Thus, the real-time motion state of the body when the composite arm motion data 122 is acquired can be deducted from the composite motion state of the arm. Therefore, the individual motion state of the arm can be determined more accurately.

[0106] Furthermore, the composite arm motion data 122 is generated based on the detection results of the composite motion state at each of the plurality of first detection timings, and the body motion data 123 is generated based on the detection results of the motion state of the body at each of the plurality of second detection timings synchronized with the plurality of first detection timings. Thus, since the motion state of the body at each time point in the exercise can be deducted from the composite motion state of the arm, the individual motion state of the arm can be determined more accurately.

[0107] Furthermore, the composite arm motion data 122 may be data related to the composite motion state of at least one of the arm and the leg of the user U in the first period, and the body motion history record data 125 may be data related to the motion state of the body of the user U in the second period before the first period. Thus, since the motion state of the body of the user U can also be deducted from the composite motion state of the arm, the individual motion state of the arm can be determined more accurately compared to the prior art that determines the composite motion state of the arm or the leg using only the electronic device equipped on the arm. Furthermore, since the past body motion history record data 125 is used, when obtaining the composite arm motion data 122, it is not necessary to determine the motion state of the body in real time to generate the body motion data 123. Therefore, even in the state where only the first device 10 is equipped, the individual motion state of the arm can be determined. Furthermore, even in the case where the communication connection between the first device 10 and the second device 20 is cut off and the body motion data 123 cannot be obtained from the second device 20, the individual motion state of the arm can be determined.

[0108] In addition, as the user U moves, the CPU 11 obtains the body motion history record data 125 related to the motion state of the body in each of the plurality of intervals formed by dividing the movement path after the movement starts in the second period, determines to which interval of the plurality of intervals the movement distance of the user U after the movement starts in the first period belongs, and generates the arm motion data 124 based on the composite arm motion data 122 and the body motion history record data 125 corresponding to the determined interval. Thus, even if the running posture (body motion state) changes due to fatigue or the like depending on the running distance, the body motion state reflecting the change can be deducted from the composite motion state of the arm. Therefore, the individual motion state of the arm can be determined more accurately.

[0109] Furthermore, when the moving distance of the user U in the first period is longer than the total of the plurality of intervals, the CPU 11 generates the arm movement data 124 based on the compound arm movement data 122 and the body movement history record data 125 corresponding to the last interval among the plurality of intervals. Thus, the arm movement data 124 can be generated using the body movement history record data 125 of the movement state closest to the movement state of the body at the current moving distance. Therefore, even when there is no interval corresponding to the current moving distance, the individual movement state of the arm can be correctly determined within the possible range.

[0110] Furthermore, at least one of the arms and legs of the user U who is exercising performs periodic motions, and the composite arm motion data 122 is information related to the composite motion state in one motion cycle T (1 cycle) of the periodic motions, and the body motion data 123 is information related to the motion state of the body B in the motion cycle T. Thus, the individual motion state of the arm in the motion cycle T can be determined.

[0111] In addition, the first device 10 includes a first sensor unit 13, which is equipped on the arm or leg of the user U for use, and is used as a detection unit for detecting the motion state of the device, and the CPU 11 generates composite arm motion data 122 based on the detection result of the first sensor unit 13. Thus, the arm motion data 124 can be generated in the first device 10 equipped on the arm to determine the individual motion state of the arm.

[0112] Furthermore, the information processing system 1 according to the present embodiment includes: a first device 10, which is provided for use on at least one of the arm and the leg of the user U and has a first sensor unit 13 for detecting the motion state of the device; a second device 20, which is provided for use on the body of the user U and has a second sensor unit 23 for detecting the motion state of the device; and a CPU 11 as a processing unit. The CPU 11 obtains composite arm motion data 122 related to the composite motion state of at least one of the arm and the leg of the user U, which is generated based on the detection result of the first sensor unit 13 and is superimposed with the motion state of the body of the user U who is exercising, obtains body motion data 123 related to the motion state of the body of the user U who is exercising, which is generated based on the detection result of the second sensor unit 23, and generates arm motion data 124 related to the individual motion state of at least one of the arm and the leg of the user U based on the composite arm motion data 122 and the body motion data 123. According to this, compared with the existing technology that uses only electronic equipment equipped on the arm to determine the complex movement state of the arm or the leg, the individual movement state of at least one of the arm and the leg can be determined more accurately.

[0113] In addition, the program 121 according to the present embodiment causes the CPU 11 as a computer to implement the following functions: obtaining composite arm motion data 122 related to the composite motion state of at least one of the arm and the leg of the user U superimposed with the motion state of the body of the user U who is exercising; obtaining body motion data 123 related to the motion state of the body of the user U who is exercising; and generating arm motion data 124 related to the individual motion state of at least one of the arm and the leg of the user U based on the composite arm motion data 122 and the body motion data 123. Thus, compared with the conventional technology that determines the composite motion state of the arm or the leg using only the electronic device equipped on the arm, the individual motion state of at least one of the arm and the leg can be determined more accurately.

[0114] Furthermore, the information processing method performed by the CPU 11 as a computer according to the present embodiment includes the steps of: obtaining composite arm motion data 122 related to the composite motion state of at least one of the arm and the leg of the user U superimposed with the motion state of the body of the user U who is exercising; obtaining the composite arm motion data 122 related to the motion state of the body of the user U who is exercising; and generating arm motion data 124 related to the individual motion state of at least one of the arm and the leg of the user U based on the composite arm motion data 122 and the body motion data 123. Thus, compared with the conventional technique of determining the composite motion state of the arm or the leg using only the electronic device equipped on the arm, the individual motion state of at least one of the arm and the leg can be determined more accurately.

[0115] <Others>

[0116] In addition, the description in the above-mentioned embodiment is an example of the information processing device, information processing system, program, and information processing method according to the present invention, but the present invention is not limited thereto.

[0117] For example, in the above-mentioned embodiment, the arm motion data 124 as the third information is generated based on the difference between the compound arm motion data 122 and the body motion data 123 (or the body motion history record data 125), but the third information may be generated by a method that does not use the difference between the compound arm motion data 122 and the body motion data 123 (or the body motion history record data 125). For example, the compound arm motion data 122 and the body motion data 123 (or the body motion history record data 125) may be input to a recognizer generated by supervised machine learning, and the third information related to the independent motion state of the arm may be output from the recognizer. The machine learning of the recognizer in this case is performed by using the compound arm motion data 122 and the body motion data 123 (or the body motion history record data 125) as inputs and the arm motion data obtained based on the image of the arm in motion captured by a camera as teaching data. In addition, the third information in this case may be, for example, the determination result of the arm swing state (evaluation data related to the evaluation level of the arm swing). That is, a recognition device generated by machine learning may be used to directly obtain the determination result of the arm swing state as the third information without calculating the difference between the composite arm motion data 122 and the body motion data 123 (or the body motion history data 125) (without passing through the arm motion data 124). The recognition device is not particularly limited, and may be, for example, a neural network or a support vector machine.

[0118] In addition, the first device 10 may have a plurality of processors (e.g., a plurality of CPUs) that execute the plurality of processes executed by the CPU 11 of the first device 10 in the above-mentioned embodiment. In this case, the plurality of processors corresponds to a "processing unit". In addition, the second device 20 may have a plurality of processors (e.g., a plurality of CPUs) that execute the plurality of processes executed by the CPU 21 of the second device 20 in the above-mentioned embodiment. In these cases, the plurality of processors may participate in a common process, or the plurality of processors may independently and parallelly execute different processes.

[0119] In addition, an external device (e.g., a smartphone, a PC (Personal Computer), etc.) separate from the first device 10 and the second device 20 may obtain the composite arm motion data 122 from the first device 10 and the body motion data 123 from the second device 20, and generate the arm motion data 124 based on the obtained composite arm motion data 122 and body motion data 123. In this case, the external device is equivalent to an "information processing device". According to such an external device, the motion state of the user U can be analyzed afterwards based on the arm motion data 124.

[0120] Alternatively, the second device 20 may receive the composite arm motion data 122 from the first device 10, and the CPU 21 of the second device 20 may generate the arm motion data 124 based on the composite arm motion data 122 and the body motion data 123. In this case, the second device 20 corresponds to an "information processing device".

[0121] In addition, the first sensor unit 13 can detect the motion state of the first device 10 and is not limited to the structure illustrated in the above embodiment. In addition, the second sensor unit 23 can detect the motion state of the second device 20 and is not limited to the structure illustrated in the above embodiment. For example, in the case of a three-axis geomagnetic sensor 133 (233), it can be used as a magnetic gyroscope instead of a three-axis angular velocity sensor to detect the magnitude of the angular velocity centered on each axis.

[0122] In addition, in the above-mentioned embodiment, an example of using measurement data based on the amount of movement cycle T to generate composite arm motion data 122, body motion data 123 and arm motion data 124 is described, but it is not limited to this. Composite arm motion data 122, body motion data 123 and arm motion data 124 can also be generated based on measurement data corresponding to an arbitrary period.

[0123] In addition, in the above description, an example of using the memory 12 as a computer-readable medium of the program involved in the present invention is disclosed, but it is not limited to this example. As other computer-readable media, information recording media such as HDD, SSD, flash memory, CD-ROM, etc. can be used. In addition, as a medium for providing data of the program involved in the present invention via a communication line, a carrier wave is also used in the present invention.

[0124] Furthermore, it is needless to say that the detailed structures and detailed operations of the components of the first device 10 and the second device 20 in the above-described embodiment can be appropriately modified within the scope not departing from the gist of the present invention.

[0125] Although the embodiment of the present invention has been described, the scope of the present invention is not limited to the above-described embodiment, and includes the scope of the invention described in the claims and the scope equivalent thereto.

Claims

1. An information processing device, characterized in that: Equipped with processing department, The processing unit performs the following processing: obtaining first information related to a first movement state of at least one of an arm and a leg of a user superimposed with a movement state of a body of the user who is exercising, obtaining second information related to a second movement state of a body of the user performing the exercise, generating third information related to a third movement state of at least one of the arm and the leg of the user based on the first information and the second information, The first information is information related to the first movement state of at least one of the arm and the leg of the user in the first period, The second information is information related to the second movement state of the user's body in a second period before the first period, The motion accompanies the movement of the user, The processing unit performs the following processing: acquiring the second information related to the second movement state in each of a plurality of sections formed by dividing the movement path after the movement starts in the second period, It is determined which section of the plurality of sections the user's movement distance after starting the movement in the first period belongs to, and the third information is generated based on the first information and the second information corresponding to the determined section.

2. The information processing device according to claim 1, characterized in that The first information is generated based on the detection result of the first motion state at each of a plurality of first detection timings.

3. The information processing device according to claim 1, characterized in that The processing unit performs the following processing: When the movement distance of the user in the first period is longer than the total of the plurality of sections, the third information is generated based on the first information and the second information corresponding to the last section among the plurality of sections.

4. The information processing device according to any one of claims 1 to 3, characterized in that: At least one of the arm and the leg of the user performing the exercise performs a periodic movement, The first information is information related to the first motion state in one cycle of the periodic action. The second information is information related to the second motion state in the one cycle.

5. The information processing device according to any one of claims 1 to 3, characterized in that: The information processing device comprises: A detection unit is provided on the arm or leg of the user to detect the motion state of the self-device, The processing unit generates the first information based on the detection result of the detection unit.

6. An information processing system, characterized in that: have: A first device is mounted on at least one of an arm and a leg of a user for use, and has a first detection unit for detecting a motion state of the device; a second device, which is mounted on the body of the user and has a second detection unit for detecting a motion state of the device; and Processing Department, The processing unit performs the following processing: acquiring first information related to a first movement state of at least one of the arm and the leg of the user, which is generated based on the detection result of the first detection unit and superimposed with the movement state of the body of the user who is exercising, acquiring second information related to a second movement state of a body of the user performing the exercise, which is generated based on a detection result of the second detection unit, generating third information related to a third movement state of at least one of the arm and the leg of the user based on the first information and the second information, The first information is information related to the first movement state of at least one of the arm and the leg of the user in the first period, The second information is information related to the second movement state of the user's body in a second period before the first period, The motion accompanies the movement of the user, The processing unit performs the following processing: acquiring the second information related to the second movement state in each of a plurality of sections formed by dividing the movement path after the movement starts in the second period, It is determined which section of the plurality of sections the user's movement distance after starting the movement in the first period belongs to, and the third information is generated based on the first information and the second information corresponding to the determined section.

7. A non-transitory storage medium storing a program, characterized in that: The program enables the computer to implement the following functions: acquiring first information related to a first movement state of at least one of an arm and a leg of a user performing exercise, superimposed with a movement state of a body of the user performing exercise; acquiring second information related to a second movement state of a body of the user performing the exercise; and generating third information related to a third movement state of at least one of the arm and the leg of the user based on the first information and the second information, The first information is information related to the first movement state of at least one of the arm and the leg of the user in the first period, The second information is information related to the second movement state of the user's body in a second period before the first period, The motion accompanies the movement of the user, The computer performs the following processing: acquiring the second information related to the second movement state in each of a plurality of sections formed by dividing the movement path after the movement starts in the second period, It is determined which section of the plurality of sections the user's movement distance after starting the movement in the first period belongs to, and the third information is generated based on the first information and the second information corresponding to the determined section.

8. An information processing method is an information processing method performed by a computer, characterized in that: The following steps are included: acquiring first information related to a first movement state of at least one of an arm and a leg of a user performing exercise, superimposed with a movement state of a body of the user performing exercise; acquiring second information related to a second movement state of a body of the user performing the exercise; and generating third information related to a third movement state of at least one of the arm and the leg of the user based on the first information and the second information, The first information is information related to the first movement state of at least one of the arm and the leg of the user in the first period, The second information is information related to the second movement state of the user's body in a second period before the first period, The motion accompanies the movement of the user, The computer performs the following processing: acquiring the second information related to the second movement state in each of a plurality of sections formed by dividing the movement path after the movement starts in the second period, It is determined which section of the plurality of sections the user's movement distance after starting the movement in the first period belongs to, and the third information is generated based on the first information and the second information corresponding to the determined section.

Images