Information processing device, electronic device, information processing system, information processing method, and program

Abstract

This information processing device comprises a control unit. The control unit acquires sensor data indicating the motion of a part of a user's body from at least one piece of sensor equipment worn on the part of the body. From the acquired sensor data and a learning model, the control unit infers a state of a part of the user's body different from the part of the user's body where the sensor equipment is worn.

Description

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority from Japanese Patent Application No. 2021-090695, filed with the Japan Patent Office on May 28, 2021, and the entire disclosure of said prior application is incorporated herein by reference. Technical Field

[0002] The present disclosure relates to an information processing apparatus, an electronic device, an information processing system, an information processing method, and a program. Background Art

[0003] Conventionally, a technique for estimating a person's walking state is known (for example, Patent Document 1). The motion analysis device described in Patent Document 1 includes a detection unit attached to a person's body and an analysis unit. The analysis unit analyzes a walking state and / or running state based on a signal from the detection unit. In addition, as an interpretation of good walking, the interpretation described in Non-Patent Document 1 is known. Prior Art Literature Patent Literature

[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2016-150193 Non-Patent Literature

[0005] Non-Patent Document 1 Asics Sports Science Institute, "The Ultimate Way of Walking", Kodansha Gendai Shinsho, published September 2019, pp. 92, 94, 95 Summary of the Invention

[0006] An information processing apparatus according to an embodiment of the present disclosure is: acquiring sensor data indicating movement of a user's body part from at least one sensor device attached to said body part, The system includes a control unit that estimates the state of a body part of the user that is different from the body part to which the sensor device is attached, based on the acquired sensor data and a learning model.

[0007] An electronic device according to one embodiment of the present disclosure includes a notification unit that notifies information about the state of the body part estimated by the information processing device.

[0008] An information processing system according to one embodiment of this disclosure is At least one sensor device attached to a part of the user's body, The system includes an information processing device that acquires sensor data indicating the movement of the body part from the sensor device, and estimates the state of a body part in the user that is different from the body part to which the sensor device is attached, using the acquired sensor data and a learning model.

[0009] An information processing method according to one embodiment of this disclosure is: To acquire sensor data indicating the movement of a body part from at least one sensor device attached to a body part of the user, This includes estimating the state of a body part of the user that is different from the body part to which the sensor device is attached, using the acquired sensor data and a learning model.

[0010] A program according to one embodiment of this disclosure is On the computer, To acquire sensor data indicating the movement of a body part from at least one sensor device attached to a body part of the user, The system is configured to estimate the state of a body part of the user that is different from the body part to which the sensor device is attached, using the acquired sensor data and the learning model. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows a schematic configuration of an information processing system according to one embodiment of the present disclosure. [Figure 2]FIG. 1 is a diagram for explaining a local coordinate system and a global coordinate system. [Figure 3] FIG. 2 is a functional block diagram showing the configuration of the information processing system shown in FIG. 1. [Figure 4] FIG. 3 is a diagram showing a schematic configuration of a learning model according to an embodiment of the present disclosure. [Figure 5] FIG. 4 is a diagram showing an example of a score according to an embodiment of the present disclosure. [Figure 6] FIG. 5 is a diagram showing an example of association according to an embodiment of the present disclosure. [Figure 7] FIG. 6 is a graph showing the accuracy of a learning model. [Figure 8] FIG. 7 is a graph showing the precision of a learning model. [Figure 9] FIG. 8 is a flowchart showing the operation of evaluation processing executed by the electronic device shown in FIG. 1. [Figure 10] FIG. 9 is a functional block diagram showing the configuration of an information processing system according to another embodiment of the present disclosure. [Figure 11] FIG. 10 is a sequence diagram showing the operation of evaluation processing executed by the information processing system shown in FIG. 9. DETAILED DESCRIPTION OF EMBODIMENTS

[0012] A novel technique for estimating the state of a body part of a walking user is desired. According to the present disclosure, a novel technique for estimating the state of a body part of a walking user can be provided.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the components shown in the following drawings, the same components are denoted by the same reference numerals.

[0014] System Configuration An information processing system 1 as shown in FIG. 1 can estimate the state of any body part among a plurality of body parts of a user during walking. By using the information processing system 1, a user can grasp whether the state of his / her own body part during walking is good or bad.

[0015] The information processing system 1 may be used for any application where it is required to grasp the state of a body part of a user during walking. For example, the information processing system 1 may be used to grasp the state of a user's body part when the user walks for exercise, when the user practices walking as training, when the user practices footwork for mountain climbing, or when the user practices race walking, etc.

[0016] The information processing system 1 includes a sensor device 10A, a sensor device 10B, a sensor device 10C, a sensor device 10D, and an electronic device 20. However, the information processing system 1 does not need to include all of the sensor device 10A, the sensor device 10B, the sensor device 10C, and the sensor device 10D. The information processing system 1 only needs to include at least one of the sensor device 10A, the sensor device 10B, the sensor device 10C, and the sensor device 10D.

[0017] Hereinafter, when the sensor devices 10A to 10D are not particularly distinguished from each other, they are also collectively referred to as "sensor device 10".

[0018] The sensor device 10 and the electronic device 20 are capable of communicating via a communication line. The communication line is configured to include at least one of wired and wireless communication.

[0019] The sensor device 10 is attached to a body part of a user. The sensor device 10 detects sensor data indicating the movement of the user's body part to which the sensor device 10 is attached. The sensor data is data in a local coordinate system.

[0020] The local coordinate system is a coordinate system based on the position of the sensor device 10, as shown in Figure 2. In Figure 2, the position of sensor device 10A is shown by a dashed line as an example of the position of sensor device 10. The local coordinate system consists of, for example, the x-axis, y-axis, and z-axis. The x-axis, y-axis, and z-axis are orthogonal to each other. The x-axis is parallel to the front-to-back direction as viewed from sensor device 10. The y-axis is parallel to the left-to-right direction as viewed from sensor device 10. The z-axis is parallel to the up-to-down direction as viewed from sensor device 10. The direction parallel to the x-axis of the local coordinate system is also referred to as the "front-to-back direction of the local coordinate system". The direction parallel to the y-axis of the local coordinate system is also referred to as the "left-to-right direction of the local coordinate system". The direction parallel to the z-axis of the local coordinate system is also referred to as the "up-to-down direction of the local coordinate system".

[0021] The global coordinate system, as shown in Figure 2, is a coordinate system based on the user's position in the space in which they walk. The global coordinate system consists of, for example, the X, Y, and Z axes. The X, Y, and Z axes are orthogonal to each other. The X axis is parallel to the front-to-back direction as seen from the user. The Y axis is parallel to the left-to-right direction as seen from the user. The Z axis is parallel to the up-to-down direction as seen from the user. The direction parallel to the X axis of the global coordinate system is also referred to as the "front-to-back direction of the global coordinate system." The direction parallel to the Y axis of the global coordinate system is also referred to as the "left-to-right direction of the global coordinate system." The direction parallel to the Z axis of the global coordinate system is also referred to as the "up-to-down direction of the global coordinate system." The front-to-back direction of the global coordinate system can also be rephrased as the user's front-to-back direction. The left-to-right direction of the global coordinate system can also be rephrased as the user's left-to-right direction. The up-to-down direction of the global coordinate system can also be rephrased as the user's up-to-down direction.

[0022] The sagittal plane, as shown in Figure 2, is the plane that divides the user's body symmetrically from left to right, or the plane parallel to the plane that divides the user's body symmetrically from left to right. The frontal plane is the plane that divides the user's body into ventral and dorsal halves, or the plane parallel to the plane that divides the user's body into ventral and dorsal halves. The horizontal plane is the plane that divides the user's body vertically, or the plane parallel to the plane that divides the user's body vertically. The sagittal plane, the frontal plane, and the horizontal plane are perpendicular to each other.

[0023] As shown in Figure 1, the sensor device 10A is attached to the user's head. For example, the sensor device 10A is attached to the user's ear. The sensor device 10A may be a wearable device. The sensor device 10A may be an earphone. The sensor device 10A may be included in an earphone. Alternatively, the sensor device 10A may be a device that can be retrofitted to existing glasses or earphones, etc. The sensor device 10A may be attached to the user's head by any method. The sensor device 10A may be attached to the user's head by being installed in a hair accessory such as a hairband or hairpin, earrings, helmet, hat, hearing aid, dentures or implants, etc.

[0024] The sensor device 10A may be attached to the user's head such that the front-to-back direction as seen from the sensor device 10A coincides with the front-to-back direction of the head as seen from the user, the left-to-right direction as seen from the sensor device 10A coincides with the left-to-right direction of the head as seen from the user, and the up-and-down direction as seen from the sensor device 10 coincides with the up-and-down direction of the head as seen from the user. In other words, the sensor device 10A may be attached to the user's head such that the x-axis of the local coordinate system based on the position of the sensor device 10A is parallel to the front-to-back direction of the head as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the head as seen from the user, and the z-axis of the local coordinate system is parallel to the up-and-down direction of the head as seen from the user. However, the front-to-back, left-to-right, and up-and-down directions as seen from the sensor device 10A do not necessarily coincide with the front-to-back, left-to-right, and up-and-down directions of the head as seen from the user. In this case, the relative orientation of the sensor device 10A with respect to the user's head may be initialized or made known as appropriate. The initialization or determination of the relative posture may be performed by utilizing information about the shape of the jig used to attach the sensor device 10A to the user's head, or by using image information generated by imaging the user's head to which the sensor device 10A is attached.

[0025] The sensor device 10A detects sensor data indicating the movement of the user's head. The sensor data detected by the sensor device 10A includes, for example, data indicating at least one of the following: the velocity of the user's head, the acceleration of the user's head, the angle of the user's head, the angular velocity of the user's head, the temperature of the user's head, and the geomagnetic field at the position of the user's head.

[0026] The sensor device 10B is attached to the user's forearm. For example, the sensor device 10B is attached to the user's wrist. The sensor device 10B may be a wristwatch-type wearable device. The sensor device 10B may be attached to the user's forearm by any method. The sensor device 10B may be attached to the user's forearm by being installed on a band, bracelet, friendship bracelet, glove, ring, artificial nail, or prosthetic hand, etc. The bracelet may be worn by the user for decorative purposes, or it may be used to attach keys to a locker, etc., to the wrist.

[0027] The sensor device 10B may be attached to the user's forearm such that the front-to-back direction as seen from the sensor device 10B coincides with the front-to-back direction of the wrist as seen from the user, the left-to-right direction as seen from the sensor device 10B coincides with the left-to-right direction of the wrist as seen from the user, and the up-and-down direction as seen from the sensor device 10B coincides with the rotational direction of the wrist as seen from the user. The rotational direction of the wrist is the direction in which the wrist twists and rotates. In other words, the sensor device 10B may be attached to the user's forearm such that the x-axis of the local coordinate system, based on the position of the sensor device 10B, is parallel to the front-to-back direction of the wrist as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the wrist as seen from the user, and the z-axis of the local coordinate system is parallel to the rotational direction of the wrist as seen from the user.

[0028] The sensor device 10B detects sensor data indicating the movement of the user's forearm. The sensor data detected by the sensor device 10B includes, for example, data indicating at least one of the following: the velocity of the user's forearm, the acceleration of the user's forearm, the angle of the user's forearm, the angular velocity of the user's forearm, the temperature of the user's forearm, and the geomagnetic field at the position of the user's forearm.

[0029] The sensor device 10C is attached to the user's thigh. The sensor device 10C may be a wearable device. The sensor device 10C may be attached to the user's thigh by any method. The sensor device 10C may be attached to the user's thigh by a belt. The sensor device 10C may be attached to the thigh by being placed in a pocket near the thigh of the trousers worn by the user. The sensor device 10C may be attached to the user's thigh by being installed in trousers, underwear, shorts, a support, a prosthesis or implant, etc.

[0030] The sensor device 10C may be attached to the user's thigh such that the front-to-back direction as seen from the sensor device 10C coincides with the front-to-back direction of the thigh as seen from the user, the left-to-right direction as seen from the sensor device 10C coincides with the left-to-right direction of the thigh as seen from the user, and the up-and-down direction as seen from the sensor device 10C coincides with the rotational direction of the thigh as seen from the user. The rotational direction of the thigh is the direction in which the thigh twists and rotates. In other words, the sensor device 10C may be attached to the user's thigh such that the x-axis of the local coordinate system, based on the position of the sensor device 10C, is parallel to the front-to-back direction of the thigh as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the thigh as seen from the user, and the z-axis of the local coordinate system is parallel to the rotational direction of the thigh as seen from the user.

[0031] The sensor device 10C detects sensor data indicating the movement of the user's thigh. The sensor data detected by the sensor device 10C includes, for example, data indicating at least one of the following: the velocity of the user's thigh, the acceleration of the user's thigh, the angle of the user's thigh, the angular velocity of the user's thigh, the temperature of the user's thigh, and the geomagnetic field at the position of the user's thigh.

[0032] The sensor device 10D is attached to the user's foot. In this embodiment, the foot is the part of the user's foot from the ankle to the toes. The sensor device 10D may be a shoe-type wearable device. The sensor device 10D may be attached to the user's foot by any method. The sensor device 10D may be provided in a shoe. The sensor device 10D may be attached to the user's foot by being installed in an anklet, band, friendship bracelet, artificial nail, tattoo sticker, supporter, cast, sock, insole, prosthetic leg, ring or implant, etc.

[0033] The sensor device 10D may be attached to the user's foot such that the front-to-back direction as seen from the sensor device 10D coincides with the front-to-back direction of the foot as seen from the user, the left-to-right direction as seen from the sensor device 10D coincides with the left-to-right direction of the foot as seen from the user, and the up-and-down direction as seen from the sensor device 10 coincides with the up-and-down direction of the foot as seen from the user. In other words, the sensor device 10D may be attached to the user's foot such that the x-axis of the local coordinate system, based on the position of the sensor device 10D, is parallel to the front-to-back direction of the foot as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the foot as seen from the user, and the z-axis of the local coordinate system is parallel to the up-and-down direction of the foot as seen from the user.

[0034] The sensor device 10D detects sensor data indicating the movement of the user's foot. The sensor data detected by the sensor device 10D includes, for example, data indicating at least one of the following: the velocity of the user's foot, the acceleration of the user's foot, the angle of the user's foot, the angular velocity of the user's foot, the temperature of the user's foot, and the geomagnetic field at the position of the user's foot.

[0035] The electronic device 20 is carried, for example, by a user while walking. The electronic device 20 functions as an information processing device and can estimate the state of any of the user's multiple body parts based on sensor data detected by the sensor device 10. In this embodiment, the electronic device 20 determines an evaluation of the state of the user's body parts by estimating the state of the user's body parts. The electronic device 20 is, for example, a mobile device such as a mobile phone, smartphone, or tablet.

[0036] As shown in Figure 3, the sensor device 10 comprises at least a communication unit 11 and a sensor unit 12. The sensor device 10 may further comprise a notification unit for notifying information, a storage unit 14, and a control unit 15. In this embodiment, the notification unit is an output unit 13. However, the notification unit is not limited to the output unit 13. Sensor devices 10C and 10D do not need to include an output unit 13.

[0037] The communication unit 11 comprises at least one communication module capable of communicating with the electronic device 20 via a communication line. The communication module is a communication module that conforms to the communication line standard. The communication line standard is a short-range wireless communication standard that includes, for example, Bluetooth®, Wi-Fi®, infrared, and NFC (Near Field Communication).

[0038] The sensor unit 12 is configured to include any sensor corresponding to the sensor data to be detected by the sensor device 10. The sensor unit 12 is configured to include at least one of the following: a 3-axis motion sensor, a 3-axis acceleration sensor, a 3-axis velocity sensor, a 3-axis gyro sensor, a 3-axis geomagnetic sensor, a temperature sensor, a barometric pressure sensor, and a camera. If the sensor unit 12 is configured to include a camera, the movement of the user's body part can be detected by analyzing the image generated when the camera captures the body part of the user.

[0039] If the sensor unit 12 includes an acceleration sensor and a geomagnetic sensor, the data detected by the acceleration sensor and the geomagnetic sensor may be used to calculate the initial angle of the body part to be detected by the sensor device 10. In addition, the data detected by the acceleration sensor and the geomagnetic sensor may be used to correct the angle data detected by the sensor device 10.

[0040] If the sensor unit 12 includes a gyro sensor, the angle of the body part to be detected by the sensor device 10 may be calculated by integrating the angular velocity detected by the gyro sensor over time.

[0041] If the sensor unit 12 includes a pressure sensor, the data detected by the pressure sensor may be used when the control unit 26 of the electronic device 20, which will be described later, determines the evaluation of the user's body parts.

[0042] The output unit 13 is capable of outputting data. The output unit 13 is configured to include at least one output interface capable of outputting data. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.

[0043] If the output unit 13 is included in the sensor device 10A, it may include a speaker. If the output unit 13 is included in the sensor device 10B, it may include a display.

[0044] The storage unit 14 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or at least two combinations thereof. The semiconductor memory is, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The RAM is, for example, SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory). The ROM is, for example, EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 14 may function as a main memory, auxiliary memory, or cache memory. The storage unit 14 stores data used for the operation of the sensor device 10 and data obtained by the operation of the sensor device 10. For example, the storage unit 14 stores system programs, application programs, and embedded software.

[0045] The control unit 15 is configured to include at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 15 controls each part of the sensor device 10 and executes processes related to the operation of the sensor device 10.

[0046] The control unit 15 receives a signal from the electronic device 20 via the communication unit 11 to instruct the start of data detection. Upon receiving this signal, the control unit 15 starts data detection. For example, the control unit 15 acquires data detected by the sensor unit 12. The control unit 15 transmits the acquired data as sensor data to the electronic device 20 via the communication unit 11. The signal instructing the start of data detection is transmitted as a broadcast signal from the electronic device 20 to multiple sensor devices 10. By transmitting the signal instructing the start of data detection as a broadcast signal to multiple sensor devices 10, the multiple sensor devices 10 can start data detection simultaneously.

[0047] The control unit 15 acquires data from the sensor unit 12 at a preset time interval and transmits the acquired data as sensor data via the communication unit 11. This time interval may be set based on the walking speed of a typical user, etc. This time interval may be the same for each of the multiple sensor devices 10. By having the same time interval for each of the multiple sensor devices 10, the timing at which each of the multiple sensor devices 10 detects data can be synchronized.

[0048] As shown in Figure 3, the electronic device 20 is composed of a communication unit 21, an input unit 22, a notification unit for notifying information, a storage unit 25, and a control unit 26. In this embodiment, the notification unit is an output unit 23 and a vibration unit 24. However, the notification unit is not limited to the output unit 23 and the vibration unit 24. The output unit 23 and the vibration unit 24 may be mounted on the electronic device 20 or may be located near any of the sensor devices 10B, 10C, or 10D.

[0049] The communication unit 21 comprises at least one communication module capable of communicating with the sensor device 10 via a communication line. The communication module is at least one communication module that conforms to the communication line standard. The communication line standard is a short-range wireless communication standard that includes, for example, Bluetooth®, Wi-Fi®, infrared, and NFC.

[0050] The communication unit 21 may further include at least one communication module that can connect to the network 2 as shown in Figure 10 below. The communication module is, for example, a communication module that supports a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation).

[0051] The input unit 22 is capable of receiving input from the user. The input unit 22 is configured to include at least one input interface capable of receiving input from the user. The input interface may be, for example, a physical key, a capacitive key, a pointing device, a touchscreen integrated with a display, or a microphone.

[0052] The output unit 23 is capable of outputting data. The output unit 23 is configured to include at least one output interface capable of outputting data. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display.

[0053] The vibrating unit 24 is capable of vibrating the electronic device 20. The vibrating unit 24 is composed of a vibrating element, such as a piezoelectric element.

[0054] The storage unit 25 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or at least two combinations thereof. The semiconductor memory is, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 25 may function as a main memory, auxiliary memory, or cache memory. The storage unit 25 stores data used for the operation of the electronic device 20 and data obtained by the operation of the electronic device 20. For example, the storage unit 25 stores system programs, application programs, and embedded software. For example, the storage unit 25 stores learning models described later and correspondences as shown in Figure 6 described later.

[0055] The control unit 26 is configured to include at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA or ASIC. The control unit 26 controls each part of the electronic device 20 and executes processes related to the operation of the electronic device 20.

[0056] The control unit 26 receives an input from the input unit 22 instructing it to perform a gait evaluation. This input causes the electronic device 20 to perform a decision process to determine the evaluation of the state of a body part. This input is, for example, received from the input unit 22 by a user wearing a sensor device 10. The user, for example, receives this input from the input unit 22 before starting to walk. When the control unit 26 receives this input from the input unit 22, it sends a signal instructing the start of data detection as a broadcast signal to the multiple sensor devices 10 via the communication unit 21. After the signal instructing the start of data detection is sent to the multiple sensor devices 10, sensor data is transmitted from at least one of the sensor devices 10 to the electronic device 20.

[0057] The control unit 26 receives sensor data from at least one sensor device 10 via the communication unit 21. The control unit 26 acquires sensor data from the sensor device 10 by receiving the sensor data from the sensor device 10. As will be described later, the control unit 26 determines an evaluation of the state of the user's body parts based on the acquired sensor data and a learning model. The content of the evaluation of the state of the body parts may be set based on an interpretation of a generally accepted good way of walking. An example of an interpretation of a generally accepted good way of walking is the interpretation described in "ASICS Sports Research Institute, 'The Ultimate Way of Walking,' Kodansha Gendai Shinsho, September 2019, pp. 92, 94, 95." Before explaining the details of the evaluation determination process by the control unit 26, an example of an interpretation of a generally accepted good way of walking and the content of the evaluation will be explained. The control unit 26 may determine an evaluation of at least one of the following: (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet.

[0058] (1) Condition of the head An assessment of the condition of the head may be made.

[0059] Generally, it is understood that a user's body should move as little as possible while walking. When a user's body moves less while walking, their head also moves less than when their body moves more. Therefore, when a user's head moves less while walking, a higher evaluation of their head condition can be determined compared to when their head moves more.

[0060] When a user's chin is tucked in while walking, the user's head movement is reduced compared to when their chin is not tucked in. Therefore, when a user's chin is tucked in while walking, a higher evaluation of the head position can be determined compared to when their chin is not tucked in.

[0061] When a user's gaze is directed towards the distance while walking, the user's head movement is less than when their gaze is directed towards the nearby area. Therefore, when a user's gaze is directed towards the distance while walking, a higher evaluation of the state of their head can be determined compared to when their gaze is directed towards the nearby area.

[0062] (2) Condition of the arm An assessment of the condition of the arm may be determined.

[0063] Generally, a good arm position during walking is considered to be one where the arms are swung freely. Therefore, a larger arm swing during walking should be given a higher evaluation of the arm position than a smaller arm swing.

[0064] Generally, a good state for a user's arm while walking is considered to be when the arm is pulled backward. Therefore, when a user's arm is pulled backward while walking, a higher evaluation of the arm's condition should be determined compared to when the user's arm is not pulled backward.

[0065] (3) Condition of the trunk The assessment of the trunk condition can be determined.

[0066] Generally, a good posture for a user's torso while walking is considered to be one where the shoulders are open and the spine is straight. Therefore, a posture in which a user's shoulders are open and their spine is straight should be given a higher rating for their torso than a posture in which their shoulders are closed and their spine is curved.

[0067] Generally, a good posture for a user's torso while walking is considered to be one where the pelvis is upright and the lower back is extended. Therefore, a user with an upright pelvis and extended lower back should receive a higher rating for their torso posture than a user with an upright pelvis and a bent lower back.

[0068] (4) Condition of the knee The assessment of the knee condition can be determined.

[0069] Generally, a good knee position during walking is considered to be one where the knee is not bent. Therefore, a higher evaluation of the knee position should be given when the user's knee is not bent during walking compared to when it is bent.

[0070] (5) Condition of the foot The assessment of the foot condition may be determined.

[0071] Generally, it is understood that a user's stride length while walking should be as wide as possible. Therefore, a user with a wide stride while walking should receive a higher evaluation of their foot condition than a user with a narrow stride.

[0072] [Evaluation decision process] The control unit 26 determines an evaluation of the state of the user's body parts based on sensor data and a learned model. The learned model is machine-trained to output information evaluating the state of a predetermined body part of the user when sensor data or feature data is input. In other words, the control unit 26 inputs sensor data or feature data into the learned model and obtains information evaluating the state of the user's body parts from the learned model to determine an evaluation of the state of the user's body parts. Feature data is data that shows the characteristics of the movement of the body part to which the sensor device 10 is attached in the user. The control unit 26 obtains feature data from the sensor data. An example of feature data will be described later. Here, the learned model can be machine-trained to output information evaluating the state of a body part different from the body part to which the sensor device 10 is attached in the user when sensor data or feature data is input. This is because multiple body parts of the user move while influencing each other during walking. With such a learned model, the control unit 26 can determine an evaluation of the state of a body part different from the body part to which the sensor device 10 is attached in the user. Furthermore, this learning model allows the control unit 26 to determine the state of more body parts than the number of body parts on which the sensor devices 10 are attached.

[0073] The learning model according to this embodiment has been trained to output a score as information evaluating the state of a predetermined body part when feature data is input. The score indicates the evaluation of the state of the predetermined body part. The higher the score, the higher the evaluation of the state of the predetermined body part corresponding to the score. The control unit 26 obtains the score from the learning model and determines the evaluation of the state of the predetermined body part corresponding to the score.

[0074] The following describes an example of feature data that is input into a learning model.

[0075] Feature data may be data that shows statistical values ​​of sensor data. By the feature data being statistical values ​​of sensor data, the feature data can indicate the characteristics of body part movements. For example, feature data may be the maximum value, minimum value, mean value, or variance of sensor data over a predetermined period. The predetermined period may be, for example, the user's walking cycle or a part of the walking cycle. The walking cycle is, for example, the period from when one of the user's two feet lands on the ground until it lands on the ground again. A part of the walking cycle may be, for example, the stance phase or the swing phase. The stance phase is, for example, the period from when one of the user's two feet lands on the ground until it leaves the ground. The swing phase is, for example, the period from when one of the user's two feet leaves the ground until it lands on the ground again. The control unit 26 may detect the user's walking cycle and a part of the walking cycle by analyzing the sensor data. The control unit 26 may obtain feature data from the sensor data by performing calculations on the sensor data.

[0076] The feature data may be sensor data at a predetermined timing. By the feature data being sensor data at a predetermined timing, the feature data can indicate the characteristics of the movement of body parts. For example, the feature data may be sensor data at the time of the user's landing. The landing timing is the time when the user's feet touch the ground. The control unit 26 may detect the landing timing by analyzing the sensor data.

[0077] The feature data may be data in any coordinate system. The feature data may be data in the local coordinate system or data in the global coordinate system. If the feature data is in the global coordinate system, the control unit 26 obtains the feature data in the global coordinate system by performing a coordinate transformation on the sensor data in the local coordinate system.

[0078] An example of a learning model will be explained with reference to Figure 4. The learning model 30 shown in Figure 4 is a neural network learning model. The learning model 30 consists of an input layer 31, a hidden layer 32, a hidden layer 33, and an output layer 34.

[0079] When the learning model 30 receives three feature data from the input layer 31, it outputs one score from the output layer 34.

[0080] The input layer 31 contains three neurons. Feature data is input to each of the three neurons in the input layer 31. The hidden layers 32 and 33 each contain 64 neurons. The output layer 34 contains one neuron. A score is output from the neuron in the output layer 34. In the layers of input layer 31, hidden layer 32, hidden layer 33, and output layer 34, neurons in one of two adjacent layers are connected to neurons in the other layer. During the training of the learning model 30, weight coefficients corresponding to the connection strength between each neuron are adjusted.

[0081] The number of neurons in each of the input layer 31, hidden layer 32, hidden layer 33, and output layer 34 may be adjusted according to the number of feature data used.

[0082] Figure 5 shows an example of a score according to one embodiment of the present disclosure. As described above, the control unit 26 inputs feature data into a learning model to obtain a score as shown in Figure 5. In Figure 5, the control unit 26 obtains scores for (1) the state of the head, (2) the state of the arms, (3) the state of the torso, (4) the state of the knees, and (5) the state of the feet, respectively, using five learning models.

[0083] In Figure 5, the score is a numerical value from 1 to 5. The control unit 26 obtains a score of 5 as an evaluation of (1) the state of the head. The control unit 26 obtains a score of 4 as an evaluation of (2) the state of the arms. The control unit 26 obtains a score of 3 as an evaluation of (3) the state of the trunk. The control unit 26 obtains a score of 1 as an evaluation of (4) the state of the knees. The control unit 26 obtains a score of 2 as an evaluation of (5) the state of the feet.

[0084] [Evaluation signal generation process] When the control unit 26 determines an evaluation of the state of the user's body parts, it may generate an evaluation signal corresponding to the determined evaluation. If the control unit 26 determines multiple evaluations, it may generate an evaluation signal corresponding to at least one of the multiple determined evaluations. If the determined evaluation is higher than the evaluation threshold, the evaluation signal may be a signal indicating praise for the user. If the determined evaluation is lower than the evaluation threshold, the evaluation signal may be a signal indicating advice for the user. The evaluation threshold may be set based on the average value of evaluations from typical users, etc. If a learning model is used, the evaluation threshold may be the average value of scores from typical users. The content of praise and advice for the user may be set based on the interpretation of generally good walking styles as described above.

[0085] In Figure 5, the evaluation threshold is a score of 3. In Figure 5, "Good" indicates that the evaluation is above the evaluation threshold. "Poor" indicates that the evaluation is below the evaluation threshold. "Average" indicates that the evaluation is at the same level as the evaluation threshold.

[0086] In Figure 5, the evaluation for (1) the state of the head and (2) the state of the arms are higher than the evaluation threshold. The control unit 26 generates evaluation signals for each of the states of the head and (2) the arms, which are signals that praise the user. For example, as an evaluation signal for the state of the head, the control unit 26 generates a signal indicating that the user's head movement is small and the state of the head is good. For example, as an evaluation signal for the state of the arms, the control unit 26 generates a signal indicating that the user's arm swing is large and the arm is pulled backward, indicating that the state of the arm is good.

[0087] In Figure 5, the evaluations for (4) the knee condition and (5) the foot condition are lower than the evaluation threshold. The control unit 26 generates an evaluation signal for each of the (4) knee condition and (5) foot condition, which is a signal that provides advice to the user. For example, as an evaluation signal for the (4) knee condition, the control unit 26 generates a signal that provides advice to avoid bending the knee. For example, as an evaluation signal for the (5) foot condition, the control unit 26 generates a signal that provides advice to widen the stride.

[0088] The control unit 26 may transmit the generated evaluation signal to an external device via the communication unit 21. The control unit 26 may transmit the evaluation signal via the communication unit 21 to any sensor device 10 having an output unit 13 as an external device. In this case, the control unit 15 in the sensor device 10 receives the evaluation signal via the communication unit 21. The control unit 15 causes the output unit 13, which acts as a notification unit, to notify the content indicated by the received evaluation signal. As an example of notification, the control unit 15 causes the output unit 13 to output the content indicated by the evaluation signal. With this configuration, the user can understand the evaluation of the state of body parts while walking.

[0089] The control unit 26 may, for example, transmit an evaluation signal to the earphone as an external device via the communication unit 21 if the sensor device 10A is an earphone or is included in an earphone. In this case, the control unit 15 of the sensor device 10A receives the evaluation signal via the communication unit 11. The control unit 15 of the sensor device 10A causes the output unit 13, which acts as a notification unit, to notify the user of the content indicated by the evaluation signal. As an example of notification, the control unit 15 of the sensor device 10A notifies the user by outputting the content indicated by the evaluation signal as sound from the speaker of the output unit 13. With this configuration, the evaluation of the state of a body part can be notified to the user by sound. Notifying the user by sound reduces the possibility of interfering with the user's walking.

[0090] The control unit 26 may inform the user of the content indicated by the generated evaluation signal via the notification unit. As an example of notification, the control unit 26 may output the content indicated by the generated evaluation signal to the output unit 23. As another example of notification, the control unit 26 may vibrate the vibration unit 24 with a vibration pattern corresponding to the determined evaluation.

[0091] [Selection process for the learning model] The control unit 26 may select a learning model from among several learning models to be used in the evaluation decision process described above, depending on the type of sensor device 10 that transmitted sensor data to the electronic device 20. The control unit 26 may select a learning model to be used in the evaluation decision process by referring to the correspondence shown in Figure 6 stored in the memory unit 25.

[0092] The learning model shown in Figure 6 was generated by the learning model generation method described later. The numbers in parentheses shown with the learning model represent the accuracy and confidence of the learning model calculated in the learning model generation method described later. The method for calculating the accuracy and confidence of the learning model will be described later. The number on the left in parentheses is the accuracy of the learning model. The number on the right in parentheses is the confidence of the learning model.

[0093] In Figure 6, a learning model marked with a double circle is a learning model with an accuracy of 90% or higher and a certainty of 70% or higher. A learning model marked with a single circle is a learning model that does not meet the accuracy and certainty conditions of a learning model marked with a double circle. A learning model marked with a single circle is a learning model with an accuracy of 80% or higher and a certainty of 60% or higher. A learning model marked with a triangle is a learning model with an accuracy of less than 80% or a certainty of less than 60%. The control unit 26 may determine the evaluation of some states from (1) the state of the head to (5) the state of the feet, or it may determine the evaluation of all states from (1) the state of the head to (5) the state of the feet, depending on the accuracy and certainty of the learning model.

[0094] In Figure 6, the control unit 26 selects a learning model to be used for the evaluation decision process by selecting Case C1, Case C2, Case C3, Case C4, or Case C5. Cases C1 to C5 associate the learning model used for the evaluation decision process with the type of sensor device 10 used to acquire the feature data to be input to the learning model. However, the learning model is not limited to those shown in Figure 6. A learning model using sensor data or feature data from any combination of sensor devices 10 may be adopted.

[0095] The control unit 26 may select one of cases C1 to C5 depending on the type of sensor device 10 that transmits sensor data to the electronic device 20. For example, the control unit 26 may select any combination of sensor devices 10 from among multiple sensor devices 10 that have transmitted sensor data to the electronic device 20. The control unit 26 may select a case from cases C1 to C5 that corresponds to the selected combination of sensor devices 10. Information on the feature data used in cases C1 to C5, as described below, may be stored in the storage unit 25 in association with cases C1 to C5.

[0096] <Case C1> The control unit 26 may select case C1 if sensor device 10A is the only sensor device 10 that transmits sensor data to the electronic device 20. For example, if sensor device 10A is the only sensor device 10 that a user wears, then sensor device 10A will be the only sensor device 10 that transmits sensor data to the electronic device 20. Alternatively, the control unit 26 may select case C1 if it selects sensor device 10A from among multiple sensor devices 10 that transmit sensor data to the electronic device 20.

[0097] In case C1, when the control unit 26 determines the evaluation of (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, it selects learning models 30A, 30B, 30C, 30D, and 30E, respectively.

[0098] In Case C1, the feature data input to the learning models 30A to 30E includes feature data that shows the characteristics of the user's head movement in the vertical direction of the global coordinate system, and feature data that shows the characteristics of the user's head movement in the horizontal direction of the global coordinate system. However, the feature data input to the learning models 30A to 30E only needs to include feature data that shows the characteristics of head movement in at least one of the forward / backward, left / right, and up / down directions of the global coordinate system.

[0099] In case C1 of this embodiment, the control unit 26 inputs three feature data to each of the learning models 30A to 30E.

[0100] One of the three feature data input to the learning models 30A to 30E corresponds to a feature data representing the characteristics of the user's head movement in the vertical direction of the global coordinate system. In this embodiment, this one feature data is the average value of the user's head angle in the vertical direction of the global coordinate system. This one feature data is obtained from sensor data representing the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0101] Two of the three feature data input to the learning models 30A to 30E correspond to feature data that describes the characteristics of the user's head movement in the left-right direction of the global coordinate system. In this embodiment, these two feature data are the maximum value and the average value of the user's head angle in the left-right direction of the global coordinate system. These two feature data are obtained from sensor data showing the user's head movement detected by the sensor device 10A. The maximum and average values ​​of the user's head angle in the feature data may be the maximum and average values ​​of the head angle during the user's walking cycle, respectively.

[0102] <Case C2> The control unit 26 may select case C2 if the only sensor devices 10 that transmitted sensor data to the electronic device 20 are sensor device 10A and sensor device 10D. Alternatively, the control unit 26 may select case C2 if it selects sensor device 10A and sensor device 10D from among multiple sensor devices 10 that transmitted sensor data to the electronic device 20.

[0103] In case C2, when the control unit 26 determines the evaluation of (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, it selects learning models 30F, 30G, 30H, 30I, and 30J, respectively.

[0104] In Case C2, the feature data input to the learning models 30F-30J includes, in the same or similar manner as in Case C1, feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system, and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. Furthermore, the feature data input to the learning models 30F-30J also includes feature data indicating the characteristics of the user's foot movement.

[0105] In case C2 of this embodiment, the control unit 26 inputs three feature data to each of the learning models 30F to 30J.

[0106] Two of the three feature data input to the learning models 30F to 30J correspond to feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. In this embodiment, these two feature data are the average value of the user's head angle in the vertical direction of the global coordinate system and the maximum value of the user's head angle in the horizontal direction of the global coordinate system. These two feature data are obtained from data indicating the user's head movement detected by the sensor device 10A. The average and maximum values ​​of the user's head angle in the feature data may be the average and maximum values ​​of the head angle during the user's walking cycle, respectively.

[0107] One of the three feature data input to the learning models 30F to 30J corresponds to the feature data representing the characteristics of the user's foot movement. In this embodiment, this one feature data is the maximum value of the user's foot acceleration in the vertical direction of the local coordinate system based on the position of the sensor device 10D. This one feature data is obtained from sensor data representing the user's foot movement detected by the sensor device 10D. The maximum value of the user's foot acceleration in the feature data may be the maximum value of the foot acceleration during the user's walking cycle.

[0108] <Case C3> The control unit 26 may select case C3 if the only sensor devices 10 that transmitted sensor data to the electronic device 20 are sensor devices 10A and 10C. Alternatively, the control unit 26 may select case C3 if it selects sensor devices 10A and 10C from among multiple sensor devices 10 that transmitted sensor data to the electronic device 20.

[0109] In case C3, when the control unit 26 determines the evaluation of (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, it selects learning models 30K, 30L, 30M, 30N, and 30O, respectively.

[0110] In Case C3, the feature data input to the learning models 30K-30O includes, in the same or similar manner as in Case C1, feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system, and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. Furthermore, the feature data input to the learning models 30K-30O also includes feature data indicating the characteristics of the user's thigh movement.

[0111] In case C3 of this embodiment, the control unit 26 inputs three feature data to each of the learning models 30K to 30O.

[0112] Two of the three feature data input to the learning models 30K~30O correspond to feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. In this embodiment, these two feature data are the average values ​​of the user's head angle in the vertical direction of the global coordinate system and the average values ​​of the user's head angle in the horizontal direction of the global coordinate system. These two feature data are obtained from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0113] One of the three feature data input to the learning models 30K-30O is a feature data that describes the movement characteristics of the user's thigh. In this embodiment, this feature data is the variance of the angular velocity of the user's thigh during the stance phase in the left-right direction of the local coordinate system with respect to the position of the sensor device 10C. This feature data is obtained from sensor data showing the movement of the user's thigh detected by the sensor device 10D.

[0114] <Case C4> The control unit 26 may select case C4 if the only sensor devices 10 that transmitted sensor data to the electronic device 20 are sensor devices 10A and 10B. Alternatively, the control unit 26 may select case C4 if it selects sensor devices 10A and 10B from among multiple sensor devices 10 that transmitted sensor data to the electronic device 20.

[0115] In case C4, when the control unit 26 determines the evaluation of (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, it selects learning models 30P, 30Q, 30R, 30S, and 30T, respectively.

[0116] In case C4, the feature data input to the learning models 30P to 30T includes, in the same or similar manner as in case C1, feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system, and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. Furthermore, the feature data input to the learning models 30P to 30T includes feature data indicating the characteristics of the user's forearm movement.

[0117] In case C4 of this embodiment, the control unit 26 inputs three feature data to each of the learning models 30P to 30T.

[0118] Two of the three feature data input to the learning models 30P to 30T correspond to feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. In this embodiment, these two feature data are the average value of the user's head angle in the vertical direction of the global coordinate system and the angle of the user's head at the landing timing in the horizontal direction of the global coordinate system. These two feature data are obtained from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0119] One of the three feature data input to the learning models 30P to 30T corresponds to the feature data representing the movement characteristics of the user's forearm. In this embodiment, this one feature data is the variance of the user's forearm acceleration in the anterior-posterior direction of the local coordinate system with respect to the position of the sensor device 10B. This one feature data is obtained from sensor data representing the movement of the user's forearm detected by the sensor device 10B. The variance of the user's forearm acceleration in the feature data may be the variance of the forearm acceleration during the user's walking cycle.

[0120] <Case C5> The control unit 26 may select case C5 if the sensor devices 10 that transmitted sensor data to the electronic device 20 are only sensor devices 10A, 10B, and 10D. Alternatively, the control unit 26 may select case C5 if it selects sensor devices 10A, 10B, and 10D from among multiple sensor devices 10 that transmitted sensor data to the electronic device 20.

[0121] In case C5, when the control unit 26 determines the evaluation of (1) the state of the head, (2) the state of the arms, (3) the state of the torso, (4) the state of the knees, and (5) the state of the feet, it selects learning models 30U, 30V, 30W, 30X, and 30Y, respectively.

[0122] In case C5, the feature data input to the learning models 30U~30Y is the same as or similar to that in case C1, including feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. Furthermore, the feature data input to the learning models 30U~30Y includes feature data indicating the characteristics of the user's forearm movement and feature data indicating the characteristics of the thigh movement.

[0123] In case C5 of this embodiment, the control unit 26 inputs four feature data to each of the learning models 30U to 30Y.

[0124] Two of the four feature data input to the learning models 30U~30Y correspond to feature data indicating the characteristics of the user's head movement in the vertical direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the horizontal direction of the global coordinate system. In this embodiment, these two feature data are the average value of the user's head angle in the vertical direction of the global coordinate system and the angle of the user's head at the landing timing in the horizontal direction of the global coordinate system. These two feature data are obtained from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0125] One of the four feature data input to the learning models 30U~30Y corresponds to a feature data representing the movement characteristics of the user's forearm. In this embodiment, this one feature data is the maximum value of the user's forearm acceleration in the anterior-posterior direction of the local coordinate system based on the position of the sensor device 10B. This one feature data is obtained from sensor data representing the movement of the user's forearm detected by the sensor device 10B. The maximum value of the user's forearm acceleration in the feature data may be the maximum value of the forearm acceleration during the user's walking cycle.

[0126] One of the four feature data input to the learning models 30U~30Y corresponds to a feature data representing the movement characteristics of the user's thigh. In this embodiment, this one feature data is the maximum value of the user's foot acceleration in the vertical direction of the local coordinate system with respect to the position of the sensor device 10D. This one feature data is obtained from sensor data representing the movement of the user's thigh detected by the sensor device 10D. The maximum value of the user's foot acceleration in the feature data may be the maximum value of the foot acceleration during the user's walking cycle.

[0127] [Method for generating a learning model] The following describes the method for generating the learning model. A gait database of the subjects was used to generate the learning model. The gait database of the subjects was "Yoshiyuki Kobayashi, Naoto Hida, Kanako Nakajima, Masahiro Fujimoto, Masaaki Mochimaru, "2019: AIST Gait Database 2019", [Online], [Accessed May 24, 2021], Internet<https: / / unit.aist.go.jp / harc / ExPART / GDB2019_e.html> The data provided in "[the relevant document]" was used. This gait database contains gait data from multiple subjects. The subjects' gait data was detected by a motion capture system and a force plate.

[0128] The gait of subjects in the gait database was evaluated by a gait instructor. The instructor evaluated the subject's gait by assigning a numerical score from 1 to 5 to the following aspects of the subject's walking: (1) head position, (2) arm position, (3) trunk position, (4) knee position, and (5) foot position. The instructor evaluated the subject's gait based on the interpretation of generally good walking as described above.

[0129] Feature data was obtained from data showing the movements of subjects detected by a motion capture system. A dataset was generated by associating this feature data with scores assigned by instructors. 980 datasets were generated using walking data from 98 subjects (10 steps each). A learning model was generated using cross-validation with this dataset. In the cross-validation process, 800 datasets (80% of the 980) were allocated to the training dataset for the learning model. 180 datasets (20% of the 980) were allocated to the evaluation dataset for the learning model. The 980 datasets were divided into 80% training datasets and 20% evaluation datasets using 10 different methods. The total number of trials was 846,000.

[0130] The accuracy and confidence of the generated learning model were calculated. The accuracy and confidence of the learning model were calculated based on the number of correct and incorrect estimation results of the learning model. In determining whether the learning model's estimation result was correct or incorrect, the score was divided into three levels. Specifically, the score was divided into three levels: a score greater than 3, a score of 3, and a score less than 3. A score greater than 3 was also written as "Good". A score of 3 was also written as "Average". A score less than 3 was also written as "Poor". If the learning model's score of "Good" or "Poor" matched the score of "Good" or "Poor" assigned by the instructor, the learning model's estimation result was determined to be correct. On the other hand, if the learning model's score of "Good" or "Poor" did not match the score of "Good" or "Poor" assigned by the instructor, the learning model's estimation result was determined to be incorrect. If the learning model's score was 3, the learning model's estimation result was determined to be neither correct nor incorrect.

[0131] The accuracy of the learning model was calculated by dividing the number of correct estimations by the sum of the number of correct estimations and the number of incorrect estimations. For example, the accuracy of the learning model was calculated using equation (1). Accuracy of the learning model = (CR) / (CR+ICR) Equation (1) In equation (1), CR is the number of correct estimation results, and ICR is the number of incorrect estimation results.

[0132] The accuracy of the learning model was calculated by dividing the number of correct estimation results by the total number of estimation results. For example, the precision of the learning model was calculated using equation (2). Accuracy of the learning model = (CR) / (CR+ICR+NR) Equation (2) In equation (2), CR is the number of correct estimation results. ICR is the number of incorrect estimation results. NR is the number of learning model estimation results that are neither correct nor incorrect. In other words, NR is the number of learning model scores that are 3.

[0133] Here, the inventors set the number of feature data inputs to the learning model to a minimum of three, and while changing the combination of feature data inputs to the learning model, they searched for a combination of feature data that would improve the accuracy and confidence evaluation of the learning model. The inventors obtained the results shown in Figures 7 and 8.

[0134] Figure 7 shows a graph illustrating the accuracy of the learning model. Figure 8 shows a graph illustrating the accuracy of the learning model. Cases C1 to C5 shown in Figures 7 and 8 are the same as the aforementioned cases C1 to C5, referring to Figure 6. Figures 7 and 8 show the accuracy and accuracy of the learning model for each of the following states, from (1) the state of the head to (5) the state of the feet.

[0135] As shown in Figure 7, in Case C1, the accuracy of the learning model for each of the states from (1) head to (5) feet was highest for (1) head. As shown in Figure 8, in Case C1, the accuracy of the learning model for each of the states from (1) head to (5) feet was highest for (1) head. In Case C1, the feature data input to the learning model is feature data that shows the characteristics of the subject's (user's) head movements. In Case C1, it is thought that the use of feature data that shows the characteristics of the user's head movements resulted in the highest accuracy and accuracy of the learning model for each of the states from (1) head to (5) feet being highest for (1) head.

[0136] As shown in Figure 7, in case C1, the accuracy of the learning model for each of the body parts other than the user's head, from (2) the state of the arms to (5) the state of the feet, was 80% or higher. As shown in Figure 8, in case C1, the accuracy of the learning model for each of the body parts other than the user's head, from (2) the state of the arms to (5) the state of the feet, was 50% or higher. These results show that feature data representing the characteristics of the user's head movements can be used to evaluate the state of body parts other than the user's head with a certain degree of accuracy.

[0137] Here, the movement of the user's head in the vertical and horizontal directions of the global coordinate system reflects the movement of different body parts than the user's head while walking. For example, when a user swings their arms or kicks their feet while walking, the user's body moves in the vertical and horizontal directions of the global coordinate system. When the user's body moves in the vertical and horizontal directions of the global coordinate system, the user's head also moves in the vertical and horizontal directions of the global coordinate system.

[0138] Thus, the movement of the user's head in the vertical and horizontal directions of the global coordinate system reflects the movement of body parts different from the user's head while walking. Therefore, it is inferred that by using feature data that shows the characteristics of the user's head movement in the vertical and horizontal directions of the global coordinate system as the feature data to be input into the learning model, it is possible to evaluate the state of body parts different from the user's head while walking. As described above, in case C1, feature data that shows the characteristics of the user's head movement in the vertical and horizontal directions of the global coordinate system is used as the feature data to be input into the learning model. With this configuration, it is inferred that in case C1, the state of body parts different from the user's head could be evaluated with a certain degree of accuracy.

[0139] As shown in Figure 7, in case C2, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. Also, in case C2, the accuracy of the learning model for the state of the feet was higher than in cases C1, C3, and C4. As shown in Figure 8, in case C2, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. Also, in case C2, the accuracy of the learning model for the state of the feet was higher than in cases C1 and C3.

[0140] In Case C2, the feature data used to input the learning model represents the movement characteristics of more body parts of the user than in Case C1. For example, in Case C2, the feature data used is the same or similar as in Case C1, representing the movement characteristics of the user's head in the vertical and horizontal directions of the global coordinate system. In addition to this feature data, feature data representing the movement characteristics of the user's feet is also used in Case C2. Because Case C2 uses feature data representing the movement characteristics of more body parts of the user than in Case C1, it is inferred that the accuracy and confidence of the learning model for each of the states from (1) the head to (5) the feet was higher than in Case C1. Furthermore, because Case C2 uses feature data representing the movement characteristics of the user's feet, it is inferred that the accuracy and confidence of the learning model for the state of (5) the feet was higher than in Case C1, etc.

[0141] As shown in Figure 7, in case C3, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. Also, in case C3, the accuracy of the learning model for the state of (4) the thigh was higher than in cases C1, C2, and C4. As shown in Figure 8, in case C3, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. Also, in case C3, the accuracy of the learning model for the state of (4) the thigh was higher than in cases C1, C2, and C4.

[0142] In Case C3, the feature data used to input the learning model includes feature data that shows the movement of more body parts of the user than in Case C1. For example, in Case C3, the feature data used is the same or similar as in Case C1, showing the characteristics of the user's head movement in the vertical and horizontal directions of the global coordinate system. In addition to this feature data, in Case C3, feature data showing the characteristics of the user's thigh movement is used. In Case C3, since feature data showing the characteristics of the movement of more body parts of the user is used than in Case C1, it is inferred that the accuracy and confidence of the learning model for each of the states from (1) the head to (5) the feet was higher than in Case C1. Furthermore, in Case C3, since feature data showing the characteristics of the user's thigh movement is used, it is inferred that the accuracy and confidence of the learning model for the state of (4) the thigh was higher than in Case C1, etc.

[0143] As shown in Figure 7, in case C4, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. Also, in case C4, the accuracy of the learning model for (2) the arm state was higher than in cases C1, C2, and C3. As shown in Figure 8, in case C4, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. Also, in case C4, the accuracy of the learning model for (2) the arm state was higher than in cases C1, C2, and C3.

[0144] In Case C4, the feature data used to input the learning model includes feature data that shows the movement of more body parts of the user than in Case C1. For example, in Case C4, the feature data used is the same or similar as in Case C1, showing the characteristics of the user's head movement in the vertical and horizontal directions of the global coordinate system. In addition to this feature data, in Case C4, feature data showing the characteristics of the user's forearm movement is used. In Case C4, since feature data showing the characteristics of the movement of more body parts of the user is used than in Case C1, it is inferred that the accuracy and confidence of the learning model for each of the states from (1) the head to (5) the feet was higher than in Case C1. Furthermore, in Case C4, since feature data showing the characteristics of the user's forearm movement is used, it is inferred that the accuracy and confidence of the learning model for the state of (2) the arms was higher than in Case C1, etc.

[0145] As shown in Figure 7, in case C5, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1. As shown in Figure 8, in case C5, the accuracy of the learning model for each of the states from (1) the head to (5) the feet was higher than in case C1.

[0146] In Case C5, the feature data used to input the learning model represents the characteristics of more body parts of the user than in Cases C1 through C4. Because Case C5 uses feature data representing the characteristics of more body parts than Cases C1 through C4, it is inferred that the accuracy and confidence of the learning model for each of the body parts, from (1) the state of the head to (5) the state of the feet, is higher than in Case C1.

[0147] (System operation) Figure 9 is a flowchart showing the operation of the evaluation process performed by the electronic device 20 shown in Figure 1. This operation corresponds to an example of the information processing method according to this embodiment. When the control unit 26 receives an input from the input unit 22 instructing it to perform a gait evaluation, for example, it starts the evaluation process from the process in step S10.

[0148] The control unit 26 receives an input from the input unit 22 instructing it to perform a gait evaluation (step S10). This input is provided by the user wearing the sensor device 10 through the input unit 22.

[0149] The control unit 26 transmits a signal to start data detection as a broadcast signal to multiple sensor devices 10 via the communication unit 21 (step S11). After the processing in step S11 is performed, sensor data is transmitted from at least one sensor device 10 to the electronic device 20.

[0150] The control unit 26 receives sensor data from at least one sensor device 10 via the communication unit 21 (step S12).

[0151] The control unit 26 refers to a correspondence, for example, as shown in Figure 6, and selects a learning model from among several learning models that corresponds to the type of sensor device 10 that transmitted sensor data to the electronic device 20 (step S13).

[0152] The control unit 26 acquires feature data from the sensor data received in step S12 (step S14). The control unit 26 inputs the feature data acquired in step S14 into the learning model selected in step S13, thereby acquiring a score, for example, as shown in Figure 5 (step S15). By acquiring the score, the control unit 26 determines an evaluation of the state of the body part corresponding to the score.

[0153] The control unit 26 generates an evaluation signal corresponding to the determined evaluation (step S16). The control unit 26 transmits the evaluation signal generated in step S16 to an external device via the communication unit 21 (step S17). After executing the process in step S17, the control unit 26 terminates the evaluation process.

[0154] After completing the evaluation process, the control unit 26 may execute the evaluation process again when the user walks a set number of steps. This set number of steps may be pre-entered by the user from the input unit 22. In the evaluation process executed again, the control unit 26 may start from the process of step S11. The control unit 26 may repeatedly execute the evaluation process each time the user walks a set number of steps until it receives an input from the input unit 22 instructing it to terminate the evaluation process. The input instructing it to terminate the evaluation process may be, for example, entered by the user from the input unit 22. For example, when the user finishes walking, it enters an input from the input unit 22 instructing it to terminate the evaluation process.

[0155] In this electronic device 20, which functions as an information processing device, the control unit 26 can estimate the state of a body part of the user that is different from the body part to which the sensor device 10 is attached, using a learning model. As described above, for example, the control unit 26 can estimate the state of a body part of the user that is different from the body part to which the sensor device 10 is attached, and determine an evaluation for that body part's state. For example, the control unit 26 can select case C1 as shown in Figure 6 and determine an evaluation for each of the body parts of the user, from (2) the state of the arms to (5) the state of the feet, which are different from the head to which the sensor device 10A is attached. With this configuration, in this embodiment, the state of any body part can be estimated, not limited to the body part of the user to which the sensor device 10 is attached.

[0156] In recent years, walking has gained attention as an easy form of exercise. However, users are required to pay attention to obstacles in front of or near them while walking. Because users are required to pay attention to obstacles in front of or near them while walking, they may not be able to pay attention to their own posture. If users are unable to pay attention to their own posture while walking, they may walk with incorrect posture without realizing it. If users walk with incorrect posture, the exercise effect of walking may be reduced. Also, since walking is often a familiar form of exercise for users, it is often difficult for them to correct their posture while walking on their own.

[0157] In the electronic device 20 according to this embodiment, as described above, the control unit 26 can estimate the state of the user's body parts. This configuration allows the user to have the opportunity to correct their posture while walking. By giving the user the opportunity to correct their posture while walking, the user can walk with the correct posture. By enabling the user to walk with the correct posture, the exercise effect of walking can be enhanced.

[0158] Therefore, according to this embodiment, a novel technique is provided for estimating the state of a user's body parts while they are walking.

[0159] Furthermore, the control unit 26 may use a learning model to estimate the state of more body parts than the number of body parts on which the sensor devices 10 are attached to the user. For example, suppose the user is fitted with N sensor devices 10 (where N is an integer greater than or equal to 1). In this case, the control unit 26 may acquire N sensor data from the N sensor devices 10 and determine an evaluation for the state of N+1 or more body parts of the user.

[0160] As an example, let's assume that N is 1 and the user is fitted with a sensor device 10A. In this case, the control unit 26 acquires one sensor data from the sensor device 10A. The control unit 26 selects case C1 as shown in Figure 6 and, based on the acquired sensor data, determines an evaluation of the state of two or more body parts, for example, the state of five body parts from (1) the state of the head to (5) the state of the feet.

[0161] As another example, suppose N is 2 and the user is fitted with sensor device 10A and sensor device 10D. In this case, the control unit 26 acquires two sensor data from each of sensor device 10A and sensor device 10D. The control unit 26 selects case C2 as shown in Figure 6 and, based on the two acquired sensor data, determines an evaluation of the state of three or more body parts, for example, the state of five body parts from (1) the state of the head to (5) the state of the feet.

[0162] By estimating the state of more body parts than the number of body parts to which the sensor device 10 is attached, the electronic device 20 becomes more user-friendly.

[0163] Furthermore, at least one sensor device 10 included in the information processing system 1 may include a sensor device 10A that is attached to the user's head. The control unit 26 may acquire sensor data from the sensor device 10A indicating the user's head movements. In this case, the control unit 26 may select case C1 as shown in Figure 6. That is, the feature data may include feature data indicating the characteristics of the user's head movements in the vertical direction, i.e., the vertical direction of the global coordinate system, and feature data indicating the characteristics of the user's head movements in the horizontal direction, i.e., the horizontal direction of the global coordinate system. With such a configuration, even if the sensor device 10 worn by the user is only the sensor device 10A, it is possible to determine the evaluation of the state of the user's body parts. In other words, the user only needs to wear the sensor device 10A. Therefore, user convenience can be improved. Furthermore, if the sensor device 10A is an earphone or is included in an earphone, the user can easily attach the sensor device 10A to their head. The ability for the user to easily attach the sensor device 10A to their head can further improve user convenience. Furthermore, by using only the sensor data detected by sensor device 10A, it becomes unnecessary to synchronize the timing of data detection by each of the multiple sensor devices 10. By eliminating the need to synchronize the timing of data detection by each of the multiple sensor devices 10, it becomes possible to more easily determine the evaluation of the state of the user's body parts.

[0164] Furthermore, the information processing system 1 may include at least one sensor device 10, which is attached to the user's head, and a sensor device 10D, which is attached to the user's feet. The control unit 26 may acquire sensor data indicating the movement of the user's head from sensor device 10A and sensor data indicating the movement of the user's feet from sensor device 10D. In this case, the control unit 26 may select case C2 as shown in Figure 6. That is, the feature data may include feature data indicating the characteristics of head movement in the vertical direction of the global coordinate system, feature data indicating the characteristics of head movement in the horizontal direction of the global coordinate system, and feature data indicating the characteristics of foot movement. As described above with reference to Figures 7 and 8, in case C2, the accuracy and precision of the learning model for each of (1) the state of the head to (5) the state of the feet is higher than in case C1. Therefore, the control unit 26 can determine the evaluation of the state of the user's body parts with greater accuracy. Also, if sensor device 10A is an earphone or is included in an earphone, the user can easily attach sensor device 10A to their head. Furthermore, if the sensor device 10D is a shoe-shaped wearable device, the user can easily attach the sensor device 10D to their foot. Therefore, user convenience can be improved.

[0165] Furthermore, the information processing system 1 may include at least one sensor device 10, which is attached to the user's head, and a sensor device 10C, which is attached to the user's thigh. The control unit 26 may acquire sensor data indicating the user's head movement from sensor device 10A and data indicating the user's thigh movement from sensor device 10D. In this case, the control unit 26 may select case C3 as shown in Figure 6. That is, the feature data may include feature data indicating the characteristics of head movement in the vertical direction of the global coordinate system, feature data indicating the characteristics of head movement in the horizontal direction of the global coordinate system, and feature data indicating the characteristics of thigh movement. As described above with reference to Figures 7 and 8, in case C3, the accuracy and precision for each of (1) the state of the head to (5) the state of the feet was higher than in case C1. Therefore, the control unit 26 can determine the state of the user's body parts with greater accuracy. Also, if sensor device 10A is an earphone or is included in an earphone, the user can easily attach sensor device 10A to their head. This configuration can improve user convenience.

[0166] Furthermore, the information processing system 1 may include at least one sensor device 10, which is attached to the user's head, and a sensor device 10B, which is attached to the user's forearm. The control unit 26 may acquire sensor data indicating the user's head movement from sensor device 10A and sensor data indicating the user's forearm movement from sensor device 10B. In this case, the control unit 26 may select case C4 as shown in Figure 6. That is, the feature data may include feature data indicating the characteristics of head movement in the vertical direction of the global coordinate system, feature data indicating the characteristics of head movement in the horizontal direction of the global coordinate system, and feature data indicating the characteristics of forearm movement. As described above with reference to Figures 7 and 8, in case C4, the accuracy and precision for each of (1) the state of the head to (5) the state of the feet were high. Therefore, the control unit 26 can determine the evaluation of the state of the user's body parts with greater accuracy. Also, if sensor device 10A is an earphone or is included in an earphone, the user can easily attach sensor device 10A to their head. Furthermore, if the sensor device 10B is a wristwatch-type wearable device, the user can easily attach the sensor device 10B to their forearm. This configuration improves user convenience.

[0167] Furthermore, the information processing system 1 may include at least one sensor device 10, which is attached to the user's head, a sensor device 10B attached to the user's forearm, and a sensor device 10D attached to the user's foot. The control unit 26 may acquire sensor data indicating the movement of the user's head from sensor device 10A, sensor data indicating the movement of the user's forearm from sensor device 10B, and sensor data indicating the movement of the user's foot from sensor device 10D. In this case, the control unit 26 may select case C5 as shown in Figure 6. That is, the feature data may include feature data indicating the characteristics of head movement in the vertical direction of the global coordinate system, and feature data indicating the characteristics of head movement in the horizontal direction of the global coordinate system. Furthermore, the feature data may include feature data indicating the characteristics of forearm movement and feature data indicating the characteristics of thigh movement. As described above with reference to Figures 7 and 8, in case C5, the accuracy and precision for each of (1) the state of the head to (5) the state of the foot are higher than in case C1. Therefore, the control unit 26 can determine the evaluation of the user's body parts with greater accuracy.

[0168] (Configuration of other systems) Figure 10 is a functional block diagram showing the configuration of an information processing system 101 according to another embodiment of the present disclosure.

[0169] The information processing system 101 includes a sensor device 10, an electronic device 20, and a server 40. In the information processing system 101, the server 40 functions as an information processing device and estimates the state of the user's body parts.

[0170] The electronic device 20 and the server 40 can communicate with each other via network 2. Network 2 may be any network, including mobile communication networks and the Internet.

[0171] The control unit 26 of the electronic device 20 receives sensor data from the sensor device 10 via the communication unit 21, in the same or similar manner as the information processing system 1. In the information processing system 101, the control unit 26 transmits the sensor data to the server 40 via the network 2 using the communication unit 21.

[0172] Server 40 is, for example, a server belonging to a cloud computing system or other computing system. Server 40 has a communication unit 41, a storage unit 42, and a control unit 43.

[0173] The communication unit 41 is configured to include at least one communication module that can connect to the network 2. The communication module is, for example, a communication module that complies with standards such as wired LAN (Local Area Network) or wireless LAN. The communication unit 41 is connected to the network 2 via the communication module through the wired LAN or wireless LAN.

[0174] The storage unit 42 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or at least two combinations thereof. The semiconductor memory is, for example, RAM or ROM. The RAM is, for example, SRAM or DRAM. The ROM is, for example, EEPROM. The storage unit 42 may function as a main memory, auxiliary memory, or cache memory. The storage unit 42 stores data used for the operation of the server 40 and data obtained by the operation of the server 40. For example, the storage unit 42 stores system programs, application programs, and embedded software. For example, the storage unit 42 stores learning models and correspondences as shown in Figure 6.

[0175] The control unit 43 is configured to include at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA or ASIC. The control unit 43 controls each part of the server 40 and executes processes related to the operation of the server 40.

[0176] The control unit 43 receives sensor data from the electronic device 20 via the network 2 using the communication unit 41. The control unit 43 estimates the state of the user's body parts based on the sensor data by performing the same or similar processing as the control unit 26 of the electronic device 20 described above.

[0177] (Operation of other systems) Figure 11 is a sequence diagram showing the operation of the evaluation process performed by the information processing system 101 shown in Figure 10. This operation corresponds to an example of the information processing method according to this embodiment. When the electronic device 20 receives an input instructing it to perform a gait evaluation, the information processing system 101 starts the evaluation process from step S20. Hereinafter, it is assumed that the learning model has been trained to output a score when feature data is input.

[0178] In the electronic device 20, the control unit 26 receives an input from the input unit 22 instructing it to perform a gait evaluation (step S20). The control unit 26 transmits a signal instructing the start of data detection as a broadcast signal to the multiple sensor devices 10 via the communication unit 21 (step S21).

[0179] In the sensor device 10, the control unit 15 receives a signal from the electronic device 20 via the communication unit 11 to instruct the start of data detection (step S22). Upon receiving this signal, the control unit 15 starts data detection. The control unit 15 acquires the data detected by the sensor unit 12 from the sensor unit 12 and transmits the acquired data as sensor data to the electronic device 20 via the communication unit 11 (step S23).

[0180] In the electronic device 20, the control unit 26 receives sensor data from the sensor device 10 via the communication unit 21 (step S24). The control unit 26 transmits the sensor data to the server 40 via the network 2 via the communication unit 21 (step S25).

[0181] In the server 40, the control unit 43 receives sensor data from the electronic device 20 via the network 2 using the communication unit 41 (step S26). The control unit 43 selects a learning model from among several learning models that corresponds to the type of sensor device 10 that transmitted sensor data to the server 40 via the electronic device 20 (step S27). The control unit 43 acquires feature data from the sensor data received in step S26 (step S28). The control unit 43 inputs the feature data acquired in step S28 into the learning model selected in step S27, thereby obtaining a score from the learning model (step S29). By obtaining the score, the control unit 43 determines an evaluation of the state of the part corresponding to the score.

[0182] In server 40, the control unit 43 generates an evaluation signal corresponding to the determined evaluation (step S30). The control unit 43 transmits the evaluation signal generated in step S30 to the electronic device 20, which is an external device, via the network 2 using the communication unit 41 (step S31).

[0183] In the electronic device 20, the control unit 26 receives an evaluation signal from the server 40 via the network 2 using the communication unit 21 (step S32). The control unit 26 causes the notification unit to notify the contents of the evaluation signal (step S33). As an example of notification, the control unit 26 may output the contents of the evaluation signal to the output unit 23. As another example of notification, the control unit 26 may vibrate the vibration unit 24 with a vibration pattern corresponding to the evaluation signal. As yet another example of notification, the control unit 26 may transmit the evaluation signal to the sensor device 10 using the communication unit 21, causing the sensor device 10 to notify the contents of the evaluation signal. In this case, the control unit 15 in the sensor device 10 may output the contents of the evaluation signal to the output unit 13, which acts as a notification unit. When the sensor device 10A receives the evaluation signal, the control unit 15 in the sensor device 10A may output the contents of the evaluation signal as sound from the speaker of the output unit 13.

[0184] After executing the process in step S33, the information processing system 101 terminates the evaluation process.

[0185] After completing the evaluation process, the information processing system 101 may perform the evaluation process again when the user walks the specified number of steps. In the evaluation process performed again, the information processing system 101 may start from the process in step S23. The information processing system 101 may repeatedly perform the evaluation process each time the user walks the specified number of steps until the electronic device 20 receives an input from the input unit 22 indicating the termination of the evaluation process.

[0186] Information processing system 101 can achieve the same or similar effects as information processing system 1.

[0187] While embodiments relating to this disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on this disclosure. Therefore, it should be noted that these modifications or alterations are within the scope of this disclosure. For example, the functions included in each component can be rearranged in a logically consistent manner, and multiple components can be combined into one or separated.

[0188] For example, the learning model may be trained to output information about the state of a body part of the user that is different from the body part to which the sensor device 10 is attached, when sensor data or feature data is input. In this case, the control unit 26 estimates the state of a body part of the user that is different from the body part to which the sensor device 10 is attached, based on the sensor data and the learning model.

[0189] For example, the feature data input to the learning model is not limited to those described above. As can be seen from the results shown in Figures 7 and 8, for example, the more body parts to which the sensor devices 10 are attached to the user, the greater the variety of feature data indicating the characteristics of the body part's movement, and the higher the accuracy and reliability of the learning model. Therefore, in order to improve the accuracy and reliability of the learning model, the number of feature data input to the learning model may be increased by increasing the number of sensor devices 10 attached to the user's body parts.

[0190] For example, the electronic device 20 may be a glasses-type wearable device. In this case, the output unit 23 may be configured to include a projector that projects an image onto the lenses of the glasses. The control unit 26 may output an evaluation of the state of the determined body part as an image to the output unit 23. This image may include, for example, an image showing the ideal movement of a body part that has a low evaluation among several of the user's body parts.

[0191] For example, evaluation thresholds may be set based on the user's age and gender. In this case, the content of praise and advice given to the user may be set according to the user's age and gender.

[0192] For example, the memory unit 25 of the electronic device 20 may store a learning model for each piece of physical data that can distinguish the physical characteristics of multiple users. The physical data includes, for example, at least one of the following: age, gender, height, and weight. In this case, the control unit 26 may receive input indicating the user's physical data from the input unit 22. The control unit 26 may select a learning model from among the multiple learning models stored in the memory unit 25 that corresponds to the received user's physical data. With such a configuration, it is possible to evaluate the state of body parts that match an individual's physical data.

[0193] For example, the storage unit 42 of the server 40 may store a learning model for each piece of physical data. In this case, the control unit 26 of the electronic device 20 may receive an input indicating the user's physical data from the input unit 22. When the control unit 26 of the electronic device 20 receives an input indicating the user's physical data from the input unit 22, it may transmit a signal indicating the user's physical data to the server 40 via the network 2 using the communication unit 21. In the server 40, the control unit 43 receives the signal indicating the user's physical data from the electronic device 20 via the network 2 using the communication unit 41. In the server 40, the control unit 43 may select a learning model corresponding to the user's physical data from among a plurality of learning models stored in the storage unit 42 based on the received signal indicating the user's physical data. With this configuration, it is possible to evaluate the state of body parts that match the individual's physical data.

[0194] For example, the control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may calculate the rhythm of the user's body movement, the user's stride length, and the user's walking speed based on sensor data. The control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may calculate the walking time of the user based on sensor data. The control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may calculate the walking distance of the user based on sensor data. If the walking time exceeds a time threshold or the walking distance exceeds a distance threshold, the control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may generate a signal to prompt a break or a signal to prompt the end of walking. In the electronic device 20, the control unit 26 may transmit the generated signal to an external device as described above via the communication unit 21. In the server 40, the control unit 43 may transmit the generated signal to the electronic device 20 or sensor device 10 via the network 2 via the communication unit 41. The time threshold may be set based on the average value of a typical user's walking time, etc. The distance threshold may be set based on the average distance a typical user walks at one time, or similar criteria.

[0195] For example, the communication unit 21 of the electronic device 20 may be configured to include at least one receiving module corresponding to a satellite positioning system. The receiving module is, for example, a receiving module corresponding to GPS (Global Positioning System). However, the receiving module is not limited to this. The receiving module may be a receiving module corresponding to any satellite positioning system. In this case, the storage unit 25 may store map data. The control unit 26 may also acquire the user's location information via the communication unit 21. The control unit 26 may output the user's location information and map data to the output unit 23.

[0196] For example, the communication unit 11 of the sensor device 10 may further include at least one communication module that can be connected to a network 2 as shown in Figure 10. The communication module is, for example, a communication module that supports a mobile communication standard such as LTE, 4G, or 5G. In this case, in the information processing system 101 as shown in Figure 10, the control unit 15 of the sensor device 10 may transmit the data detected by the sensor device 10 to the server 40 via the network 2 using the communication unit 11.

[0197] For example, in the embodiment described above, the control unit 26 of the electronic device 20 may estimate the overall state of the user by combining two or more body parts. However, the control unit 26 of the electronic device 20 or the control unit 43 of the server may determine an overall evaluation by combining two or more of the above-mentioned (1) head state, (2) arm state, (3) torso state, (4) knee state, and (5) foot state.

[0198] For example, an embodiment is also possible in which a general-purpose computer functions as the electronic device 20 according to this embodiment. Specifically, a program describing the processing content that realizes each function of the electronic device 20 according to this embodiment is stored in the memory of the general-purpose computer, and the processor reads and executes the program. Therefore, the configuration according to this embodiment can also be realized as a program that can be executed by a processor or as a non-temporary computer-readable medium that stores said program. [Explanation of Symbols]

[0199] 1,101 Information Processing Systems 2 Network 10, 10A, 10B, 10C, 10D Sensor Equipment 11 Communications Department 12 Sensor section 13 Output section 14 Storage section 15 Control Unit 20 Electronic equipment 21 Communications Department 22 Input section 23 Output section 24 Vibration section 25 Memory section 26 Control Unit 30, 30A~30E, 30F~30J, 30K~30O, 30P~30T, 30U~30Y Learning Model 31 Input Layer 32 Hidden Layers 33 Hidden Layers 34 Output Layers 40 servers 41 Communications Department 42 Storage section 43 Control Unit

Claims

1. Sensor data indicating the movement of the body part is acquired from at least one sensor device attached to the user's body part. The system includes a control unit that estimates the state of a body part of the user that is different from the body part to which the sensor device is attached, based on the acquired sensor data and a learning model. When the learning model receives the sensor data, it outputs information evaluating the state of a body part of the user that is different from the body part to which the sensor device is attached. The control unit determines an evaluation of the state of the user's body parts by obtaining evaluation information from the learning model, thereby estimating the state of the user's body parts.

2. The information processing apparatus according to claim 1, wherein the control unit estimates the state of more body parts than the number of body parts on which the sensor device is attached in the user, using the learning model.

3. When the learning model receives feature data indicating the characteristics of the movement of the body part, it outputs evaluation information for the state of a body part different from the body part to which the sensor device is attached in the user. The control unit, The feature data is obtained from the sensor data, The information processing device according to claim 1 or 2, wherein the state of the user's body parts is estimated, and an evaluation of the state of the user's body parts is determined by obtaining the evaluation information from the learning model.

4. The at least one sensor device includes a sensor device that is attached to the user's head, The control unit acquires the sensor data indicating the movement of the user's head, The information processing apparatus according to claim 3, wherein the feature data includes feature data indicating the characteristics of the user's head movement in at least one of the forward / backward, left / right, and up / down directions.

5. The information processing apparatus according to claim 4, wherein the feature data includes feature data indicating the characteristics of the user's head movement in the vertical direction and feature data indicating the characteristics of the user's head movement in the horizontal direction.

6. The aforementioned at least one sensor device further includes a sensor device to be attached to the user's foot, The control unit further acquires the sensor data indicating the movement of the user's foot, The information processing apparatus according to claim 5, further comprising feature data indicating the characteristics of the movement of the foot.

7. The aforementioned at least one sensor device further includes a sensor device to be attached to the user's thigh, The control unit further acquires the sensor data indicating the movement of the user's thigh, The information processing apparatus according to claim 5, wherein the feature data further includes feature data indicating the characteristics of the movement of the thigh.

8. The aforementioned at least one sensor device further includes a sensor device to be attached to the user's forearm, The control unit further acquires the sensor data indicating the movement of the user's forearm, The information processing apparatus according to claim 5, further comprising feature data indicating the characteristics of the movement of the forearm.

9. The at least one sensor device further includes a sensor device to be attached to the user's forearm and a sensor device to be attached to the user's foot. The control unit further acquires the sensor data indicating the movement of the user's forearm and the sensor data indicating the movement of the user's foot. The information processing apparatus according to claim 5, wherein the feature data further includes feature data indicating the characteristics of the movement of the forearm and feature data indicating the characteristics of the movement of the foot.

10. The information processing apparatus according to claim 1, wherein the control unit determines an evaluation of at least one of the user's head state, arm state, torso state, knee state, and foot state.

11. The information processing apparatus according to claim 1, wherein the control unit selects a learning model from among a plurality of learning models to acquire the evaluation information according to the type of sensor device that transmitted the sensor data to the information processing apparatus.

12. With an additional communications unit, The control unit, Generate an evaluation signal corresponding to at least one of the evaluations, The information processing apparatus according to claim 1, wherein the generated evaluation signal is transmitted to an external device by the communication unit.

13. The information processing apparatus according to claim 12, wherein the external device is an earphone.

14. An electronic device comprising an information processing device according to claim 1, and a notification unit that notifies information about the state of the body part estimated by the information processing device.

15. At least one sensor device attached to a part of the user's body, The system includes an information processing device that acquires sensor data indicating the movement of the body part from the sensor device, and estimates the state of a body part in the user that is different from the body part to which the sensor device is attached, using the acquired sensor data and a learning model. When the learning model receives the sensor data, it outputs information evaluating the state of a body part of the user that is different from the body part to which the sensor device is attached. The information processing device is an information processing system that estimates the state of the user's body parts and determines an evaluation of the state of the user's body parts by obtaining the evaluation information from the learning model.

16. To acquire sensor data indicating the movement of a body part from at least one sensor device attached to a body part of the user, This includes estimating the state of a body part of the user that is different from the body part to which the sensor device is attached, using the acquired sensor data and a learning model. When the learning model receives the sensor data, it outputs information evaluating the state of a body part of the user that is different from the body part to which the sensor device is attached. An information processing method for estimating the state of the user's body parts, comprising determining an evaluation of the state of the user's body parts by obtaining the evaluation information from the learning model.

17. On the computer, To acquire sensor data indicating the movement of a body part from at least one sensor device attached to a body part of the user, The system is configured to estimate the state of a body part of the user that is different from the body part to which the sensor device is attached, using the acquired sensor data and the learning model. When the learning model receives the sensor data, it outputs information evaluating the state of a body part of the user that is different from the body part to which the sensor device is attached. A program that estimates the state of the user's body parts, including determining an evaluation of the state of the user's body parts by obtaining the evaluation information from the learning model.

Images