Information processing device, electronic equipment, information processing system, information processing method, and program

Abstract

This information processing device comprises a control unit. The control unit acquires sensor data indicating the motion of a part of a user's body from at least one piece of sensor equipment worn on the part of the body. From the acquired sensor data and a learning model, the control unit infers a state of a part of the user's body different from the part of the user's body where the sensor equipment is worn.

Description

Information processing device, electronic device, information processing system, information processing method, and program Cross-reference to related applications

[0001] This application claims priority to Japanese Patent Application No. 2021-090695, filed on May 28, 2021, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to an information processing device, an electronic device, an information processing system, an information processing method, and a program.

[0003] Conventionally, there are known techniques for estimating a person's walking state (for example, Patent Document 1). The motion analysis device described in Patent Document 1 includes a detection means attached to the person's body and an analysis means. The analysis means analyzes the walking state and / or running state based on signals from the detection means. Furthermore, the interpretation described in Non-Patent Document 1 is known as an interpretation of good walking style.

[0004] JP 2016-150193 A

[0005] ASICS Sports Research Institute, "The Ultimate Walking Method," Kodansha Gendai Shinsho, September 2019, pp. 92, 94, 95

[0006] An information processing device according to one embodiment of the present disclosure includes a control unit that acquires sensor data indicating the movement of a body part of a user from at least one sensor device attached to the body part, and estimates the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.

[0007] The electronic device according to an embodiment of the present disclosure includes a notification unit that notifies information about the state of the body part estimated by the information processing device.

[0008] An information processing system according to one embodiment of the present disclosure includes at least one sensor device attached to a body part of a user; and an information processing device that acquires sensor data indicating the movement of the body part from the sensor device, and estimates the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.

[0009] An information processing method according to one embodiment of the present disclosure includes acquiring sensor data indicating the movement of a body part of a user from at least one sensor device attached to the body part, and estimating the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.

[0010] A program according to one embodiment of the present disclosure causes a computer to: acquire sensor data indicating the movement of a body part of a user from at least one sensor device attached to the body part; and estimate the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.

[0011] 10 is a diagram illustrating a schematic configuration of an information processing system according to an embodiment of the present disclosure. FIG. 11 is a diagram for explaining a local coordinate system and a global coordinate system. FIG. 12 is a functional block diagram illustrating a configuration of the information processing system shown in FIG. 1. FIG. 13 is a diagram illustrating a schematic configuration of a learning model according to an embodiment of the present disclosure. FIG. 14 is a diagram illustrating an example of a score according to an embodiment of the present disclosure. FIG. 15 is a diagram illustrating an example of association according to an embodiment of the present disclosure. FIG. 16 is a graph illustrating the accuracy of the learning model. FIG. 17 is a flowchart illustrating the operation of an evaluation process performed by the electronic device shown in FIG. 1. FIG. 18 is a functional block diagram illustrating a configuration of an information processing system according to another embodiment of the present disclosure. FIG. 19 is a sequence diagram illustrating the operation of the evaluation process performed by the information processing system shown in FIG.

[0012] There is a need for a new technique for estimating the state of a body part of a user while walking. According to the present disclosure, it is possible to provide a new technique for estimating the state of a body part of a user while walking.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following drawings, the same components are denoted by the same reference numerals.

[0014] 1 can estimate the condition of any of a plurality of body parts of a user while walking. By using the information processing system 1, the user can understand whether the condition of the user's own body part while walking is good or bad.

[0015] The information processing system 1 may be used for any purpose that requires grasping the state of a user's body parts while walking. For example, the information processing system 1 may be used to grasp the state of a user's body parts when the user walks as exercise, when the user practices walking as a model, when the user practices footwork for mountain climbing, or when the user practices race walking.

[0016] The information processing system 1 includes a sensor device 10A, a sensor device 10B, a sensor device 10C, a sensor device 10D, and an electronic device 20. However, the information processing system 1 does not need to include all of the sensor devices 10A, 10B, 10C, and 10D. The information processing system 1 only needs to include at least one of the sensor devices 10A, 10B, 10C, and 10D.

[0017] Hereinafter, when there is no need to particularly distinguish between the sensor devices 10A to 10D, they will also be collectively referred to as "sensor devices 10."

[0018] The sensor device 10 and the electronic device 20 can communicate with each other via a communication line. The communication line may be wired or wireless.

[0019] The sensor device 10 is attached to a body part of the user. The sensor device 10 detects sensor data that indicates the movement of the body part of the user to which the sensor device 10 is attached. The sensor data is data in a local coordinate system.

[0020] As shown in FIG. 2, the local coordinate system is a coordinate system based on the position of the sensor device 10. In FIG. 2, the position of the sensor device 10A is indicated by a dashed line as an example of the position of the sensor device 10. The local coordinate system is composed of, for example, an x-axis, a y-axis, and a z-axis. The x-axis, the y-axis, and the z-axis are perpendicular to one another. The x-axis is parallel to the front-to-back direction as seen from the sensor device 10. The y-axis is parallel to the left-to-right direction as seen from the sensor device 10. The z-axis is parallel to the up-to-down direction as seen from the sensor device 10. A direction parallel to the x-axis of the local coordinate system is also referred to as the "front-to-back direction of the local coordinate system." A direction parallel to the y-axis of the local coordinate system is also referred to as the "left-to-right direction of the local coordinate system." A direction parallel to the z-axis of the local coordinate system is also referred to as the "up-to-down direction of the local coordinate system."

[0021] As shown in FIG. 2 , the global coordinate system is a coordinate system based on a position in the space in which the user walks. The global coordinate system is composed of, for example, an X-axis, a Y-axis, and a Z-axis. The X-axis, the Y-axis, and the Z-axis are perpendicular to one another. The X-axis is parallel to the front-to-back direction as seen from the user. The Y-axis is parallel to the left-to-right direction as seen from the user. The Z-axis is parallel to the up-to-down direction as seen from the user. The direction parallel to the X-axis of the global coordinate system is also referred to as the "front-to-back direction of the global coordinate system." The direction parallel to the Y-axis of the global coordinate system is also referred to as the "left-to-right direction of the global coordinate system." The direction parallel to the Z-axis of the global coordinate system is also referred to as the "up-to-down direction of the global coordinate system." The front-to-back direction of the global coordinate system can also be referred to as the front-to-back direction of the user. The left-to-right direction of the global coordinate system can also be referred to as the left-to-right direction of the user. The up-to-down direction of the global coordinate system can also be referred to as the up-to-down direction of the user.

[0022] As shown in Figure 2, the sagittal plane is a plane that divides the user's body symmetrically left and right, or a plane parallel to the plane that divides the user's body symmetrically left and right. The frontal plane is a plane that divides the user's body into ventral and dorsal sides, or a plane parallel to the plane that divides the user's body into ventral and dorsal sides. The horizontal plane is a plane that divides the user's body into upper and lower sides, or a plane parallel to the plane that divides the user's body into upper and lower sides. The sagittal plane, the frontal plane, and the horizontal plane are perpendicular to each other.

[0023] As shown in FIG. 1 , the sensor device 10A is worn on the user's head. For example, the sensor device 10A is worn on the user's ear. The sensor device 10A may be a wearable device. The sensor device 10A may be an earphone. The sensor device 10A may be included in the earphone. Alternatively, the sensor device 10A may be a device that can be retrofitted to existing glasses, earphones, or the like. The sensor device 10A may be worn on the user's head by any method. The sensor device 10A may be worn on the user's head by being attached to a hair accessory such as a hairband or a hairpin, earrings, a helmet, a hat, a hearing aid, dentures, an implant, or the like.

[0024] The sensor device 10A may be worn on the user's head so that the front-to-back direction as seen from the sensor device 10A coincides with the front-to-back direction of the head as seen from the user, the left-to-right direction as seen from the sensor device 10A coincides with the left-to-right direction of the head as seen from the user, and the up-to-down direction as seen from the sensor device 10A coincides with the up-to-down direction of the head as seen from the user. That is, the sensor device 10A may be worn on the user's head so that the x-axis of a local coordinate system based on the position of the sensor device 10A is parallel to the front-to-back direction of the head as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the head as seen from the user, and the z-axis of the local coordinate system is parallel to the up-to-down direction of the head as seen from the user. However, the front-to-back direction, left-to-right direction, and up-to-down direction as seen from the sensor device 10A do not necessarily coincide with the front-to-back direction, left-to-right direction, and up-to-down direction of the head as seen from the user. In this case, the relative orientation of the sensor device 10A with respect to the user's head may be initialized or known as appropriate. The initialization or making known of the relative posture may be performed by utilizing information on the shape of the jig used to attach the sensor device 10A to the user's head or image information generated by capturing an image of the user's head wearing the sensor device 10A.

[0025] The sensor device 10A detects sensor data indicating the movement of the user's head, including, for example, data indicating at least one of the speed of the user's head, the acceleration of the user's head, the angle of the user's head, the angular velocity of the user's head, the temperature of the user's head, and the geomagnetism at the position of the user's head.

[0026] The sensor device 10B is worn on the user's forearm. For example, the sensor device 10B is worn on the user's wrist. The sensor device 10B may be a wristwatch-type wearable device. The sensor device 10B may be worn on the user's forearm by any method. The sensor device 10B may be attached to the user's forearm by being attached to a band, bracelet, friendship bracelet, glove, ring, false nail, prosthetic arm, or the like. The bracelet may be worn by the user for decorative purposes, or may be used to attach a key to a locker or the like to the wrist.

[0027] The sensor device 10B may be worn on the user's forearm so that the front-to-back direction as seen from the sensor device 10B corresponds to the front-to-back direction of the wrist as seen from the user, the left-to-right direction as seen from the sensor device 10B corresponds to the left-to-right direction of the wrist as seen from the user, and the up-to-down direction as seen from the sensor device 10B corresponds to the rotation direction of the wrist as seen from the user. The rotation direction of the wrist is the direction in which the wrist twists and rotates. In other words, the sensor device 10B may be worn on the user's forearm so that the x-axis of a local coordinate system based on the position of the sensor device 10B is parallel to the front-to-back direction of the wrist as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the wrist as seen from the user, and the z-axis of the local coordinate system is parallel to the rotation direction of the wrist as seen from the user.

[0028] The sensor device 10B detects sensor data indicating the movement of the user's forearm. The sensor data detected by the sensor device 10B includes, for example, data indicating at least one of the speed of the user's forearm, the acceleration of the user's forearm, the angle of the user's forearm, the angular velocity of the user's forearm, the temperature of the user's forearm, and the geomagnetism at the position of the user's forearm.

[0029] The sensor device 10C is attached to the user's thigh. The sensor device 10C may be a wearable device. The sensor device 10C may be attached to the user's thigh by any method. The sensor device 10C may be attached to the user's thigh by a belt. The sensor device 10C may be attached to the user's thigh by being placed in a pocket near the thigh of pants worn by the user. The sensor device 10C may be attached to the user's thigh by being attached to pants, underwear, shorts, a supporter, a prosthetic limb, an implant, or the like.

[0030] The sensor device 10C may be attached to the user's thigh so that the front-to-back direction as seen from the sensor device 10C coincides with the front-to-back direction of the thigh as seen from the user, the left-to-right direction as seen from the sensor device 10C coincides with the left-to-right direction of the thigh as seen from the user, and the up-to-down direction as seen from the sensor device 10C coincides with the rotation direction of the thigh as seen from the user. The rotation direction of the thigh is the direction in which the thigh twists and rotates. In other words, the sensor device 10C may be attached to the user's thigh so that the x-axis of a local coordinate system based on the position of the sensor device 10C is parallel to the front-to-back direction of the thigh as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the thigh as seen from the user, and the z-axis of the local coordinate system is parallel to the rotation direction of the thigh as seen from the user.

[0031] The sensor device 10C detects sensor data indicating the movement of the user's thigh, including, for example, data indicating at least one of the speed of the user's thigh, the acceleration of the user's thigh, the angle of the user's thigh, the angular velocity of the user's thigh, the temperature of the user's thigh, and the geomagnetism at the position of the user's thigh.

[0032] The sensor device 10D is attached to the user's foot. In this embodiment, the foot refers to the part of the user's foot from the ankle to the toes. The sensor device 10D may be a shoe-shaped wearable device. The sensor device 10D may be attached to the user's foot by any method. The sensor device 10D may be provided in a shoe. The sensor device 10D may be attached to the user's foot by being attached to an anklet, a band, a friendship bracelet, a false nail, a tattoo sticker, a support, a cast, a sock, an insole, a prosthetic limb, a ring, an implant, or the like.

[0033] The sensor device 10D may be attached to the user's foot so that the front-to-rear direction as seen from the sensor device 10D coincides with the front-to-rear direction of the foot as seen from the user, the left-to-right direction as seen from the sensor device 10D coincides with the left-to-right direction of the foot as seen from the user, and the up-to-down direction as seen from the sensor device 10D coincides with the up-to-down direction of the foot as seen from the user. In other words, the sensor device 10D may be attached to the user's foot so that the x-axis of a local coordinate system based on the position of the sensor device 10D is parallel to the front-to-rear direction of the foot as seen from the user, the y-axis of the local coordinate system is parallel to the left-to-right direction of the foot as seen from the user, and the z-axis of the local coordinate system is parallel to the up-to-down direction of the foot as seen from the user.

[0034] The sensor device 10D detects sensor data indicating the movement of the user's feet, including, for example, data indicating at least one of the speed, acceleration, angle, angular velocity, temperature, and geomagnetism at the position of the user's feet.

[0035] The electronic device 20 is carried by, for example, a user while walking. The electronic device 20 functions as an information processing device and can estimate the state of any one of a plurality of body parts of the user based on sensor data detected by the sensor device 10. In this embodiment, the electronic device 20 estimates the state of the user's body part and determines an evaluation of the state of the user's body part. The electronic device 20 is, for example, a mobile device such as a mobile phone, a smartphone, or a tablet.

[0036] 3 , the sensor device 10 is configured to include at least a communication unit 11 and a sensor unit 12. The sensor device 10 may further include a notification unit that notifies information, a storage unit 14, and a control unit 15. In this embodiment, the notification unit is the output unit 13. However, the notification unit is not limited to the output unit 13. The sensor devices 10C and 10D do not need to be configured to include the output unit 13.

[0037] The communication unit 11 includes at least one communication module capable of communicating with the electronic device 20 via a communication line. The communication module is a communication module that complies with the standard of the communication line. The standard of the communication line is, for example, a short-range wireless communication standard including Bluetooth (registered trademark), Wi-Fi (registered trademark), infrared, and NFC (Near Field Communication).

[0038] The sensor unit 12 is configured to include any sensor corresponding to the sensor data to be detected by the sensor device 10. The sensor unit 12 is configured to include at least one of, for example, a three-axis motion sensor, a three-axis acceleration sensor, a three-axis speed sensor, a three-axis gyro sensor, a three-axis geomagnetic sensor, a temperature sensor, a barometric pressure sensor, and a camera. When the sensor unit 12 is configured to include a camera, the camera can capture an image of a body part of the user and analyze the generated image to detect the movement of the body part.

[0039] When the sensor unit 12 is configured to include an acceleration sensor and a geomagnetic sensor, data detected by each of the acceleration sensor and the geomagnetic sensor may be used to calculate the initial angle of the body part to be detected by the sensor device 10. In addition, data detected by each of the acceleration sensor and the geomagnetic sensor may be used to correct data indicating the angle detected by the sensor device 10.

[0040] When the sensor unit 12 is configured to include a gyro sensor, the angle of the body part to be detected by the sensor device 10 may be calculated by integrating the angular velocity detected by the gyro sensor over time.

[0041] If the sensor unit 12 is configured to include a barometric pressure sensor, data detected by the barometric pressure sensor may be used when determining an evaluation of the condition of the user's body part by a control unit 26 of the electronic device 20, which will be described later.

[0042] The output unit 13 is capable of outputting data. The output unit 13 includes at least one output interface capable of outputting data. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.

[0043] When the output unit 13 is included in the sensor device 10A, the output unit 13 may be configured to include a speaker. When the output unit 13 is included in the sensor device 10B, the output unit 13 may be configured to include a display.

[0044] The storage unit 14 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory may be, for example, a random access memory (RAM) or a read-only memory (ROM). The RAM may be, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The ROM may be, for example, an electrically erasable programmable read-only memory (EEPROM). The storage unit 14 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 14 stores data used in the operation of the sensor device 10 and data obtained by the operation of the sensor device 10. For example, the storage unit 14 stores system programs, application programs, embedded software, and the like.

[0045] The control unit 15 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 15 controls each part of the sensor device 10 and executes processing related to the operation of the sensor device 10.

[0046] The control unit 15 receives a signal instructing the start of data detection from the electronic device 20 via the communication unit 11. Upon receiving this signal, the control unit 15 starts data detection. For example, the control unit 15 acquires data detected by the sensor unit 12 from the sensor unit 12. The control unit 15 transmits the acquired data as sensor data to the electronic device 20 via the communication unit 11. The signal instructing the start of data detection is transmitted as a broadcast signal from the electronic device 20 to the multiple sensor devices 10. By transmitting the signal instructing the start of data detection as a broadcast signal to the multiple sensor devices 10, the multiple sensor devices 10 can simultaneously start data detection.

[0047] The control unit 15 acquires data from the sensor unit 12 at a preset time interval and transmits the acquired data as sensor data via the communication unit 11. This time interval may be set based on the walking speed of a typical user, etc. This time interval may be the same for each of the multiple sensor devices 10. By setting this time interval to be the same for the multiple sensor devices 10, the timing at which each of the multiple sensor devices 10 detects data can be synchronized.

[0048] 3, the electronic device 20 includes a communication unit 21, an input unit 22, a notification unit that notifies information, a storage unit 25, and a control unit 26. In this embodiment, the notification unit is an output unit 23 and a vibration unit 24. However, the notification unit is not limited to the output unit 23 and the vibration unit 24. The output unit 23 and the vibration unit 24 may be mounted on the electronic device 20, or may be disposed near any of the sensor devices 10B, 10C, and 10D.

[0049] The communication unit 21 includes at least one communication module capable of communicating with the sensor device 10 via a communication line. The communication module is at least one communication module compatible with the communication line standard. The communication line standard is, for example, a short-range wireless communication standard including Bluetooth (registered trademark), Wi-Fi (registered trademark), infrared, and NFC.

[0050] The communication unit 21 may further include at least one communication module connectable to a network 2 as shown in Fig. 10 described below. The communication module is a communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation).

[0051] The input unit 22 can receive input from a user. The input unit 22 includes at least one input interface that can receive input from a user. The input interface is, for example, a physical key, a capacitance key, a pointing device, a touch screen that is integrated with a display, a microphone, or the like.

[0052] The output unit 23 is capable of outputting data. The output unit 23 includes at least one output interface capable of outputting data. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display.

[0053] The vibration section 24 is capable of vibrating the electronic device 20. The vibration section 24 includes a vibration element. The vibration element is, for example, a piezoelectric element.

[0054] The storage unit 25 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 25 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 25 stores data used in the operation of the electronic device 20 and data obtained by the operation of the electronic device 20. For example, the storage unit 25 stores system programs, application programs, embedded software, etc. For example, the storage unit 25 stores a learning model (described below) and associations such as those shown in FIG. 6 (described below).

[0055] The control unit 26 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA or an ASIC. The control unit 26 controls each part of the electronic device 20 and executes processes related to the operation of the electronic device 20.

[0056] The control unit 26 receives an input instructing the execution of gait evaluation via the input unit 22. This input causes the electronic device 20 to execute a determination process for determining an evaluation of the condition of a body part. This input is, for example, input via the input unit 22 by a user wearing the sensor device 10. The user inputs this input via the input unit 22, for example, before starting to walk. When the control unit 26 receives this input via the input unit 22, it transmits a signal instructing the start of data detection as a broadcast signal to the multiple sensor devices 10 via the communication unit 21. After the signal instructing the start of data detection is transmitted to the multiple sensor devices 10, sensor data is transmitted from at least one sensor device 10 to the electronic device 20.

[0057] The control unit 26 receives sensor data from at least one sensor device 10 via the communication unit 21. The control unit 26 acquires the sensor data from the sensor device 10 by receiving the sensor data from the sensor device 10. As described below, the control unit 26 determines an evaluation of the condition of the user's body parts based on the acquired sensor data and a learning model. The content of the evaluation for evaluating the condition of the body parts may be set based on an interpretation of a walking style that is generally considered good. An example of an interpretation of a walking style that is generally considered good is the interpretation described in "ASICS Sports Research Institute, 'The Ultimate Walking Style,' Kodansha Gendai Shinsho, published in September 2019, pp. 92, 94, and 95." Before describing the details of the evaluation determination process by the control unit 26, an example of the interpretation and evaluation content of a walking style that is generally considered good will be described. The control unit 26 may determine an evaluation of at least one of (1) the condition of the head, (2) the condition of the arms, (3) the condition of the trunk, (4) the condition of the knees, and (5) the condition of the feet, which will be described below.

[0058] (1) Head Condition An assessment of the head condition may be determined.

[0059] It is generally understood that it is better for a user's body to sway as little as possible while walking. When the user's body sways little while walking, the user's head sways less than when the user's body sways much while walking. Therefore, when the user's head sways little while walking, a higher evaluation may be determined as the evaluation of the head state than when the user's head sways much while walking.

[0060] When a user's chin is tucked while walking, the user's head shakes less than when the user's chin is not tucked while walking. Therefore, when a user's chin is tucked while walking, a higher evaluation may be determined as the evaluation of the state of the user's head than when the user's chin is not tucked while walking.

[0061] When a user is walking and looking away, the user's head shakes less than when the user is walking and looking closer. Therefore, when the user is walking and looking away, a higher evaluation may be determined for the state of the user's head than when the user is walking and looking closer.

[0062] (2) Arm Condition An assessment of the arm condition may be determined.

[0063] Generally, a state in which the arms of a user are swung firmly while walking is interpreted as being in a good state. Therefore, when the user's arms are swung widely while walking, a higher evaluation may be determined for the arm state than when the user's arms are swung less while walking.

[0064] Generally, the state of a user's arms while walking is interpreted as being good when the arms are pulled backward. Therefore, when a user's arms are pulled backward while walking, a higher evaluation may be determined for the arm state than when the user's arms are not pulled backward while walking.

[0065] (3) Trunk Condition An assessment of the trunk condition may be determined.

[0066] Generally, a state in which the shoulders are open and the back is straight is interpreted as a good state for the state of the trunk of a user while walking. Therefore, when the user's shoulders are open and the back is straight, the state of the trunk may be evaluated higher than when the user's shoulders are closed and the back is bent.

[0067] Generally, the state of the trunk of a user walking is interpreted as being good when the pelvis is upright and the waist is straight. Therefore, when the user's pelvis is upright and the waist is straight, the state of the trunk may be evaluated higher than when the user's pelvis is not upright and the waist is bent.

[0068] (4) Knee Condition An assessment of the knee condition may be determined.

[0069] Generally, the condition of the knees of a user who is walking is interpreted as being in a good state when the knees are not bent. Therefore, when the knees of a user who is walking are not bent, the knee condition may be evaluated higher than when the knees of a user who is walking are bent.

[0070] (5) Foot Condition An assessment of the foot condition may be determined.

[0071] It is generally understood that it is better for a user to take as wide a stride as possible while walking, and therefore, when a user takes a wide stride while walking, a higher evaluation may be determined for the condition of the foot than when a user takes a narrow stride while walking.

[0072] [Evaluation Determination Process] The control unit 26 determines an evaluation of the condition of a user's body part based on the sensor data and the learning model. The learning model is machine-learned so that, when sensor data or feature data is input, it outputs evaluation information for the condition of a specific body part of the user. That is, the control unit 26 inputs the sensor data or feature data into the learning model and acquires evaluation information for the condition of the user's body part from the learning model, thereby determining an evaluation of the condition of the user's body part. The feature data is data indicating the characteristics of the movement of the body part to which the sensor device 10 is attached. The control unit 26 acquires the feature data from the sensor data. An example of the feature data will be described later. Here, the learning model can be machine-learned so that, when sensor data or feature data is input, it outputs evaluation information for the condition of a body part other than the body part to which the sensor device 10 is attached. This is because the user's multiple body parts move while influencing each other during walking. Using such a learning model, the control unit 26 can determine an evaluation of the condition of a body part other than the body part to which the sensor device 10 is attached. Furthermore, by using such a learning model, the control unit 26 can determine evaluations of the states of more body parts than the number of body parts to which the sensor devices 10 are attached on the user.

[0073] The learning model according to this embodiment is machine-learned so that when feature data is input, it outputs a score as information on the evaluation of the condition of a specific body part. The score indicates the evaluation of the condition of the specific body part. The higher the score, the higher the evaluation of the condition of the specific body part corresponding to the score. The control unit 26 obtains the score from the learning model and determines the evaluation of the condition of the specific body part corresponding to the score.

[0074] An example of feature data input to the learning model will be described below.

[0075] The feature data may be data indicating statistical values ​​of the sensor data. Because the feature data is statistical values ​​of the sensor data, the feature data can indicate characteristics of the movement of body parts. For example, the feature data may be a maximum value, minimum value, average value, or variance of the sensor data over a predetermined period. The predetermined period may be, for example, the user's walking cycle or a portion of the walking cycle. The walking cycle may be, for example, the period from when one of the user's two feet lands on the ground until it lands on the ground again. The portion of the walking cycle may be, for example, a stance phase or a swing phase. The stance phase may be, for example, the period from when one of the user's two feet lands on the ground until it lifts off. The swing phase may be, for example, the period from when one of the user's two feet lifts off the ground until it lands on the ground. The control unit 26 may detect the user's walking cycle and a portion of the walking cycle by analyzing the sensor data. The control unit 26 may acquire the feature data from the sensor data by performing a calculation on the sensor data.

[0076] The feature data may be sensor data at a predetermined timing. When the feature data is sensor data at a predetermined timing, the feature data can indicate the characteristics of the movement of a body part. For example, the feature data is sensor data at the time when the user lands. The landing timing is the time when the user's foot lands on the ground. The control unit 26 may detect the landing timing by analyzing the sensor data.

[0077] The feature data may be data in any coordinate system. The feature data may be data in a local coordinate system or data in a global coordinate system. When the feature data is data in the global coordinate system, the control unit 26 acquires the feature data in the global coordinate system by performing coordinate transformation on the sensor data in the local coordinate system.

[0078] An example of a learning model will be described with reference to Fig. 4. The learning model 30 shown in Fig. 4 is a neural network learning model. The learning model 30 includes an input layer 31, a hidden layer 32, a hidden layer 33, and an output layer 34.

[0079] When three pieces of feature data are input to the input layer 31, the learning model 30 outputs one score from the output layer 34.

[0080] The input layer 31 includes three neurons. Feature data is input to each of the three neurons in the input layer 31. The hidden layers 32 and 33 each include 64 neurons. The output layer 34 includes one neuron. A score is output from the neuron in the output layer 34. In the input layer 31, the hidden layer 32, the hidden layer 33, and the output layer 34, neurons in one of two adjacent layers are connected to neurons in the other layer. In learning the learning model 30, weight coefficients corresponding to the connection strength between each neuron are adjusted.

[0081] The number of neurons included in each of the input layer 31, the hidden layer 32, the hidden layer 33, and the output layer 34 may be adjusted depending on the amount of feature data used.

[0082] Fig. 5 shows an example of scores according to an embodiment of the present disclosure. As described above, the control unit 26 inputs feature data into learning models to obtain scores such as those shown in Fig. 5. In Fig. 5, the control unit 26 obtains scores for (1) head state, (2) arm state, (3) trunk state, (4) knee state, and (5) foot state using five learning models.

[0083] In FIG. 5 , the scores are numbers ranging from 1 to 5. The control unit 26 (1) acquires a score of 5 as an evaluation of the state of the head. The control unit 26 (2) acquires a score of 4 as an evaluation of the state of the arms. The control unit 26 (3) acquires a score of 3 as an evaluation of the state of the trunk. The control unit 26 (4) acquires a score of 1 as an evaluation of the state of the knees. The control unit 26 (5) acquires a score of 2 as an evaluation of the state of the feet.

[0084] [Evaluation Signal Generation Process] When the control unit 26 determines an evaluation of the condition of the user's body part, it may generate an evaluation signal corresponding to the determined evaluation. If the control unit 26 determines multiple evaluations, it may generate an evaluation signal corresponding to at least one of the determined multiple evaluations. The evaluation signal may be a signal indicating content praising the user if the determined evaluation is higher than an evaluation threshold. The evaluation signal may be a signal indicating advice to the user if the determined evaluation is lower than the evaluation threshold. The evaluation threshold may be set based on, for example, the average value of evaluations of general users. If a learning model is used, the evaluation threshold may be the average value of scores of general users. The content of praise and advice to the user may be set based on the interpretation of a walking style that is generally considered good, as described above.

[0085] In Figure 5, the evaluation threshold is a score of 3. In Figure 5, "Good" indicates that the evaluation is higher than the evaluation threshold. "Poor" indicates that the evaluation is lower than the evaluation threshold. "Average" indicates that the evaluation is the same as the evaluation threshold.

[0086] 5, the evaluations for (1) the head state and (2) the arm state are higher than the evaluation threshold. The control unit 26 generates, as evaluation signals for (1) the head state and (2) the arm state, signals indicating content praising the user. For example, as an evaluation signal for (1) the head state, the control unit 26 generates a signal indicating that the user's head shaking is small and the head state is good. For example, as an evaluation signal for (2) the arm state, the control unit 26 generates a signal indicating that the user's arm swing is large and the arm is pulled backward and the arm state is good.

[0087] In Fig. 5, the evaluations for (4) the knee condition and (5) the foot condition are each lower than the evaluation threshold. The control unit 26 generates, as an evaluation signal, a signal indicating advice to the user for each of (4) the knee condition and (5) the foot condition. For example, as an evaluation signal for (4) the knee condition, the control unit 26 generates a signal indicating advice to not bend the knee. For example, as an evaluation signal for (5) the foot condition, the control unit 26 generates a signal indicating advice to increase the stride length.

[0088] The control unit 26 may transmit the generated evaluation signal to an external device via the communication unit 21. The control unit 26 may transmit the evaluation signal via the communication unit 21 to any sensor device 10 having an output unit 13 as an external device. In this case, in the sensor device 10, the control unit 15 receives the evaluation signal via the communication unit 21. The control unit 15 causes the output unit 13, which serves as a notification unit, to notify the content indicated by the received evaluation signal. As an example of notification, the control unit 15 causes the output unit 13 to output the content indicated by the evaluation signal. With this configuration, the user can grasp the evaluation of the state of their body parts while walking.

[0089] For example, if the sensor device 10A is an earphone or is included in an earphone, the control unit 26 may transmit the evaluation signal to the earphone as an external device via the communication unit 21. In this case, in the sensor device 10A, the control unit 15 receives the evaluation signal via the communication unit 11. In the sensor device 10A, the control unit 15 causes the output unit 13, which serves as a notification unit, to notify the content indicated by the evaluation signal. As an example of notification, in the sensor device 10A, the control unit 15 notifies the content indicated by the evaluation signal by outputting it as sound from the speaker of the output unit 13. With this configuration, the evaluation of the state of the body part can be notified to the user by sound. Notifying the user by sound reduces the possibility of interfering with the user's walking.

[0090] The control unit 26 may notify the user of the content indicated by the generated evaluation signal by the notification unit. As an example of the notification, the control unit 26 may cause the output unit 23 to output the content indicated by the generated evaluation signal. As another example of the notification, the control unit 26 may vibrate the vibration unit 24 with a vibration pattern corresponding to the determined evaluation.

[0091] [Learning Model Selection Process] The control unit 26 may select a learning model to be used in the above-described evaluation determination process from among a plurality of learning models, depending on the type of sensor device 10 that transmitted sensor data to the electronic device 20. The control unit 26 may select a learning model to be used in the evaluation determination process by referring to the correspondence as shown in FIG. 6 stored in the memory unit 25.

[0092] The learning model shown in FIG. 6 was generated by a learning model generation method described below. The numbers in parentheses shown together with the learning model are the accuracy and precision of the learning model calculated by the learning model generation method described below. The method for calculating the accuracy and precision of the learning model will be described later. The number on the left side of the parentheses is the accuracy of the learning model. The number on the right side of the parentheses is the precision of the learning model.

[0093] In Figure 6, learning models marked with double circles are learning models with an accuracy of 90% or more and a probability of 70% or more. Learning models marked with single circles are learning models that do not satisfy the accuracy and probability conditions of learning models marked with double circles. Learning models marked with single circles are learning models with an accuracy of 80% or more and a probability of 60% or more. Learning models marked with triangles are learning models with an accuracy of less than 80% or a probability of less than 60%. The control unit 26 may determine an evaluation of some of the states from (1) the head state to (5) the foot state according to the accuracy and probability of the learning model, or may determine an evaluation of all of the states from (1) the head state to (5) the foot state.

[0094] In Fig. 6, the control unit 26 selects a learning model to be used in the evaluation determination process by selecting Case C1, Case C2, Case C3, Case C4, or Case C5. Cases C1 to C5 correspond to the learning model to be used in the evaluation determination process and the type of sensor device 10 used to acquire feature data to be input to the learning model. However, the learning model is not limited to the one shown in Fig. 6. A learning model using sensor data or feature data of any combination of sensor devices 10 may be adopted.

[0095] The control unit 26 may select one of cases C1 to C5 depending on the type of sensor device 10 that transmits sensor data to the electronic device 20. As an example, the control unit 26 may select any combination of sensor devices 10 from among a plurality of sensor devices 10 that have transmitted sensor data to the electronic device 20. The control unit 26 may select a case from cases C1 to C5 that corresponds to the selected combination of sensor devices 10. Information on the feature data used in cases C1 to C5 as described below may be stored in the storage unit 25 in association with cases C1 to C5.

[0096] <Case C1> The control unit 26 may select case C1 when the sensor device 10A is the only sensor device 10 that transmits sensor data to the electronic device 20. For example, when the sensor device 10A is the only sensor device 10 worn by the user, the sensor device 10A is the only sensor device 10 that transmits sensor data to the electronic device 20. Alternatively, the control unit 26 may select case C1 when the sensor device 10A is selected from among the multiple sensor devices 10 that transmitted sensor data to the electronic device 20.

[0097] In case C1, when determining the evaluation for (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, the control unit 26 selects the learning models 30A, 30B, 30C, 30D, and 30E, respectively.

[0098] In case C1, the feature data input to the learning models 30A to 30E includes feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. However, it is sufficient that the feature data input to the learning models 30A to 30E include feature data indicating the characteristics of the head movement in at least one of the forward-backward, left-right, and up-down directions of the global coordinate system.

[0099] In case C1 according to this embodiment, the control unit 26 inputs three pieces of feature data to each of the learning models 30A to 30E.

[0100] One of the three pieces of feature data input to the learning models 30A to 30E corresponds to feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system. In this embodiment, this piece of feature data is the average value of the user's head angle in the up-down direction of the global coordinate system. This piece of feature data is acquired from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0101] Two of the three pieces of feature data input to the learning models 30A to 30E correspond to feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. In this embodiment, these two pieces of feature data are the maximum value of the user's head angle in the left-right direction of the global coordinate system and the average value of the user's head angle in the left-right direction of the global coordinate system. These two pieces of feature data are acquired from sensor data indicating the user's head movement detected by the sensor device 10A. The maximum and average values ​​of the user's head angle in the feature data may be the maximum and average values ​​of the head angle during the user's walking cycle, respectively.

[0102] <Case C2> The control unit 26 may select case C2 when the sensor devices 10A and 10D are the only sensor devices 10 that have transmitted sensor data to the electronic device 20. Alternatively, the control unit 26 may select case C2 when the sensor devices 10A and 10D are selected from among the multiple sensor devices 10 that have transmitted sensor data to the electronic device 20.

[0103] In case C2, when determining the evaluation for (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, the control unit 26 selects learning models 30F, 30G, 30H, 30I, and 30J, respectively.

[0104] In case C2, the feature data input to the learning models 30F to 30J includes, in the same or similar manner as in case C1, feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. Furthermore, the feature data input to the learning models 30F to 30J includes feature data indicating the characteristics of the user's foot movement.

[0105] In case C2 according to this embodiment, the control unit 26 inputs three pieces of feature data to each of the learning models 30F to 30J.

[0106] Two of the three pieces of feature data input to the learning models 30F to 30J correspond to feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. In this embodiment, these two pieces of feature data are the average value of the user's head angle in the up-down direction of the global coordinate system and the maximum value of the user's head angle in the left-right direction of the global coordinate system. These two pieces of feature data are acquired from data indicating the user's head movement detected by the sensor device 10A. The average value and maximum value of the user's head angle in the feature data may be the average value and maximum value of the head angle during the user's walking cycle, respectively.

[0107] One of the three pieces of feature data input to the learning models 30F to 30J corresponds to feature data indicating the characteristics of the user's foot movement. In this embodiment, this piece of feature data is the maximum value of the user's foot acceleration in the up-down direction of a local coordinate system based on the position of the sensor device 10D. This piece of feature data is acquired from sensor data indicating the user's foot movement detected by the sensor device 10D. The maximum value of the user's foot acceleration in the feature data may be the maximum value of the foot acceleration in the user's walking cycle.

[0108] <Case C3> The control unit 26 may select case C3 when the sensor devices 10A and 10C are the only sensor devices 10 that have transmitted sensor data to the electronic device 20. Alternatively, the control unit 26 may select case C3 when the sensor devices 10A and 10C are selected from the plurality of sensor devices 10 that have transmitted sensor data to the electronic device 20.

[0109] In case C3, when determining an evaluation for (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, the control unit 26 selects learning models 30K, 30L, 30M, 30N, and 30O, respectively.

[0110] In case C3, the feature data input to the learning models 30K to 30O includes, in the same or similar manner as in case C1, feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. Furthermore, the feature data input to the learning models 30K to 30O includes feature data indicating the characteristics of the user's thigh movement.

[0111] In case C3 according to this embodiment, the control unit 26 inputs three pieces of feature data to each of the learning models 30K to 30O.

[0112] Two of the three pieces of feature data input to the learning models 30K to 30O correspond to feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. In this embodiment, these two pieces of feature data are the average value of the user's head angle in the up-down direction of the global coordinate system and the average value of the user's head angle in the left-right direction of the global coordinate system. These two pieces of feature data are acquired from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0113] One of the three pieces of feature data input to the learning models 30K to 30O is feature data that indicates the characteristics of the movement of the user's thigh. In this embodiment, this piece of feature data is the variance during the stance phase of the angular velocity of the user's thigh in the left-right direction of a local coordinate system based on the position of the sensor device 10C. This piece of feature data is acquired from sensor data that indicates the movement of the user's thigh detected by the sensor device 10D.

[0114] <Case C4> The control unit 26 may select case C4 when the sensor devices 10A and 10B are the only sensor devices 10 that have transmitted sensor data to the electronic device 20. Alternatively, the control unit 26 may select case C4 when the sensor devices 10A and 10B are selected from among the multiple sensor devices 10 that have transmitted sensor data to the electronic device 20.

[0115] In case C4, when determining the evaluation for (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, the control unit 26 selects learning models 30P, 30Q, 30R, 30S, and 30T, respectively.

[0116] In case C4, the feature data input to the learning models 30P to 30T includes, the same as or similar to case C1, feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. Furthermore, the feature data input to the learning models 30P to 30T includes feature data indicating the characteristics of the user's forearm movement.

[0117] In case C4 according to this embodiment, the control unit 26 inputs three pieces of feature data to each of the learning models 30P to 30T.

[0118] Two of the three pieces of feature data input to the learning models 30P to 30T correspond to feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. In this embodiment, these two pieces of feature data are the average value of the user's head angle in the up-down direction of the global coordinate system and the angle of the user's head at the time of landing in the left-right direction of the global coordinate system. These two pieces of feature data are acquired from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0119] One of the three pieces of feature data input to the learning models 30P to 30T corresponds to feature data indicating the characteristics of the movement of the user's forearm. In this embodiment, this piece of feature data is the variance of the acceleration of the user's forearm in the front-to-back direction of a local coordinate system based on the position of the sensor device 10B. This piece of feature data is acquired from sensor data indicating the movement of the user's forearm detected by the sensor device 10B. The variance of the acceleration of the user's forearm in the feature data may be the variance of the acceleration of the forearm during the user's walking cycle.

[0120] <Case C5> The control unit 26 may select case C5 when the sensor devices 10 that have transmitted sensor data to the electronic device 20 are only the sensor devices 10A, 10B, and 10D. Alternatively, the control unit 26 may select case C5 when the sensor devices 10A, 10B, and 10D are selected from the plurality of sensor devices 10 that have transmitted sensor data to the electronic device 20.

[0121] In case C5, when determining the evaluation for (1) the state of the head, (2) the state of the arms, (3) the state of the trunk, (4) the state of the knees, and (5) the state of the feet, the control unit 26 selects the learning models 30U, 30V, 30W, 30X, and 30Y, respectively.

[0122] In case C5, the feature data input to the learning models 30U to 30Y includes, in the same manner as or similar to case C1, feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. Furthermore, the feature data input to the learning models 30U to 30Y includes feature data indicating the characteristics of the user's forearm movement and feature data indicating the characteristics of the user's thigh movement.

[0123] In case C5 according to this embodiment, the control unit 26 inputs four pieces of feature data to each of the learning models 30U to 30Y.

[0124] Two of the four pieces of feature data input to the learning models 30U to 30Y correspond to feature data indicating the characteristics of the user's head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the user's head movement in the left-right direction of the global coordinate system. In this embodiment, these two pieces of feature data are the average value of the user's head angle in the up-down direction of the global coordinate system and the angle of the user's head at the time of landing in the left-right direction of the global coordinate system. These two pieces of feature data are acquired from sensor data indicating the user's head movement detected by the sensor device 10A. The average value of the user's head angle in the feature data may be the average value of the head angle during the user's walking cycle.

[0125] One of the four pieces of feature data input to the learning models 30U to 30Y corresponds to feature data indicating the characteristics of the movement of the user's forearm. In this embodiment, this piece of feature data is the maximum value of the acceleration of the user's forearm in the forward / backward direction of a local coordinate system based on the position of the sensor device 10B. This piece of feature data is acquired from sensor data indicating the movement of the user's forearm detected by the sensor device 10B. The maximum value of the acceleration of the user's forearm in the feature data may be the maximum value of the acceleration of the forearm in the user's walking cycle.

[0126] One of the four pieces of feature data input to the learning models 30U to 30Y corresponds to feature data indicating the characteristics of the movement of the user's thigh. In this embodiment, this piece of feature data is the maximum value of the acceleration of the user's foot in the up-down direction of a local coordinate system based on the position of the sensor device 10D. This piece of feature data is acquired from sensor data indicating the movement of the user's thigh detected by the sensor device 10D. The maximum value of the acceleration of the user's foot in the feature data may be the maximum value of the acceleration of the foot in the user's walking cycle.

[0127] [Method for generating a learning model] The method for generating a learning model is described below. A subject's gait database was used to generate the learning model. The subject's gait database was provided by "Yoshiyuki Kobayashi, Naoto Hida, Kanako Nakajima, Masahiro Fujimoto, and Masaaki Mochimaru, "2019: AIST Gait Database 2019," [Online], [Searched May 24, 2021], Internet: <https: / / unit.aist.go.jp / harc / ExPART / GDB2019_e.html>." This gait database contains the gait data of multiple subjects. The subjects' gait data was detected using a motion capture system and a ground reaction force sensor.

[0128] The walking of the subjects in the walking database was evaluated by an instructor who provided walking instruction. The instructor evaluated the walking of the subjects by assigning a numerical score from 1 to 5 for (1) head condition, (2) arm condition, (3) trunk condition, (4) knee condition, and (5) foot condition. The instructor evaluated the walking of the subjects based on the interpretation of the generally considered good walking style described above.

[0129] Feature data was obtained from data showing the subject's movements detected by a motion capture system. A dataset was generated by correlating the feature data with the scores assigned by the instructor. 980 datasets were generated using 10-step walking data from 98 subjects. A learning model was generated using cross-validation with these datasets. In the cross-validation method, 800 datasets, or 80% of the 980 datasets, were assigned as training datasets for the learning model. 180 datasets, or 20% of the 980 datasets, were assigned as evaluation datasets for the learning model. Using 10 different methods, the 980 datasets were divided into 80% training datasets and 20% evaluation datasets. The total number of trials was 846,000.

[0130] The accuracy and precision of the generated learning model were calculated. The accuracy and precision of the learning model were calculated based on the number of correct and incorrect answers in the learning model's estimation results. To determine whether the learning model's estimation results were correct or incorrect, the scores were divided into three levels. Specifically, the scores were divided into three levels: a score greater than 3, a score of 3, and a score less than 3. A score greater than 3 is also referred to as "Good." A score of 3 is also referred to as "Average." A score less than 3 is also referred to as "Poor." If the learning model's score of "Good" or "Poor" matched the score assigned by the instructor, the learning model's estimation results were determined to be correct. On the other hand, if the learning model's score of "Good" or "Poor" did not match the score assigned by the instructor, the learning model's estimation results were determined to be incorrect. If the learning model's score was 3, the learning model's estimation results were determined to be neither correct nor incorrect.

[0131] The accuracy of the learning model was calculated by dividing the number of correct estimation results by the sum of the number of correct estimation results and the number of incorrect estimation results. For example, the accuracy of the learning model was calculated using formula (1). Accuracy of learning model = (CR) / (CR + ICR) Formula (1) In formula (1), CR is the number of correct estimation results. ICR is the number of incorrect estimation results.

[0132] The accuracy of the learning model was calculated by dividing the number of correct prediction results by the number of all prediction results. For example, the accuracy of the learning model was calculated using formula (2). Accuracy of the learning model = (CR) / (CR + ICR + NR) Formula (2) In formula (2), CR is the number of correct prediction results. ICR is the number of incorrect prediction results. NR is the number of prediction results of the learning model that are neither correct nor incorrect. In other words, NR is the number of predictions of the learning model that have a score of 3.

[0133] Here, the inventors set the number of feature data input to the learning model to a minimum of three, and while changing the combination of feature data input to the learning model, sought a combination of feature data that would improve the evaluation of the accuracy and precision of the learning model. The inventors obtained the results shown in Figures 7 and 8.

[0134] Figure 7 shows a graph indicating the accuracy of the learning model. Figure 8 shows a graph indicating the accuracy of the learning model. Cases C1 to C5 shown in Figures 7 and 8 are the same as Cases C1 to C5 described above with reference to Figure 6, respectively. Figures 7 and 8 show the accuracy and accuracy of the learning model for each of the states from (1) head state to (5) foot state.

[0135] As shown in FIG. 7 , in case C1, among the accuracies of the learning model for each of the (1) head state to (5) foot state, the accuracy for the (1) head state was the highest. As shown in FIG. 8 , in case C1, among the accuracies of the learning model for each of the (1) head state to (5) foot state, the accuracy for the (1) head state was the highest. In case C1, the feature data input to the learning model is feature data indicating the characteristics of the subject's (user's) head movement. In case C1, by using the feature data indicating the characteristics of the user's head movement, it is considered that among the accuracies and accuracies of the learning model for each of the (1) head state to (5) foot state, the accuracy and accuracies for the (1) head state were the highest.

[0136] As shown in Figure 7, in case C1, the accuracy of the learning model for each of the states (2) to (5) of the feet, excluding the state (1) of the head, was 80% or higher. As shown in Figure 8, in case C1, the accuracy of the learning model for each of the states (2) to (5) of the feet, excluding the state (1) of the head, was 50% or higher. These results show that the state of body parts other than the user's head can be evaluated with a certain degree of accuracy using feature data that indicates the characteristics of the user's head movement.

[0137] Here, the movement of the user's head in the up-down and left-right directions of the global coordinate system reflects the movement of body parts other than the user's head while walking. For example, when a user swings their arms or kicks their feet while walking, the user's body moves in the up-down and left-right directions of the global coordinate system. When the user's body moves in the up-down and left-right directions of the global coordinate system, the user's head moves in the up-down and left-right directions of the global coordinate system.

[0138] In this way, the movement of the user's head in the up-down and left-right directions of the global coordinate system reflects the movement of body parts other than the user's head while walking. Therefore, it is inferred that if feature data indicating the characteristics of the user's head movement in the up-down and left-right directions of the global coordinate system are used as feature data to be input into the learning model, it is possible to evaluate the state of body parts other than the user's head while walking. As described above, in case C1, feature data indicating the characteristics of the user's head movement in the up-down and left-right directions of the global coordinate system is used as feature data to be input into the learning model. With this configuration, it is inferred that in case C1, the state of body parts other than the user's head can be evaluated with a certain degree of accuracy.

[0139] As shown in Figure 7, in case C2, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher than in case C1. Also, in case C2, the accuracy of the learning model for (5) foot state was higher than in cases C1, C3, and C4. As shown in Figure 8, in case C2, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher than in case C1. Also, in case C2, the accuracy of the learning model for (5) foot state was higher than in cases C1 and C3.

[0140] In Case C2, feature data indicating the movement characteristics of more body parts of the user is used as feature data input to the learning model compared to Case C1. For example, in Case C2, feature data indicating the movement characteristics of the user's head in each of the up-down and left-right directions of the global coordinate system, which is the same as or similar to Case C1, is used as feature data. In addition to these feature data, Case C2 also uses feature data indicating the movement characteristics of the user's feet. Because Case C2 uses feature data indicating the movement characteristics of more body parts of the user than Case C1, it is inferred that the accuracy and precision of the learning model for each of (1) head state to (5) foot state was higher than in Case C1. Furthermore, because Case C2 uses feature data indicating the movement characteristics of the user's feet, it is inferred that the accuracy and precision of the learning model for (5) foot state was higher than in Case C1, etc.

[0141] As shown in Figure 7, in case C3, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher than in case C1. Also, in case C3, the accuracy of the learning model for (4) thigh state was higher than in cases C1, C2, and C4. As shown in Figure 8, in case C3, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher than in case C1. Also, in case C3, the accuracy of the learning model for (4) thigh state was higher than in cases C1, C2, and C4.

[0142] In Case C3, feature data indicating the movement of more body parts of the user is used as feature data input to the learning model than in Case C1. For example, in Case C3, feature data indicating the movement characteristics of the user's head in each of the up-down and left-right directions of the global coordinate system, which is the same as or similar to that in Case C1, is used as feature data. In addition to these feature data, Case C3 also uses feature data indicating the movement characteristics of the user's thighs. Because Case C3 uses feature data indicating the movement characteristics of more body parts of the user than in Case C1, it is inferred that the accuracy and precision of the learning model for each of (1) the head state to (5) the foot state was higher than in Case C1. Furthermore, because Case C3 uses feature data indicating the movement characteristics of the user's thighs, it is inferred that the accuracy and precision of the learning model for (4) the thigh state was higher than in Case C1, etc.

[0143] As shown in Figure 7, in case C4, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher than in case C1. Also, in case C4, the accuracy of the learning model for (2) arm state was higher than in cases C1, C2, and C3. As shown in Figure 8, in case C4, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher than in case C1. Also, in case C4, the accuracy of the learning model for (2) arm state was higher than in cases C1, C2, and C3.

[0144] In Case C4, feature data indicating the movement of more body parts of the user is used as feature data input to the learning model compared to Case C1. For example, in Case C4, feature data indicating the movement characteristics of the user's head in each of the up-down and left-right directions of the global coordinate system, which is the same as or similar to Case C1, is used as feature data. In addition to these feature data, Case C4 also uses feature data indicating the movement characteristics of the user's forearm. Because Case C4 uses feature data indicating the movement characteristics of more body parts of the user than Case C1, it is inferred that the accuracy and precision of the learning model for each of (1) the head state to (5) the foot state was higher than in Case C1. Furthermore, because Case C4 uses feature data indicating the movement characteristics of the user's forearm, it is inferred that the accuracy and precision of the learning model for (2) the arm state was higher than in Case C1, etc.

[0145] As shown in Fig. 7, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher in Case C5 than in Case C1. As shown in Fig. 8, the accuracy of the learning model for each of the (1) head state to (5) foot state was higher in Case C5 than in Case C1.

[0146] In case C5, feature data indicating the features of more body parts of the user is used as feature data to be input into the learning model than in cases C1 to C4. Since feature data indicating the features of more body parts is used in case C5 than in cases C1 to C4, it is estimated that the precision and accuracy of the learning model for each of the states from (1) the head to (5) the foot are higher than in case C1.

[0147] (System Operation) Fig. 9 is a flowchart showing the operation of the evaluation process executed by the electronic device 20 shown in Fig. 1. This operation corresponds to an example of an information processing method according to this embodiment. For example, when the control unit 26 receives an input instructing the execution of a gait evaluation via the input unit 22, the control unit 26 starts the evaluation process from step S10.

[0148] The control unit 26 receives an input instructing execution of gait assessment via the input unit 22 (step S10). This input is entered via the input unit 22 by the user wearing the sensor device 10.

[0149] The control unit 26 transmits a signal instructing the start of data detection as a broadcast signal to the plurality of sensor devices 10 via the communication unit 21 (step S11). After the process of step S11 is executed, sensor data is transmitted from at least one sensor device 10 to the electronic device 20.

[0150] The control unit 26 receives sensor data from at least one sensor device 10 via the communication unit 21 (step S12).

[0151] The control unit 26 refers to the correspondence shown in FIG. 6, for example, and selects a learning model from among a plurality of learning models according to the type of the sensor device 10 that transmitted the sensor data to the electronic device 20 (step S13).

[0152] The control unit 26 acquires feature data from the sensor data received in step S12 (step S14). The control unit 26 inputs the feature data acquired in step S14 into the learning model selected in step S13, thereby acquiring a score such as that shown in Fig. 5 (step S15). By acquiring the score, the control unit 26 determines an evaluation of the condition of the body part corresponding to the score.

[0153] The control unit 26 generates an evaluation signal according to the determined evaluation (step S16). The control unit 26 transmits the evaluation signal generated in the process of step S16 to the external device via the communication unit 21 (step S17). After executing the process of step S17, the control unit 26 ends the evaluation process.

[0154] After completing the evaluation process, the control unit 26 may execute the evaluation process again when the user has walked a set number of steps. This set number of steps may be input in advance by the user via the input unit 22. When executing the evaluation process again, the control unit 26 may start the process from step S11. The control unit 26 may repeatedly execute the evaluation process each time the user has walked the set number of steps until it receives an input from the input unit 22 instructing the user to end the evaluation process. The input instructing the user to end the evaluation process is, for example, input from the input unit 22 by the user. For example, when the user finishes walking, the user inputs an input instructing the user to end the evaluation process via the input unit 22.

[0155] In this way, in the electronic device 20 serving as an information processing device, the control unit 26 can use the learning model to estimate the state of a body part of the user other than the body part on which the sensor device 10 is worn. As described above, for example, the control unit 26 can estimate the state of a body part of the user other than the body part on which the sensor device 10 is worn and determine an evaluation for the state of the body part. For example, the control unit 26 can select case C1 as shown in FIG. 6 and determine an evaluation for each of the states of (2) the arm, (3) the foot, and (4) the foot, which are other than the head on which the sensor device 10A is worn. With this configuration, in this embodiment, the state of any body part can be estimated, without being limited to the body part on which the sensor device 10 is worn.

[0156] In recent years, walking has been attracting attention as a convenient form of exercise. However, a user is required to pay attention to obstacles ahead or nearby while walking. Because a user is required to pay attention to obstacles ahead or nearby while walking, the user may not be able to pay attention to their own posture. If a user is unable to pay attention to their posture while walking, the user may walk with an incorrect posture without realizing it. If a user walks with an incorrect posture, the exercise effect of walking may be reduced. Furthermore, walking is often an exercise that users are familiar with, and it is often difficult for the user to correct their posture while walking.

[0157] As described above, in the electronic device 20 according to this embodiment, the control unit 26 can estimate the state of the user's body parts. This configuration can provide the user with an opportunity to correct their posture while walking. By providing the user with an opportunity to correct their posture while walking, the user can walk with correct posture. By enabling the user to walk with correct posture, the exercise effect of walking can be improved.

[0158] Therefore, according to this embodiment, a novel technique for estimating the state of a body part of a user while walking is provided.

[0159] Furthermore, the control unit 26 may use the learning model to estimate the states of more body parts than the number of body parts on which the user wears the sensor devices 10. For example, assume that the user wears N sensor devices 10 (N is an integer equal to or greater than 1). In this case, the control unit 26 may acquire N pieces of sensor data from the N sensor devices 10 and determine an evaluation of the states of N+1 or more body parts of the user.

[0160] As an example, assume that N is 1 and the user is wearing the sensor device 10A. In this case, the control unit 26 acquires one piece of sensor data from the sensor device 10A. The control unit 26 selects case C1 as shown in Fig. 6 and determines an evaluation of the states of two or more body parts, for example, five body parts from (1) the state of the head to (5) the state of the feet, based on the acquired single piece of sensor data.

[0161] As another example, assume that N is 2 and the user is equipped with sensor device 10A and sensor device 10D. In this case, the control unit 26 acquires two pieces of sensor data from each of sensor device 10A and sensor device 10D. The control unit 26 selects case C2 as shown in FIG. 6 and determines an evaluation of the states of three or more body parts, for example, five body parts from (1) the state of the head to (5) the state of the feet, based on the acquired two pieces of sensor data.

[0162] In this way, by estimating the states of more body parts than the number of body parts on which the sensor devices 10 are attached to the user, the electronic device 20 becomes more convenient for the user.

[0163] Furthermore, at least one sensor device 10 included in the information processing system 1 may include a sensor device 10A worn on the user's head. The control unit 26 may acquire sensor data indicating the user's head movement from the sensor device 10A. In this case, the control unit 26 may select case C1 as shown in FIG. 6 . That is, the feature data may include feature data indicating the characteristics of the user's head movement in the vertical direction of the user, i.e., the vertical direction of the global coordinate system, and feature data indicating the characteristics of the user's head movement in the horizontal direction of the user, i.e., the horizontal direction of the global coordinate system. With this configuration, even if the sensor device 10A is the only sensor device 10 worn by the user, it is possible to determine an evaluation of the condition of the user's body part. In other words, the user only needs to wear the sensor device 10A. This improves user convenience. Furthermore, if the sensor device 10A is an earphone or is included in the earphone, the user can easily wear the sensor device 10A on their head. The user's ability to easily wear the sensor device 10A on their head further improves user convenience. Furthermore, by using only the sensor data detected by the sensor device 10A, it is not necessary to synchronize the timing at which each of the multiple sensor devices 10 detects data. Since it is not necessary to synchronize the timing at which each of the multiple sensor devices 10 detects data, it is possible to more easily determine an evaluation of the condition of a user's body part.

[0164] Furthermore, the at least one sensor device 10 included in the information processing system 1 may include a sensor device 10A worn on the user's head and a sensor device 10D worn on the user's feet. The control unit 26 may acquire sensor data indicating the user's head movement from the sensor device 10A and sensor data indicating the user's foot movement from the sensor device 10D. In this case, the control unit 26 may select Case C2 as shown in FIG. 6. That is, the feature data may include feature data indicating the characteristics of the head movement in the up-down direction of the global coordinate system, feature data indicating the characteristics of the head movement in the left-right direction of the global coordinate system, and feature data indicating the characteristics of the foot movement. As described above with reference to FIGS. 7 and 8, Case C2 has higher accuracy and precision than Case C1 for the learning models for each of (1) the head state to (5) the foot state. Therefore, the control unit 26 can more accurately determine the evaluation of the condition of the user's body parts. Furthermore, if the sensor device 10A is an earphone or is included in an earphone, the user can easily wear the sensor device 10A on their head. Furthermore, when the sensor device 10D is a shoe-type wearable device, the user can easily wear the sensor device 10D on the foot, thereby improving user convenience.

[0165] Furthermore, the at least one sensor device 10 included in the information processing system 1 may include a sensor device 10A worn on the user's head and a sensor device 10C worn on the user's thigh. The control unit 26 may acquire sensor data indicating the user's head movement from the sensor device 10A and data indicating the user's thigh movement from the sensor device 10D. In this case, the control unit 26 may select case C3 as shown in FIG. 6. That is, the feature data may include feature data indicating the characteristics of the head movement in the up-down direction of the global coordinate system, feature data indicating the characteristics of the head movement in the left-right direction of the global coordinate system, and feature data indicating the characteristics of the thigh movement. As described above with reference to FIGS. 7 and 8, case C3 had higher accuracy and precision for each of the states (1) to (5) of the feet than case C1. Therefore, the control unit 26 can more accurately determine the evaluation of the state of the user's body parts. Furthermore, if the sensor device 10A is an earphone or is included in an earphone, the user can easily wear the sensor device 10A on their head. Such a configuration can improve convenience for the user.

[0166] Furthermore, the at least one sensor device 10 included in the information processing system 1 may include a sensor device 10A worn on the user's head and a sensor device 10B worn on the user's forearm. The control unit 26 may acquire sensor data indicating the user's head movement from the sensor device 10A and sensor data indicating the user's forearm movement from the sensor device 10B. In this case, the control unit 26 may select case C4 as shown in FIG. 6. That is, the feature data may include feature data indicating the characteristics of the head movement in the up-down direction of the global coordinate system, feature data indicating the characteristics of the head movement in the left-right direction of the global coordinate system, and feature data indicating the characteristics of the forearm movement. As described above with reference to FIGS. 7 and 8, in case C4, the accuracy and precision for each of (1) the head state to (5) the foot state were high. Therefore, the control unit 26 can more accurately determine the evaluation of the state of the user's body parts. Furthermore, if the sensor device 10A is an earphone or is included in an earphone, the user can easily wear the sensor device 10A on their head. Furthermore, if the sensor device 10B is a wristwatch-type wearable device, the user can easily wear the sensor device 10B on the forearm. Such a configuration can improve user convenience.

[0167] Furthermore, the at least one sensor device 10 included in the information processing system 1 may include a sensor device 10A worn on the user's head, a sensor device 10B worn on the user's forearm, and a sensor device 10D worn on the user's foot. The control unit 26 may acquire sensor data indicating the user's head movement from the sensor device 10A, sensor data indicating the user's forearm movement from the sensor device 10B, and sensor data indicating the user's foot movement from the sensor device 10D. In this case, the control unit 26 may select case C5 as shown in FIG. 6. That is, the feature data may include feature data indicating the characteristics of the head movement in the up-down direction of the global coordinate system and feature data indicating the characteristics of the head movement in the left-right direction of the global coordinate system. Furthermore, the feature data may include feature data indicating the characteristics of the forearm movement and feature data indicating the characteristics of the thigh movement. As described above with reference to FIGS. 7 and 8, case C5 provides higher accuracy and precision for each of the states (1) to (5) of the foot than case C1. Therefore, the control unit 26 can more accurately determine the evaluation of the condition of the user's body part.

[0168] (Configuration of Another System) FIG. 10 is a functional block diagram showing the configuration of an information processing system 101 according to another embodiment of the present disclosure.

[0169] The information processing system 101 includes a sensor device 10, an electronic device 20, and a server 40. In the information processing system 101, the server 40 functions as an information processing device and estimates the state of a body part of a user.

[0170] The electronic device 20 and the server 40 can communicate with each other via a network 2. The network 2 may be any network including a mobile communication network, the Internet, and the like.

[0171] The control unit 26 of the electronic device 20 receives sensor data from the sensor device 10 via the communication unit 21 in the same or similar manner as in the information processing system 1. In the information processing system 101, the control unit 26 transmits the sensor data to the server 40 via the network 2 via the communication unit 21.

[0172] The server 40 is, for example, a server that belongs to a cloud computing system or another computing system. The server 40 includes a communication unit 41, a storage unit 42, and a control unit 43.

[0173] The communication unit 41 includes at least one communication module that can be connected to the network 2. The communication module is, for example, a communication module that complies with standards such as a wired LAN (Local Area Network) or a wireless LAN. The communication unit 41 is connected to the network 2 via the wired LAN or wireless LAN by the communication module.

[0174] The storage unit 42 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 42 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 42 stores data used in the operation of the server 40 and data obtained by the operation of the server 40. For example, the storage unit 42 stores system programs, application programs, embedded software, etc. For example, the storage unit 42 stores a learning model and associations such as those shown in FIG. 6.

[0175] The control unit 43 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA or ASIC. The control unit 43 controls each part of the server 40 and executes processes related to the operation of the server 40.

[0176] The control unit 43 receives the sensor data from the electronic device 20 via the network 2 using the communication unit 41. The control unit 43 estimates the state of the user's body part based on the sensor data by executing the same or similar processing as the processing by the control unit 26 of the electronic device 20 described above.

[0177] (Operation of Other Systems) Fig. 11 is a sequence diagram showing the operation of the evaluation process executed by the information processing system 101 shown in Fig. 10. This operation corresponds to an example of an information processing method according to this embodiment. When the electronic device 20 receives an input instructing execution of a gait evaluation, the information processing system 101 starts the evaluation process from the processing of step S20. Hereinafter, it is assumed that the learning model is machine-learned to output a score when feature data is input.

[0178] In the electronic device 20, the control unit 26 receives an input instructing execution of gait assessment via the input unit 22 (step S20). The control unit 26 then transmits a signal instructing the start of data detection as a broadcast signal to the multiple sensor devices 10 via the communication unit 21 (step S21).

[0179] In the sensor device 10, the control unit 15 receives a signal from the electronic device 20 via the communication unit 11 instructing the start of data detection (step S22). Upon receiving this signal, the control unit 15 starts data detection. The control unit 15 acquires data detected by the sensor unit 12 from the sensor unit 12 and transmits the acquired data as sensor data to the electronic device 20 via the communication unit 11 (step S23).

[0180] In the electronic device 20, the control unit 26 receives the sensor data from the sensor device 10 via the communication unit 21 (step S24). The control unit 26 transmits the sensor data to the server 40 via the network 2 via the communication unit 21 (step S25).

[0181] In the server 40, the control unit 43 receives sensor data from the electronic device 20 via the network 2 through the communication unit 41 (step S26). The control unit 43 selects a learning model from among a plurality of learning models according to the type of sensor device 10 that transmitted the sensor data to the server 40 via the electronic device 20 (step S27). The control unit 43 acquires feature data from the sensor data received in the process of step S26 (step S28). The control unit 43 inputs the feature data acquired in the process of step S28 into the learning model selected in the process of step S27, thereby acquiring a score from the learning model (step S29). By acquiring the score, the control unit 43 determines an evaluation of the condition of the body part corresponding to the score.

[0182] In the server 40, the control unit 43 generates an evaluation signal corresponding to the determined evaluation (step S30). The control unit 43 transmits the evaluation signal generated in the process of step S30 to the electronic device 20 as an external device via the network 2 via the communication unit 41 (step S31).

[0183] In the electronic device 20, the control unit 26 receives an evaluation signal from the server 40 via the network 2 using the communication unit 21 (step S32). The control unit 26 then causes the notification unit to notify the content indicated by the evaluation signal (step S33). As an example of notification, the control unit 26 may cause the output unit 23 to output the content indicated by the evaluation signal. As another example of notification, the control unit 26 may cause the vibration unit 24 to vibrate with a vibration pattern corresponding to the evaluation signal. As yet another example of notification, the control unit 26 may transmit the evaluation signal to the sensor device 10 via the communication unit 21 and cause the sensor device 10 to notify the content indicated by the evaluation signal. In this case, in the sensor device 10, the control unit 15 may cause the output unit 13, which functions as a notification unit, to output the content indicated by the evaluation signal. When the sensor device 10A receives the evaluation signal, the control unit 15 in the sensor device 10A may cause the speaker of the output unit 13 to output the content indicated by the evaluation signal as sound.

[0184] After executing the process of step S33, the information processing system 101 ends the evaluation process.

[0185] After completing the evaluation process, the information processing system 101 may execute the evaluation process again when the user has walked the set number of steps. When executing the evaluation process again, the information processing system 101 may start the process of step S23. The information processing system 101 may repeatedly execute the evaluation process each time the user has walked the set number of steps until the electronic device 20 receives an input from the input unit 22 instructing the end of the evaluation process.

[0186] The information processing system 101 can achieve the same or similar effects as the information processing system 1 .

[0187] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art could make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0188] For example, the learning model may be trained so that, when sensor data or feature data is input, it outputs information about the state of a body part of the user that is different from the body part to which the sensor device 10 is attached. In this case, the control unit 26 estimates the state of the body part of the user that is different from the body part to which the sensor device 10 is attached, based on the sensor data and the learning model.

[0189] For example, the feature data input into the learning model is not limited to those described above. For example, from the results shown in Figures 7 and 8, it can be seen that the more body parts to which the sensor devices 10 are attached on the user, the more types of feature data indicating the movement characteristics of the body parts are provided, and the higher the accuracy and precision of the learning model. Therefore, in order to improve the accuracy and precision of the learning model, the number of feature data input into the learning model may be increased by increasing the number of sensor devices 10 attached to the user's body parts.

[0190] For example, the electronic device 20 may be a glasses-type wearable device. In this case, the output unit 23 may be configured to include a projector that projects an image onto the lenses of the glasses. The control unit 26 may cause the output unit 23 to output the evaluation of the determined state of the body part as an image. This image may include, for example, an image showing the ideal movement of a body part that is evaluated low among the user's multiple body parts.

[0191] For example, the evaluation threshold may be set based on the age, sex, etc. of the user. In this case, the content of the praise and advice to the user may be set according to the age, sex, etc. of the user.

[0192] For example, the storage unit 25 of the electronic device 20 may store a learning model for each piece of physical data capable of distinguishing between the physical characteristics of multiple users. The physical data may include, for example, at least one of age, gender, height, and weight. In this case, the control unit 26 may receive input indicating the user's physical data from the input unit 22. The control unit 26 may select a learning model corresponding to the received user's physical data from among the multiple learning models stored in the storage unit 25. This configuration makes it possible to evaluate the condition of a body part that matches the individual's physical data.

[0193] For example, the memory unit 42 of the server 40 may store a learning model for each of the above-mentioned physical data. In this case, the control unit 26 of the electronic device 20 may receive an input indicating the user's physical data from the input unit 22. In the electronic device 20, when the control unit 26 receives the input indicating the user's physical data from the input unit 22, the control unit 26 may transmit a signal indicating the user's physical data to the server 40 via the network 2 using the communication unit 21. In the server 40, the control unit 43 receives the signal indicating the user's physical data from the electronic device 20 via the network 2 using the communication unit 41. In the server 40, the control unit 43 may select a learning model corresponding to the user's physical data from among the multiple learning models stored in the memory unit 42 based on the received signal indicating the user's physical data. With this configuration, it is possible to evaluate the condition of a body part that matches an individual's physical data.

[0194] For example, the control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may calculate the rhythm of the user's body movement, the user's stride length, and the user's walking speed based on the sensor data. The control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may calculate the walking time of the user based on the sensor data. The control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may calculate the walking distance of the user based on the sensor data. The control unit 26 of the electronic device 20 or the control unit 43 of the server 40 may generate a signal prompting the user to take a break or to end walking when the walking time exceeds a time threshold or the walking distance exceeds a distance threshold. In the electronic device 20, the control unit 26 may transmit the generated signal to an external device such as the one described above via the communication unit 21. In the server 40, the control unit 43 may transmit the generated signal to the electronic device 20 or the sensor device 10 via the network 2 via the communication unit 41. The time threshold may be set based on, for example, an average walking time of a typical user. The distance threshold may be set based on the average distance that a typical user walks in one go, or the like.

[0195] For example, the communication unit 21 of the electronic device 20 may be configured to include at least one receiving module compatible with a satellite positioning system. The receiving module is, for example, a receiving module compatible with GPS (Global Positioning System). However, the receiving module is not limited to this. The receiving module may be a receiving module compatible with any satellite positioning system. In this case, the memory unit 25 may store map data. Furthermore, the control unit 26 may acquire user location information via the communication unit 21. The control unit 26 may cause the output unit 23 to output the user location information and the map data.

[0196] For example, the communication unit 11 of the sensor device 10 may further include at least one communication module connectable to a network 2 as shown in Fig. 10. The communication module is, for example, a communication module compatible with a mobile communication standard such as LTE, 4G, or 5G. In this case, in the information processing system 101 as shown in Fig. 10, the control unit 15 of the sensor device 10 may transmit data detected by the sensor device 10 to a server 40 via the network 2 using the communication unit 11.

[0197] For example, in the above-described embodiment, the control unit 26 of the electronic device 20 may estimate an overall state that combines two or more body parts of the user. However, the control unit 26 of the electronic device 20 or the control unit 43 of the server may determine an overall evaluation that combines two or more of the above-described (1) head state, (2) arm state, (3) trunk state, (4) knee state, and (5) foot state.

[0198] For example, an embodiment is also possible in which a general-purpose computer functions as the electronic device 20 according to the present embodiment. Specifically, a program describing the processing content for realizing each function of the electronic device 20 according to the present embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the configuration according to the present embodiment can also be realized as a program executable by a processor or a non-transitory computer-readable medium storing the program.

[0199] 1, 101 Information processing system 2 Network 10, 10A, 10B, 10C, 10D Sensor device 11 Communication unit 12 Sensor unit 13 Output unit 14 Memory unit 15 Control unit 20 Electronic device 21 Communication unit 22 Input unit 23 Output unit 24 Vibration unit 25 Memory unit 26 Control unit 30, 30A to 30E, 30F to 30J, 30K to 30O, 30P to 30T, 30U to 30Y Learning model 31 Input layer 32 Hidden layer 33 Hidden layer 34 Output layer 40 Server 41 Communication unit 42 Memory unit 43 Control unit

Claims

1. acquiring sensor data indicative of a movement of a body part of a user from at least one sensor device attached to the body part of the user; An information processing device comprising: a control unit that estimates a state of a body part of the user other than the body part to which the sensor device is attached, using the acquired sensor data and a learning model.

2. The information processing device according to claim 1 , wherein the control unit estimates, by the learning model, the states of a greater number of body parts than the number of body parts to which the sensor devices are attached on the user.

3. When the sensor data is input, the learning model outputs evaluation information on a state of a body part of the user that is different from the body part to which the sensor device is attached; The information processing device according to claim 1 , wherein the control unit estimates the state of the user's body part by acquiring information on the evaluation from the learning model, and determines an evaluation of the state of the user's body part.

4. when feature data indicating features of the movement of the body part is input, the learning model outputs evaluation information on a state of a body part of the user that is different from the body part to which the sensor device is attached; The control unit is acquiring the feature data from the sensor data; The information processing device according to claim 1 , further comprising: determining an evaluation of the state of the user's body part by acquiring information on the evaluation from the learning model, in order to estimate the state of the user's body part.

5. The at least one sensor device includes a sensor device to be worn on the head of the user, The control unit acquires the sensor data indicating a movement of the user's head, The information processing device according to claim 4 , wherein the feature data includes feature data indicating a feature of movement of the head of the user in at least one of a front-back direction, a left-right direction, and an up-down direction.

6. The information processing device according to claim 5 , wherein the feature data includes feature data indicating a feature of the user's head movement in an up-down direction and feature data indicating a feature of the user's head movement in a left-right direction.

7. The at least one sensor device further includes a sensor device attached to a foot of the user, The control unit further acquires the sensor data indicating a movement of the user's foot, The information processing device according to claim 6 , wherein the feature data further includes feature data indicating a feature of the movement of the foot.

8. The at least one sensor device further includes a sensor device attached to a thigh of the user, The control unit further acquires the sensor data indicating a movement of the user's thigh, The information processing device according to claim 6 , wherein the feature data further includes feature data indicating a feature of the movement of the thigh.

9. The at least one sensor device further includes a sensor device attached to a forearm of the user, The control unit further acquires the sensor data indicating a movement of the user's forearm, The information processing device according to claim 6 , wherein the feature data further includes feature data indicating a feature of the movement of the forearm.

10. The at least one sensor device further includes a sensor device attached to a forearm of the user and a sensor device attached to a foot of the user; The control unit further acquires the sensor data indicating a movement of the user's forearm and the sensor data indicating a movement of the user's foot, The information processing apparatus according to claim 6 , wherein the feature data further includes feature data indicating a feature of the movement of the forearm, and feature data indicating a feature of the movement of the foot.

11. The information processing device according to claim 3 , wherein the control unit determines an evaluation of at least one of a state of the user's head, an state of the arms, a state of the trunk, a state of the knees, and a state of the feet.

12. The information processing device according to claim 3 , wherein the control unit selects a learning model for acquiring the evaluation information from among the plurality of learning models depending on a type of the sensor device that transmitted the sensor data to the information processing device.

13. Further comprising a communication unit, The control unit is generating an evaluation signal responsive to the at least one evaluation; The information processing apparatus according to claim 3 , wherein the generated evaluation signal is transmitted to an external device by the communication unit.

14. The information processing apparatus according to claim 13 , wherein the external device is an earphone.

15. An electronic device comprising: a notification unit that notifies information on the state of the body part estimated by the information processing device according to claim 1 .

16. At least one sensor device attached to a body part of a user; and an information processing device that acquires sensor data indicating the movement of the body part from the sensor device, and estimates the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.

17. acquiring sensor data indicative of a movement of a body part of a user from at least one sensor device attached to the body part; and estimating the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.

18. On the computer, acquiring sensor data indicative of a movement of a body part of a user from at least one sensor device attached to the body part; and estimating the state of a body part of the user other than the body part to which the sensor device is attached using the acquired sensor data and a learning model.