Walking posture state estimation system, walking posture state estimation method, walking posture state estimation device, and control program

Abstract

Provided are a walking posture state estimation system, a walking posture state estimation method, a walking posture state estimation device, and a control program that are capable of estimating a walking posture state with increased accuracy. The walking posture state estimation system comprises: a first wearable device that can be worn by a user and acquires first data that is obtained when the user walks; a second wearable device that can be worn by the user and acquires second data that is different from the first data obtained when the user walks; and an information processing device that can receive the first data and the second data and output an estimated walking posture state of the user wearing the first wearable device and the second wearable device. The information processing device includes a storage unit that stores a trained model capable of outputting the walking posture state estimated on the basis of the first data and the second data.

Description

Walking posture state estimation system, walking posture state estimation method, walking posture state estimation device, and control program

[0001] The present disclosure relates to a walking posture state estimation system, a walking posture state estimation method, a walking posture state estimation device, and a control program.

[0002] Conventionally, systems have been developed that analyze a user's walking style, posture habits, etc., and propose improvement methods. In order to propose appropriate improvement methods in such systems, it is necessary to estimate the user's walking style, posture habits, etc. with higher accuracy.

[0003] Patent Document 1 discloses an exercise posture derivation device that is worn on the user's body and derives walking or running posture.

[0004] Japanese Patent Application Laid-Open No. 2021-98027

[0005] There is a demand for providing a system that can estimate walking posture state with higher accuracy.

[0006] The purpose of the walking posture state estimation system, walking posture state estimation method, walking posture state estimation device, and control program is to enable more accurate estimation of the walking posture state.

[0007] A walking posture estimation system according to an embodiment includes a first wearable device that can be worn by a user and acquires first data obtained when the user walks, a second wearable device that can be worn by a user and acquires second data that is different from the first data obtained when the user walks, and an information processing device that can receive the first data and the second data and output an estimated walking posture state of a user wearing the first wearable device and the second wearable device, and the information processing device has a memory unit that stores a trained model that can output an estimated walking posture state based on the first data and the second data.

[0008] In the walking posture estimation system according to the embodiment, it is preferable that the first wearable device has a receiving unit that receives second data from the second wearable device, and a transmitting unit that transmits the first data and second data for a predetermined number of steps of the user to the information processing device.

[0009] In the walking posture estimation system according to the embodiment, the walking posture state preferably includes one or more of the state of stride length, the state of center of gravity position, the state of the swinging leg, the state of the supporting leg, and the state of the hip joint.

[0010] In the walking posture estimation system according to the embodiment, it is preferable that the trained model includes a first trained model capable of outputting a first walking posture state estimated based on first data, and a second trained model capable of outputting a second walking posture state estimated based on second data.

[0011] In the walking posture estimation system according to the embodiment, it is preferable that the trained model is a single trained model capable of outputting a walking posture state estimated based on the first data and the second data.

[0012] In the walking posture estimation system according to the embodiment, it is preferable that the trained model includes a first trained model capable of outputting a first walking posture state estimated based on first image data based on first data, and a second trained model capable of outputting a second walking posture state estimated based on second image data based on second data.

[0013] In the walking posture estimation system according to the embodiment, it is preferable that the trained model is a single trained model capable of outputting a walking posture state estimated based on image data based on the first data and the second data.

[0014] In the walking posture estimation system according to the embodiment, it is preferable that the information processing device acquires the walking posture state by inputting data based on the first data into a first layer of a first trained model included in the trained model, and inputting data based on the second data into a second layer different from the first layer of a second trained model included in the trained model.

[0015] In the walking posture estimation system according to the embodiment, it is preferable that the walking posture state includes information output from a first layer of the trained model when data based on the first data is input to the trained model, and information output from a second layer different from the first layer of the trained model when data based on the second data is input to the trained model.

[0016] The walking posture estimation method according to the embodiment acquires first data obtained when the user walks using a first wearable device that can be worn by the user, acquires second data different from the first data obtained when the user walks using a second wearable device that can be worn by the user, and estimates the walking posture state of a user wearing the first wearable device and the second wearable device based on the acquired first data and second data using a trained model that can output a walking posture state estimated based on the first data and the second data, and outputs the walking posture state.

[0017] A walking posture estimation device according to an embodiment includes an acquisition unit that acquires first data and second data from a first sensor that measures first data obtained when a user walks and a second sensor that measures second data different from the first data obtained when the user walks, and an estimation unit that acquires an estimated walking posture state based on the acquired first data and second data using a trained model that can output an estimated walking posture state based on the first data and second data.

[0018] A walking posture estimation method according to an embodiment acquires first data and second data from a first sensor that acquires first data obtained when a user walks and a second sensor that acquires second data different from the first data obtained when the user walks, and uses a trained model that can output a walking posture state estimated based on the first data and the second data to acquire an estimated walking posture state based on the acquired first data and second data.

[0019] A control program according to the embodiment is a control program for a walking posture state estimation device, and causes the walking posture state estimation device to acquire first data and second data from a first sensor that acquires first data obtained when a user walks and a second sensor that acquires second data different from the first data obtained when the user walks, and to acquire an estimated walking posture state based on the acquired first data and second data using a trained model that can output the estimated walking posture state based on the acquired first data and second data.

[0020] A walking posture estimation device according to an embodiment includes a first sensor that measures first data obtained when a user walks, an acquisition unit that acquires second data from a second sensor that measures second data different from the first data obtained when the user walks, a memory unit that stores a trained model that can output a walking posture state estimated based on the first data and the second data, and an estimation unit that uses the trained model to acquire a walking posture state estimated based on the acquired first data and second data.

[0021] A walking posture estimation method according to an embodiment is a walking posture state estimation method using a walking posture state estimation device having a first sensor that acquires first data obtained when a first user walks, and acquires second data from a second sensor that acquires second data different from the first data obtained when the user walks, stores a trained model that can output a walking posture state estimated based on the first data and the second data, and uses the trained model to acquire a walking posture state estimated based on the acquired first data and second data.

[0022] A control program according to an embodiment is a control program for a walking posture state estimation device having a first sensor that acquires first data obtained when a first user walks, and causes the walking posture state estimation device to acquire second data from a second sensor that acquires second data different from the first data obtained when the user walks, store a trained model that can output a walking posture state estimated based on the first data and the second data, and use the trained model to acquire a walking posture state estimated based on the acquired first data and second data.

[0023] The walking posture estimation system according to the embodiment includes a sole information acquisition device that acquires static information about the soles of the user's feet, a wearable device that can be worn by the user and acquires dynamic information obtained when the user walks, and an information processing device connected to the sole information acquisition device and the wearable device. The information processing device includes a receiving unit that receives the static information and dynamic information, a memory unit that stores a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, and an output unit that outputs the walking posture state.

[0024] In the walking posture estimation system according to the embodiment, it is preferable that the system further includes a second wearable device that can be worn by the user and acquires second dynamic information obtained when the user walks, and that the first wearable device includes a second dynamic information receiving unit that receives the second dynamic information from the second wearable device, and a transmitting unit that transmits the dynamic information for a predetermined number of steps of the user and the second dynamic information to the information processing device.

[0025] In the walking posture estimation system according to the embodiment, the walking posture state preferably includes one or more of the state of stride length, the state of center of gravity position, the state of the swinging leg, the state of the supporting leg, and the state of the hip joint.

[0026] In the walking posture estimation system according to the embodiment, it is preferable that the trained model includes a first trained model capable of outputting a first walking posture state estimated based on static information, and a second trained model capable of outputting a second walking posture state estimated based on dynamic information.

[0027] In the walking posture estimation system according to the embodiment, the trained model is preferably a single trained model capable of outputting a walking posture state estimated based on static information and dynamic information.

[0028] In the walking posture estimation system according to the embodiment, it is preferable that the information processing device acquires the walking posture state by inputting information based on static information into a first layer of a first trained model included in the trained models, and inputting information based on dynamic information into a second layer different from the first layer of a second trained model included in the trained models.

[0029] In the walking posture estimation system according to the embodiment, it is preferable that the walking posture state includes information output from a first hierarchical layer of the trained model when information based on static information is input to the trained model, and information output from a second hierarchical layer different from the first hierarchical layer of the trained model when information based on dynamic information is input to the trained model.

[0030] The walking posture estimation method according to the embodiment acquires static information about the soles of the user's feet using a sole information acquisition device, acquires dynamic information obtained when the user walks using a wearable device that can be worn by the user, and estimates and outputs the walking posture state based on the acquired static and dynamic information using a trained model that can output the walking posture state of the user estimated based on the static and dynamic information.

[0031] The walking posture estimation device according to the embodiment includes an acquisition unit that acquires static information and dynamic information from a static information sensor that measures static information related to the soles of the user's feet and a dynamic information sensor that measures dynamic information obtained when the user walks, an estimation unit that acquires an estimated walking posture state based on the acquired static information and dynamic information using a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, and an output unit that outputs the walking posture state.

[0032] The walking posture estimation method according to the embodiment acquires static information and dynamic information from a static information sensor that measures static information about the soles of the user's feet and a dynamic information sensor that measures dynamic information obtained when the user walks, and uses a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, acquires the walking posture state estimated based on the acquired static information and dynamic information, and outputs the walking posture state.

[0033] The control program according to the embodiment is a control program for a walking posture state estimation device, and causes the walking posture state estimation device to acquire static information and dynamic information from a static information sensor that measures static information about the soles of the user's feet and a dynamic information sensor that measures dynamic information obtained when the user walks, utilize a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, acquire the walking posture state estimated based on the acquired static information and dynamic information, and output the walking posture state.

[0034] The walking posture estimation device according to the embodiment includes an acquisition unit that acquires static information from a static information sensor that measures static information related to the soles of the user's feet, a sensor that measures dynamic information obtained when the user walks, an estimation unit that acquires the walking posture state estimated based on the acquired static information and dynamic information using a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, and an output unit that outputs the walking posture state.

[0035] A walking posture estimation method according to an embodiment is a walking posture state estimation method using a walking posture state estimation device having a sensor that measures dynamic information obtained when a user walks, and acquires static information from a static information sensor that measures static information related to the soles of the user's feet, and uses a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, acquires the walking posture state estimated based on the acquired static information and dynamic information, and outputs the walking posture state.

[0036] The control program according to the embodiment is a control program for a walking posture state estimation device having a sensor that measures dynamic information obtained when a user walks, and causes the walking posture state estimation device to acquire static information from a static information sensor that measures static information about the soles of the user's feet, utilize a trained model that can output the walking posture state of the user estimated based on the static information and dynamic information, acquire the walking posture state estimated based on the acquired static information and dynamic information, and output the walking posture state.

[0037] The walking posture state estimation system, walking posture state estimation method, walking posture state estimation device, and control program can estimate the walking posture state with higher accuracy.

[0038] The objects and advantages of the invention will be realized and obtained by means of the elements and combinations particularly pointed out in the claims. Both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

[0039] 1 is a diagram showing a schematic configuration of a walking posture state estimation system 1 according to an embodiment. FIG. 2 is a diagram showing a schematic configuration of a first wearable device 100. FIG. 3 is a diagram showing a schematic configuration of a second wearable device 200. FIG. 4 is a diagram showing a schematic configuration of an information processing device 300. FIG. 5 is a schematic diagram showing an example of the data structure of a data table 312. FIG. 6 is a sequence showing an example of the operation of the estimation process. FIG. 7 is a sequence showing an example of the operation of another estimation process. FIG. 8 is a sequence showing an example of the operation of yet another estimation process. FIG. 9 is a diagram showing a schematic configuration of a walking posture state estimation system 2 according to another embodiment. FIG. 10 is a diagram showing a schematic configuration of a sole information acquisition device 500. FIG. 11 is a sequence showing an example of the operation of yet another estimation process. FIG. 12 is a sequence showing an example of the operation of yet another estimation process.

[0040] Hereinafter, a walking posture state estimation system, a walking posture state estimation method, a walking posture state estimation device, and a control program according to one aspect of an embodiment will be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to the embodiment, but extends to the inventions set forth in the claims and their equivalents.

[0041] FIG. 1 is a diagram showing a schematic configuration of a walking posture state estimation system 1 according to an embodiment.

[0042] As shown in Fig. 1, the walking posture state estimation system 1 includes a first wearable device 100, one or more second wearable devices 200, an information processing device 300, and a server device 400. The first wearable device 100 and each second wearable device 200 are worn by a user. The information processing device 300 is used by an administrator or the like who manages the user's health. The server device 400 is, for example, a server or the like located on a cloud network.

[0043] The first wearable device 100 and each second wearable device 200 are connected to each other so as to be able to communicate with each other via a wireless network such as Bluetooth (registered trademark) or a wireless local area network (LAN). The first wearable device 100, the information processing device 300, and the server device 400 are connected to each other so as to be able to communicate with each other via a network N. The network N is a wired network such as the Internet or an intranet. The network N may also be a wireless network such as a wireless local area network (LAN). The second wearable devices 200, the information processing device 300, and the server device 400 may also be connected to each other so as to be able to communicate with each other via the network N.

[0044] FIG. 2 is a diagram showing a schematic configuration of the first wearable device 100.

[0045] The first wearable device 100 is an example of a walking state estimation device. The first wearable device 100 is a terminal device that can be worn by a user, such as a multi-function mobile phone (a so-called smartphone) or a tablet PC. The first wearable device 100 includes a first input device 101, a first display device 102, a first communication device 103, a first interface device 104, a first sensor 105, a first storage device 110, and a first processing device 120. The first input device 101, the first display device 102, the first communication device 103, the first interface device 104, the first sensor 105, the first storage device 110, and the first processing device 120 are connected to each other via a CPU (Central Processing Unit) bus or the like.

[0046] The first input device 101 has an input device such as a touch panel and an interface circuit that acquires signals from the input device, and outputs operation signals in response to input operations by the user.

[0047] The first display device 102 is an example of an output unit. The first display device 102 has a display such as a liquid crystal display, an organic electroluminescence (EL) display, or the like, and an interface circuit that outputs image data to the display, and displays the image data on the display.

[0048] The first communication device 103 is an example of an output unit. The first communication device 103 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit that complies with a communication protocol such as wireless LAN. The first communication device 103 is communicatively connected to the network N in accordance with a communication standard such as wireless LAN. The first communication device 103 sends data received from the information processing device 300, the server device 400, or the like via the network N to the first processing device 120. The first communication device 103 also transmits data received from the first processing device 120 to the information processing device 300, the server device 400, or the like via the network N.

[0049] The first interface device 104 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit conforming to a communication protocol such as Bluetooth (registered trademark). The first interface device 104 is communicatively connected to each second wearable device 200 in accordance with a communication standard such as Bluetooth (registered trademark). The first interface device 104 sends data received from each second wearable device 200 to the first processing device 120. The first interface device 104 also transmits data received from the first processing device 120 to each second wearable device 200.

[0050] The first sensor 105 is an example of a sensor and a dynamic information sensor. The first sensor 105 includes an acceleration sensor that measures acceleration in three axial directions applied to the first wearable device 100, and a gyro sensor that measures angular velocity in three axial directions applied to the first wearable device 100. The acceleration sensor and the gyro sensor output measurement signals indicating the measured acceleration and angular velocity to the first processing device 120.

[0051] The first sensor 105 may include a geomagnetic sensor that detects the geomagnetism acting on the first wearable device 100, i.e., the movement of the user wearing the first wearable device 100. The first sensor 105 may also include a barometric pressure sensor that detects the atmospheric pressure around the first wearable device 100, i.e., the vertical movement of the user wearing the first wearable device 100. The first sensor 105 may also include a Global Positioning System (GPS) sensor that detects the position of the first wearable device 100, i.e., the movement of the user wearing the first wearable device 100. In this case, the first sensor 105 outputs a measurement signal indicating the detected geomagnetism, barometric pressure, or position to the first processing device 120.

[0052] The first storage device 110 is an example of a storage unit. The first storage device 110 includes a memory device such as a random access memory (RAM) or a read-only memory (ROM), a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or optical disk. The first storage device 110 also stores computer programs, databases, tables, and the like used for various processes of the first wearable device 100. The computer programs may be installed into the first storage device 110 from a computer-readable portable recording medium using a known setup program or the like. The portable recording medium is, for example, a CD-ROM (compact disc read-only memory) or a DVD-ROM (digital versatile disc read-only memory). The computer programs may be stored in a recording medium owned by a predetermined server and installed via a network N.

[0053] The first processing device 120 operates based on a program stored in advance in the first storage device 110. The first processing device 120 is, for example, a CPU. A digital signal processor (DSP), a large scale integration (LSI), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like may be used as the first processing device 120. The first processing device 120 is connected to the first input device 101, the first display device 102, the first communication device 103, the first interface device 104, the first sensor 105, the first storage device 110, and the like, and controls each of these devices. The first processing device 120 acquires data measured by the first sensor 105 when the user walks. The first processing device 120 also acquires data measured by the second wearable device 200 from the second wearable device 200 via the first interface device 104 when the user holding the second wearable device 200 walks. The first processing device 120 transmits the acquired data to the information processing device 300 via the first communication device 103 .

[0054] The first processing device 120 reads the computer program stored in the first storage device 110 and operates in accordance with the read computer program. As a result, the first processing device 120 functions as a first acquisition unit 121 and a first transmission unit 122. The first acquisition unit 121 is an example of a receiving unit, and the first transmission unit 122 is an example of a transmitting unit.

[0055] FIG. 3 is a diagram showing a schematic configuration of the second wearable device 200.

[0056] The second wearable device 200 is a terminal device that can be worn by a user. For example, the second wearable device 200 is a wristwatch-type wearable computer worn on the user's wrist. Alternatively, the second wearable device 200 is an earphone-type wearable computer worn on the user's ear. Alternatively, the second wearable device 200 is an insole-type wearable computer worn inside the user's shoe. The second wearable device 200 includes a second input device 201, a second display device 202, a second communication device 203, a second interface device 204, a second sensor 205, a second storage device 210, and a second processing device 220. The second input device 201, the second display device 202, the second communication device 203, the second interface device 204, the second sensor 205, the second storage device 210, and the second processing device 220 are connected to each other via a CPU bus or the like.

[0057] The second input device 201 has an input device such as a button and an interface circuit that acquires a signal from the input device, and outputs an operation signal in response to an input operation by the user. The second input device 201 may be omitted.

[0058] The second display device 202 has a display such as a liquid crystal display, an organic electroluminescence display, or the like, and an interface circuit that outputs image data to the display, and displays the image data on the display. The second display device 202 may have an LED (Light Emitting Diode) or the like, and may notify the user by turning it on or off. The second display device 202 may be omitted.

[0059] The second communication device 203 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit that complies with a communication protocol such as a wireless LAN. The second communication device 203 is communicatively connected to the network N in accordance with a communication standard such as a wireless LAN. The second communication device 203 sends data received from the information processing device 300, the server device 400, or the like via the network N to the second processing device 220. The second communication device 203 also transmits data received from the second processing device 220 to the information processing device 300, the server device 400, or the like via the network N.

[0060] The second interface device 204 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit that complies with a communication protocol such as Bluetooth (registered trademark). The second interface device 204 is communicatively connected to the first wearable device 100 in accordance with a communication standard such as Bluetooth (registered trademark). The second interface device 204 sends data received from the first wearable device 100 to the second processing device 220. The second interface device 204 also transmits data received from the second processing device 220 to the first wearable device 100.

[0061] The second sensor 205 is an example of a dynamic information sensor. The second sensor 205 includes an acceleration sensor that measures acceleration in three axial directions applied to the second wearable device 200 and a gyro sensor that measures angular velocity in three axial directions applied to the second wearable device 200. The acceleration sensor and gyro sensor output measurement signals indicating the measured acceleration and angular velocity to the second processing device 220. The second sensor 205 may include an insole-type pressure sensor. In this case, the second sensor 205 includes resistance-change or capacitance-change pressure sensors arranged one-dimensionally or two-dimensionally, generates sole data indicating the magnitude of pressure measured by each pressure sensor, and outputs the sole data to the second processing device 220. The sole data is, for example, a sole image in which the magnitude of pressure measured by each pressure sensor is used as the grayscale value of each pixel. The sole data may also be, for example, a sole vector whose elements are the pressures measured by each pressure sensor.

[0062] Second storage device 210 includes a memory device such as RAM or ROM, a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or optical disk. Second storage device 210 also stores computer programs, databases, tables, and the like used for various processes of second wearable device 200. The computer programs may be installed into second storage device 210 from a computer-readable portable recording medium using a known setup program or the like. Portable recording media include, for example, CD-ROMs and DVD-ROMs. The computer programs may be stored in a recording medium owned by a predetermined server and installed via network N.

[0063] The second processing device 220 operates based on a program stored in advance in the second storage device 210. The second processing device 220 is, for example, a CPU. A DSP, an LSI, an ASIC, an FPGA, etc. may be used as the second processing device 220. The second processing device 220 is connected to the second input device 201, the second display device 202, the second communication device 203, the second interface device 204, the second sensor 205, the second storage device 210, etc., and controls each device. The second processing device 220 acquires data measured by the second sensor 205 when the user walks and transmits the data to the first wearable device 100 via the second interface device 204.

[0064] The second processing device 220 reads the computer program stored in the second storage device 210 and operates in accordance with the read computer program. As a result, the second processing device 220 functions as a second acquisition unit 221 and a second transmission unit 222.

[0065] FIG. 4 is a diagram showing a schematic configuration of the information processing device 300.

[0066] The information processing device 300 is an example of a walking posture state estimation device. The information processing device 300 includes a third communication device 301, a third storage device 310, and a third processing device 320. The third communication device 301, the third storage device 310, and the third processing device 320 are connected to each other via a CPU bus or the like.

[0067] The third communication device 301 is an example of an output unit. The third communication device 301 has a wired communication interface circuit conforming to a communication protocol such as TCP / IP. The third communication device 301 is communicatively connected to the network N in accordance with a communication standard such as Ethernet (registered trademark). The third communication device 301 sends data received from the first wearable device 100, the second wearable device 200, the server device 400, etc. via the network N to the third processing device 320. The third communication device 301 transmits data received from the third processing device 320 to the first wearable device 100, the second wearable device 200, the server device 400, etc. via the network N. The third communication device 301 may have an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit conforming to a communication protocol such as a wireless LAN, and may be communicatively connected to the network N in accordance with a communication standard such as a wireless LAN.

[0068] The third storage device 310 includes a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or an optical disk. The third storage device 310 also stores computer programs, databases, tables, and the like used for various processes of the information processing device 300. The computer programs may be installed into the third storage device 310 from a computer-readable portable recording medium such as a CD-ROM or a DVD-ROM using a known setup program or the like. The computer programs may be stored in a recording medium owned by a predetermined server and installed via the network N.

[0069] The third storage device 310 stores data such as a trained model 311 and a data table 312. The trained model 311 is a model for estimating the walking posture state of a user wearing the first wearable device 100 and the second wearable device 200. The data table 312 stores information measured by the first sensor 105 of the first wearable device 100 or the second sensor 205 of the second wearable device 200. Details of the data table 312 will be described later.

[0070] The third processing device 320 operates based on a program stored in advance in the third storage device 310. The third processing device 320 is, for example, a CPU. A DSP, an LSI, an ASIC, an FPGA, etc. may also be used as the third processing device 320. The third processing device 320 is connected to the third communication device 301, the third storage device 310, etc., and controls each device.

[0071] The third processing device 320 reads the computer program stored in the third storage device 310 and operates in accordance with the read computer program. As a result, the third processing device 320 functions as a third acquisition unit 321, an estimation unit 322, and an output control unit 323.

[0072] FIG. 5 is a schematic diagram showing an example of the data structure of the data table 312. As shown in FIG.

[0073] The data table 312 stores angular velocity and acceleration, geomagnetism, atmospheric pressure (not shown), position (not shown) or sole of foot data obtained from the first wearable device 100 or the second wearable device 200, in association with the identification information (device ID) of each device and the time when each data was obtained.

[0074] FIG. 6 is a sequence diagram showing an example of the operation of the estimation process in the walking posture state estimation system 1.

[0075] An example of the operation of the estimation process will be described below with reference to the flowchart shown in Fig. 6. The operation flow described below is executed mainly by each processing device of each device in cooperation with each element of each device, based on a program stored in advance in each storage device of each device included in the walking posture state estimation system 1.

[0076] First, the second acquisition unit 221 of each second wearable device 200 acquires a measurement signal from the second sensor 205. The second acquisition unit 221 acquires, as second data obtained when the user walks, the acceleration and angular velocity in three axes directions acting on the second wearable device 200 indicated in the measurement signal, or sole data indicating the magnitude of pressure acting at each position on the sole of the user's foot (step S101). The second acquisition unit 221 periodically acquires the second data.

[0077] Next, the second transmission unit 222 transmits the second data acquired by the second acquisition unit 221 to the first wearable device 100 via the second interface device 204, together with the device ID of the second wearable device 200 and the acquisition time at which the second data was acquired (step S102). The second transmission unit 222 transmits the second data to the first wearable device 100 every time the second acquisition unit 221 acquires the second data. The second transmission unit 222 may also transmit multiple pieces of second data acquired by the second acquisition unit 221 together to the first wearable device 100 at any timing.

[0078] Next, the first acquisition unit 121 of the first wearable device 100 acquires the second data, the device ID, and the acquisition time by receiving them from the second wearable device 200 via the first interface device 104 (step S103).

[0079] Next, the first acquisition unit 121 acquires a measurement signal from the first sensor 105. The first acquisition unit 121 acquires the acceleration and angular velocity in three axial directions, geomagnetism, atmospheric pressure, or position applied to the first wearable device 100 indicated in the measurement signal as first data obtained when the user walks (step S104). The first data is data different from the second data. The first acquisition unit 121 periodically acquires the first data.

[0080] Next, the first transmission unit 122 transmits the first data and the second data acquired by the first acquisition unit 121 to the information processing device 300 via the first communication device 103, along with the device ID and acquisition time corresponding to each data (step S105). The first transmission unit 122 analyzes the multiple pieces of continuously measured first data and second data to detect maximum and minimum values ​​in the chronologically arranged first data and second data. The first transmission unit 122 detects the vibration periods of the first wearable device 100 and the second wearable device 200 based on the detected maximum and minimum values. The first transmission unit 122 determines the leg swing interval of the user wearing the first wearable device 100 and the second wearable device 200 based on the detected vibration periods. The first transmission unit 122 then transmits the first data and the second data, each corresponding to a predetermined number of steps (e.g., 10 steps), to the information processing device 300.

[0081] As a result, as will be described later, when the input data to be input to the trained model 311 is data for a predetermined number of steps, the information processing device 300 does not need to extract data for the predetermined number of steps from the acquired first data and second data. The walking posture state estimation system 1 can reduce the processing load on the information processing device 300, and can reduce the overall processing load.

[0082] Note that the first transmission unit 122 may transmit the first data or the second data to the information processing device 300 every time the first acquisition unit 121 acquires the first data or the second data. Furthermore, the first transmission unit 122 may transmit a plurality of pieces of first data and second data acquired by the first acquisition unit 121 together to the information processing device 300 at any timing.

[0083] Next, the third acquisition unit 321 of the information processing device 300 acquires the first data, the second data, the device ID, and the acquisition time by receiving them from the first wearable device 100 via the third communication device 301 (step S106). The third acquisition unit 321 stores the acquired first data and second data in the data table 312 in association with the corresponding device ID and acquisition time.

[0084] Next, the estimation unit 322 estimates the walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 based on the first data and the second data acquired by the third acquisition unit 321 (step S107). The estimation unit 322 estimates the walking posture state of the user using the trained model 311.

[0085] The trained model 311 includes one or more trained models. The trained model 311 is generated in advance by the information processing device 300, the server device 400, or the like. The trained model 311 is trained in advance by supervised learning such as deep learning or a support vector machine. The trained model 311 may also be trained in advance by a random forest, or the like. When input data is input, the trained model 311 is trained to output the walking posture state of a user wearing the wearable device that generated the input data.

[0086] The input data is first data acquired by the first wearable device 100 and / or second data acquired by the second wearable device 200. The input data may be data generated based on the first data and / or the second data. For example, the input data is a set of first data and / or second data (a vector in which each data is arranged one-dimensionally or two-dimensionally) for a predetermined number of consecutive steps or a predetermined period (e.g., 5 seconds). The input data may be a statistical value such as the average, median, maximum, or minimum value of the first data and / or second data for the predetermined number of consecutive steps or the predetermined period. Furthermore, the input data may be a quaternion calculated from angular velocity, acceleration, or geomagnetic field included in the first data and / or second data.

[0087] Furthermore, the input data may be predetermined components (such as the first principal component and / or the second principal component) calculated by principal component analysis of the angular velocity or acceleration in three axial directions or the sole data included in the first data and / or the second data. In this case, if the contribution rate of the first principal component is greater than a predetermined rate (e.g., 80%), the first principal component may be used as the input data, and if the contribution rate of the first principal component is equal to or less than the predetermined rate, the second principal component may be used as the input data. Furthermore, the principal component analysis may be performed after noise is removed using a known noise processing technique. In these cases, a model pre-trained by a random forest or the like may be used as the trained model 311. In these cases, a model pre-trained by a neural network or the like may be used as the trained model 311. This allows the walking posture state estimation system 1 to further improve the accuracy of the trained model 311.

[0088] Furthermore, the input data may be a spectral image obtained by transforming each of the angular velocities and / or accelerations in the three-axis directions included in the first data and / or second data using a short-time Fourier transform or a wavelet transform. The spectral image obtained by transforming each of the angular velocities and / or accelerations in the three-axis directions included in the first data using a short-time Fourier transform or a wavelet transform is an example of first image data and image data. The spectral image obtained by transforming each of the angular velocities and / or accelerations in the three-axis directions included in the second data using a short-time Fourier transform or a wavelet transform is an example of second image data and image data. In these cases, a model pre-trained by a random forest or the like is used as the trained model 311. In these cases, a model pre-trained by a neural network or the like may also be used as the trained model 311. This allows the walking posture state estimation system 1 to further improve the accuracy of the trained model 311.

[0089] The input data may also be a feature vector having elements of the gradation value (pressure) of each pixel of the sole image or the sole vector, and elements of angular velocity, acceleration, geomagnetism, atmospheric pressure and / or position.

[0090] The walking posture state output from the trained model 311 includes five elements. The first element indicates the state of the stride, particularly whether the user's stride is sufficient or insufficient. The second element indicates the state of the center of gravity position, particularly whether the user's center of gravity is normal, leaning forward, or leaning backward. The third element indicates the state of the swinging leg, particularly whether the user's outgoing leg is not kicking, dragging, or kicking. The fourth element indicates the state of the supporting leg, particularly whether the user's supporting leg is stiff or not stiff. The fifth element indicates the state of the hip joint, particularly whether the user's hip joint is open or not. The first element is an element related to the premise of the walking posture. The second and third elements are elements related to the front-to-back axis. The fourth and fifth elements are elements related to the rotational (left-to-right) axis. The walking posture state does not have to include all of the first to fifth elements, as long as it includes at least one element.

[0091] The walking posture state output from the trained model 311 is not limited to information directly indicating each of the first to fifth elements, but may also be a predetermined feature value related to each of the first to fifth elements.

[0092] The trained model 311 includes, for example, trained models corresponding to the first wearable device 100 and each of the second wearable devices 200. That is, the trained model 311 includes a first trained model capable of outputting a first walking posture state estimated based on first data acquired from the first wearable device 100, and a second trained model capable of outputting a second walking posture state estimated based on second data acquired from each of the second wearable devices 200.

[0093] Alternatively, the trained model 311 includes a first trained model capable of outputting a first walking posture state estimated based on first image data based on first data acquired from the first wearable device 100, and a second trained model capable of outputting a second walking posture state estimated based on second image data based on second data acquired from each second wearable device 200.

[0094] As a result, the walking posture state estimation system 1 can estimate the walking posture state for each part of the body to which the wearable device is attached, and can comprehensively estimate the user's walking posture state based on the walking posture state estimated for each part.

[0095] In these cases, the estimation unit 322 may acquire the walking posture state by inputting input data based on the first data into the first layer of the first trained model, and may acquire the walking posture state by inputting input data based on the second data into the second layer of the second trained model. This first layer is a specific layer of the input layer or the intermediate layer. This second layer is a layer different from the first layer and is a specific layer of the input layer or the intermediate layer. This allows the walking posture state estimation system 1 to flexibly change the number of processing layers (number of processing steps) in each trained model, thereby suppressing increases in processing time and processing load.

[0096] Furthermore, the estimation unit 322 may acquire, as the walking posture state, information output from the first layer of the first trained model when input data based on the first data is input to the first trained model, and may acquire, as the walking posture state, information output from the second layer of the second trained model when input data based on the second data is input to the second trained model. This first layer is a specific layer among the intermediate layers or the output layers. This second layer is a layer different from the first layer and is a specific layer among the intermediate layers or the output layers. This allows the walking posture state estimation system 1 to flexibly change the number of processing layers (number of processing steps) in each trained model, thereby suppressing increases in processing time and processing load.

[0097] The trained model 311 may be generated to correspond to multiple devices among the first wearable device 100 and the second wearable device 200. In particular, the trained model 311 may be generated to correspond to all of the first wearable device 100 and the second wearable device 200. That is, the trained model 311 includes a single trained model capable of outputting a walking posture state estimated based on the first data acquired from the first wearable device 100 and the second data acquired from each second wearable device 200.

[0098] Alternatively, the trained model 311 includes a single trained model capable of outputting an estimated walking posture state based on image data based on the first data and each second data acquired from the first wearable device 100 and each second wearable device 200.

[0099] In these cases, the trained model 311 is trained to output a walking posture state when a vector whose elements are first data and second data obtained from multiple wearable devices at corresponding times (approximately the same times) is input.

[0100] This allows the walking posture state estimation system 1 to estimate the walking posture state in a composite manner based on data acquired from multiple wearable devices.

[0101] Furthermore, the trained model 311 includes a trained model corresponding to each of the first to fifth elements. That is, the trained model 311 includes a plurality of trained models capable of outputting information relating to each of the first to fifth elements. The trained model 311 may also include a single trained model capable of collectively outputting information relating to each of the first to fifth elements.

[0102] The trained model 311 may also include an upstream trained model and a downstream trained model arranged in series. In this case, the upstream trained model is trained to output a predetermined feature when input data is input, and the downstream trained model is trained to output a walking posture state when a feature output from an intermediate layer or an output layer of the upstream trained model is input. The predetermined feature is a feature related to the input data or a feature related to the walking posture state.

[0103] The trained model 311 may also include any combination of the trained models described above. For example, the trained model 311 includes a first upstream trained model, a second upstream trained model, a third upstream trained model, and a downstream trained model. The first upstream trained model is trained to output a predetermined feature amount when a first principal component of sole data is input. The second upstream trained model is trained to output a predetermined feature amount when a second principal component of sole data is input. The third upstream trained model is trained to output a predetermined feature amount when a first principal component of angular velocity or acceleration is input. Alternatively, the third upstream trained model is trained to output a predetermined feature amount when a spectral image transformed by short-time Fourier transform or wavelet transform on angular velocity and / or acceleration is input. The downstream trained model is trained to output a walking posture state when a feature vector whose elements are data output from the output layer or intermediate layer of the first upstream trained model, the second upstream trained model, and the third upstream trained model is input. This allows the walking posture state estimation system 1 to further improve the accuracy of the trained model 311.

[0104] Alternatively, the trained model 311 includes a first upstream trained model, a second upstream trained model, and a downstream trained model. The first upstream trained model is trained to output a predetermined feature amount when sole data is input. The second upstream trained model is trained to output a predetermined feature amount when angular velocity or acceleration is input. The downstream trained model is trained to output a walking posture state when a feature vector whose elements are data output from the output layer or intermediate layer of the first upstream trained model and the second upstream trained model is input. This allows the walking posture state estimation system 1 to further improve the accuracy of the trained model 311.

[0105] In this way, the trained model 311 is configured to be able to output the estimated walking posture state of the user wearing the first wearable device 100 and the second wearable device 200.

[0106] The training data for the trained model 311 includes a combination of pre-generated input data and the walking posture of a user wearing the wearable device that generated the input data. Each training data is created by an expert who views video images of a user wearing the wearable device that generated the first data or second data when the first data or second data is generated by the wearable device.

[0107] Each training data set may be composed of, for example, a set of first data or second data for a predetermined number of consecutive steps or a predetermined period of time, combined with a walking posture state. In this case, each training data set may be generated so that a portion of the first data or second data included in each training data set overlaps. That is, the first training data set may include data for a predetermined number of steps or a predetermined period of time from a first time, and the second training data set may include data for a predetermined number of steps or a predetermined period of time from a second time immediately after the first time (e.g., one second later). This allows the walking posture state estimation system 1 to efficiently generate the trained model 311 from a small amount of sample data, thereby reducing the time and effort required to generate the trained model 311.

[0108] When the trained model 311 is pre-trained using a random forest or the like, each training data is composed of, for example, a combination of a walking posture state and a vector in which first data or second data for a predetermined number of consecutive steps or a predetermined period of time is arranged in one dimension. Each training data may include, for example, only angular velocities in three axial directions of the first data or second data. The walking posture state estimation system 1 can improve the accuracy of the trained model 311 by generating the trained model 311 based on both angular velocity and acceleration. On the other hand, the walking posture state estimation system 1 can reduce the time and effort required to generate the trained model 311 by generating the trained model 311 based only on angular velocity. In these cases, each training data may be generated so that part of the first data or second data included in each training data overlaps.

[0109] The estimation unit 322 converts the first data and second data acquired in step S106 into a format compatible with the trained model 311 and inputs the converted data to the trained model 311. The estimation unit 322 estimates the walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 based on the information output from the trained model 311. The estimation unit 322 estimates the walking posture state of the user for each element included in the walking posture state.

[0110] When the information output from the trained model 311 is a feature, the information processing device 300 stores in advance a table or formula indicating the relationship between the feature and the walking posture state in the third storage device 310. The estimation unit 322 refers to the table or formula stored in the third storage device 310, and identifies the walking posture state of the user corresponding to the feature output from the trained model 311.

[0111] Furthermore, if the trained model 311 includes multiple trained models corresponding to each wearable device, the estimation unit 322 estimates that the walking posture state output from the most trained models among the walking posture states output from each trained model is the walking posture state of the user.

[0112] In this way, the estimation unit 322 estimates the walking posture state of the user based on the acquired first data and second data by using the trained model 311. That is, the estimation unit 322 uses the trained model 311 to acquire the walking posture state estimated based on the acquired first data and second data.

[0113] Next, the output control unit 323 outputs the estimated walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 by transmitting it to the first wearable device 100 via the third communication device 301 (step S108).

[0114] Next, the first acquisition unit 121 of the first wearable device 100 acquires the estimated walking posture state of the user by receiving it from the information processing device 300 via the first communication device 103 (step S109).

[0115] Next, the first acquisition unit 121 outputs the acquired walking posture state by displaying it on the first display device 102 to notify the user (step S110), and the series of steps ends. This allows the user to recognize their own walking posture state.

[0116] In step S108, the output control unit 323 of the information processing device 300 may transmit the estimated walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 to an administrator device owned by an administrator who manages the user's health. In this case, the administrator device notifies the administrator by displaying the received walking posture state. The administrator can recognize the user's walking posture state and suggest improvements to the user's walking posture state.

[0117] Furthermore, the timing of execution of the process of step S101 and the process of step S104 may be arbitrary, and may be simultaneous. For example, when the first acquisition unit 121 of the first wearable device 100 receives an instruction to start measurement from the user using the first input device 101, it transmits a measurement request signal requesting the start of measurement to the second wearable device 200 via the first interface device 104, and acquires the first data. On the other hand, when the second acquisition unit 221 of the second wearable device 200 receives the measurement request signal from the first wearable device 100 via the second interface device 204, it acquires the second data and transmits it to the first wearable device 100 via the second interface device 204. The first acquisition unit 121 repeats the above process until it receives an instruction to end measurement from the user using the first input device 101, and then executes the process of step S105.

[0118] The inventors evaluated the trained model 311 for all of the first to fifth elements and confirmed that the accuracy rate of the information output from the trained model 311 was 93% or higher, indicating sufficiently high accuracy. In other words, it was confirmed that the walking posture state estimation system 1 can estimate the user's walking posture state with higher accuracy by using the trained model 311. Furthermore, by using the trained model 311, the walking posture state estimation system 1 can accurately and efficiently calculate walking posture states for various combinations of various types of parameters. Furthermore, by using the trained model 311, the walking posture state estimation system 1 can identify walking posture states without storing walking posture states corresponding to various combinations of various types of parameters, thereby reducing the storage capacity of the storage device. Furthermore, by using the trained model 311, the walking posture state estimation system 1 can identify walking posture states without performing complex determinations according to various combinations of various types of parameters, thereby reducing the processing time and processing load of the estimation process.

[0119] The technical significance of the walking posture state including the first to fifth elements will be described below.

[0120] The main power source for walking is the lifting of the thighs, and the main power source for running is the kicking of the toes. Therefore, walking and running, i.e., running as slowly as possible, are fundamentally different actions, and clearly changing the movements between walking and running contributes to the user's health. In order to kick with the toes, the ball of the foot must be placed on the floor, causing the foot to curl inward and eliminating the arch. This loosens the bones of the foot overall, making it easier for the impact on the foot to be transmitted to the knee or hip. Running puts strain on the feet, so it is not advisable to perform running, which puts strain on the feet, while walking.

[0121] The walking posture state estimation system 1 estimates each of the first to fifth elements as the walking posture state of the user, thereby being able to identify whether the user is walking appropriately without running. In particular, the first to fifth elements are interrelated, and as the user's posture improves in the order of the first to fifth elements, the user can easily improve their walking posture.

[0122] First, the stride length must not be too short, which would slow down the walking speed, and a certain stride length or more must be ensured. By including a first element indicating whether the user's stride length is sufficient or insufficient as a premise of the walking posture state, the walking posture state estimation system 1 can ensure that the user's stride length is at least a certain length.

[0123] Next, if the user's posture is not straight, the user will be forced to perform a running motion. If the user stands upright with their center of gravity on their heels, they can lift their feet using their thighs, but if they lean forward, their upper body will bend over, making it difficult to lift their feet using their thighs, and they will be forced to kick with their toes. By including a second element indicating whether the user's center of gravity is normal, leaning forward, or leaning backward as an element related to the axis in the front-to-back direction, the walking posture state estimation system 1 can confirm whether the user's posture is being kept straight.

[0124] Next, even if the user's center of gravity is straight, if the user does not properly raise the thigh, the user will be forced to perform a running motion. Even if the user's center of gravity is straight and the foot can be raised without kicking with the toes, kicking with the toes will result in intense up-and-down movement. Furthermore, even if the user's center of gravity is straight, the user may drag their foot because they have not developed the habit of lifting their foot with their thigh. By including a third element indicating whether the user is not kicking, dragging, or kicking the leg as an element related to the front-to-back axis in the walking posture state, the walking posture state estimation system 1 can confirm whether the user is properly raising the thigh.

[0125] Next, even if the axis in the front-to-back direction is correct, if the rotational (left-to-right) axis is incorrect, the walking motion will not be performed properly. If the knee is turned inward relative to the toes, the knee will get in the way and the opposite foot will not step out correctly. In other words, if the supporting foot is stiff or the hip joint is not open, the intended foot will not rise correctly. On the other hand, if the user rotates the hip joint, space for the opposite foot to step out is secured, making it easier to lift the foot and correctly lifting the thigh. By including a fourth element indicating whether the user's supporting foot is stiff or not and a fifth element indicating whether the user's hip joint is open or not as elements related to the rotational (left-to-right) axis, the walking posture state estimation system 1 can confirm whether the rotational axis is correct.

[0126] As described above in detail, the walking posture state estimation system 1 estimates the walking posture state of the user using the trained model 311 based on the first data acquired by the first wearable device 100 and the second data acquired by the second wearable device 200. This enables the walking posture state estimation system 1 to estimate the walking posture state of the user with higher accuracy.

[0127] FIG. 7 is a sequence diagram showing an example of the operation of the estimation process in the walking posture state estimation system 1 according to another embodiment.

[0128] An example of the operation of the estimation process according to this embodiment will be described below with reference to the flowchart shown in FIG. 7. The operation flow described below is executed mainly by the processing devices of each device in cooperation with the elements of each device, based on a program previously stored in the storage device of each device included in the walking posture state estimation system 1. In this embodiment, the trained model 311 is stored in the server device 400. The processes of steps S201 to S206 and S213 to S215 in FIG. 7 are similar to the processes of steps S101 to S106 and S108 to S110 in FIG. 6, so their description will be omitted. Only steps S207 to S212 will be described below.

[0129] In step S207, the estimation unit 322 of the information processing device 300 transmits a request signal requesting transmission of the user's walking posture information to the server device 400 via the third communication device 301 (step S207). The request signal includes the first data and the second data acquired in step S106 and converted into a format corresponding to the trained model 311.

[0130] Next, the server device 400 receives a request signal from the information processing device 300 (step S208). Next, the server device 400 inputs the first data and the second data included in the received request signal into the trained model 311 and acquires output information output from the trained model 311 (step S209). Next, the server device 400 transmits the acquired output information to the information processing device 300 (step S210).

[0131] Next, the estimation unit 322 of the information processing device 300 acquires output information by receiving it from the server device 400 via the third communication device 301 (step S211). Next, the estimation unit 322 estimates the walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 based on the acquired output information (step S212). The estimation unit 322 estimates the walking posture state of the user in the same manner as the processing of step S107 in FIG. 6 .

[0132] As described above in detail, the walking posture state estimation system 1 is able to estimate the user's walking posture state with greater accuracy even when the trained model 311 is stored in a server device 400 other than the information processing device 300.

[0133] By utilizing the trained model 311 stored in the server device 400, the walking posture state estimation system 1 can estimate the walking posture state of the user using the latest trained model updated by the server device 400. Furthermore, by utilizing the trained model 311 stored in the server device 400, the walking posture state estimation system 1 can reduce the storage capacity of the information processing device 300. Meanwhile, by utilizing the trained model 311 stored in the information processing device 300, the information processing device 300 can estimate the walking posture state of the user even when communication with the server device 400 is disconnected. Furthermore, by utilizing the trained model 311 stored in the information processing device 300, the information processing device 300 can reduce the amount of communication between the information processing device 300 and the server device 400.

[0134] FIG. 8 is a sequence diagram showing an example of the operation of the estimation process in the walking posture state estimation system 1 according to yet another embodiment.

[0135] An example of the operation of the estimation process according to this embodiment will be described below with reference to the flowchart shown in FIG. 8 . The operation flow described below is executed primarily by each processing device of each device in cooperation with each element of each device, based on a program previously stored in each storage device of each device included in the walking posture state estimation system 1. In this embodiment, the trained model 311 and the data table 312 are stored in the first storage device 110 of the first wearable device 100. The first processing device 120 functions as a first acquisition unit 121 and a first transmission unit 122, as well as an estimation unit and an output control unit having functions similar to those of the estimation unit 322 and the output control unit 323. The processes of steps S301 to S303 and S306 in FIG. 8 are similar to those of steps S101 to S103 and S110 in FIG. 6 , and therefore will not be described again. Below, only steps S304 to S305 will be described.

[0136] In step S304, the first acquirer 121 of the first wearable device 100 acquires first data obtained when the user walks, similar to the process of step S104 (step S304). The first acquirer 121 stores the acquired first data and second data in the data table 312 in association with the corresponding device ID and acquisition time.

[0137] Next, the estimation unit of the first wearable device 100 estimates the walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 based on the first data and the second data, similar to the processing of step S107 (step S305).

[0138] As described above in detail, the walking posture state estimation system 1 can estimate the walking posture state of the user with higher accuracy even when the first wearable device 100 estimates the walking posture state of the user.

[0139] Although the preferred embodiments have been described above, the embodiments are not limited thereto. For example, even when the first wearable device 100 estimates the walking posture state of the user, the first wearable device 100 may estimate the walking posture state of the user based on the output information of the trained model 311 stored in the server device 400.

[0140] Furthermore, the second wearable device 200 may transmit the second data directly to the information processing device 300, rather than transmitting the second data to the information processing device 300 via the first wearable device 100. This allows the walking posture state estimation system 1 to reduce the processing load on the first wearable device 100. On the other hand, the first wearable device 100 aggregates the second data from the second wearable device 200, allowing the walking posture state estimation system 1 to reduce the processing load on the information processing device 300.

[0141] Furthermore, in the walking posture state estimation system 1, a plurality of first wearable devices 100 and / or a plurality of information processing devices 300 may cooperate to share the respective steps of the above-described processes.

[0142] FIG. 9 is a diagram showing a schematic configuration of a walking posture state estimation system 2 according to another embodiment.

[0143] The walking posture state estimation system 2 includes the same devices as the walking posture state estimation system 1. However, as shown in Fig. 9, the walking posture state estimation system 2 further includes one or more sole information acquisition devices 500. Each sole information acquisition device 500 is worn by a user when in use.

[0144] The first wearable device 100 and each second wearable device 200, and the first wearable device 100 and each sole information acquisition device 500 are communicably connected to each other via a wireless network such as Bluetooth (registered trademark) or a wireless LAN (Local Area Network). Furthermore, the second wearable devices 200, each sole information acquisition device 500, the information processing device 300, and the server device 400 may be communicably connected to each other via a network N.

[0145] In this embodiment, the third communication device 301 of the information processing device 300 sends data received from the first wearable device 100, the second wearable device 200, the sole of foot information acquisition device 500, the server device 400, or the like via the network N to the third processing device 320. The third communication device 301 transmits the data received from the third processing device 320 to the first wearable device 100, the second wearable device 200, the sole of foot information acquisition device 500, the server device 400, or the like via the network N.

[0146] In this embodiment, the trained model 311 is a model for estimating the walking posture state of a user wearing the first wearable device 100, the second wearable device 200, and the sole information acquisition device 500. The data table 312 stores information measured by the first sensor 105 of the first wearable device 100, the second sensor 205 of the second wearable device 200, or the fifth sensor 505 of the sole information acquisition device 500. The data table 312 stores angular velocity and acceleration, geomagnetism, atmospheric pressure (not shown), position (not shown), or sole data acquired from the first wearable device 100, the second wearable device 200, or the sole information acquisition device 500, in association with identification information (device ID) of each device and the acquisition time at which each data was acquired.

[0147] FIG. 10 is a diagram showing a schematic configuration of the sole information acquisition device 500.

[0148] The sole information acquisition device 500 is a terminal device that can be worn by a user. The sole information acquisition device 500 is an insole-type wearable computer that is worn inside the user's shoe. The sole information acquisition device 500 includes a fifth input device 501, a fifth display device 502, a fifth communication device 503, a fifth interface device 504, a fifth sensor 505, a fifth storage device 510, and a fifth processing device 520. The fifth input device 501, the fifth display device 502, the fifth communication device 503, the fifth interface device 504, the fifth sensor 505, the fifth storage device 510, and the fifth processing device 520 are connected to each other via a CPU bus or the like.

[0149] The fifth input device 501 has an input device such as a button and an interface circuit that acquires a signal from the input device, and outputs an operation signal in response to an input operation by the user. The fifth input device 501 may be omitted.

[0150] The fifth display device 502 has a display such as a liquid crystal display, an organic electroluminescence display, or the like, and an interface circuit that outputs image data to the display, and displays the image data on the display. The fifth display device 502 may have an LED or the like, and may notify the user by turning it on or off. The fifth display device 502 may be omitted.

[0151] The fifth communication device 503 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit that complies with a communication protocol such as wireless LAN. The fifth communication device 503 is communicatively connected to the network N in accordance with a communication standard such as wireless LAN. The fifth communication device 503 sends data received from the information processing device 300, the server device 400, or the like via the network N to the fifth processing device 520. In addition, the fifth communication device 503 transmits data received from the fifth processing device 520 to the information processing device 300, the server device 400, or the like via the network N.

[0152] The fifth interface device 504 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit conforming to a communication protocol such as Bluetooth (registered trademark). The fifth interface device 504 is communicatively connected to the first wearable device 100 in accordance with a communication standard such as Bluetooth (registered trademark). The fifth interface device 504 sends data received from the first wearable device 100 to the fifth processing device 520. The fifth interface device 504 also transmits data received from the fifth processing device 520 to the first wearable device 100.

[0153] The fifth sensor 505 is an example of a static information sensor. The fifth sensor 505 includes an insole-type pressure sensor. The insole-type pressure sensor is a resistance change type or capacitance change type pressure sensor arranged one-dimensionally or two-dimensionally. The fifth sensor 505 measures the magnitude of pressure applied to each position on the sole of the user with each pressure sensor, generates sole data in which the measured pressure magnitude is the gradation value of each pixel, and outputs the sole data to the fifth processing device 520. The sole data is, for example, a sole image in which the magnitude of pressure measured by each pressure sensor is the gradation value of each pixel. The sole data may be, for example, a sole vector in which the pressure measured by each pressure sensor is used as an element.

[0154] The fifth storage device 510 includes a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or an optical disk. The fifth storage device 510 also stores computer programs, databases, tables, etc. used for various processes of the sole-of-foot information acquisition device 500. The computer programs may be installed into the fifth storage device 510 from a computer-readable portable recording medium using a known setup program, etc. The portable recording medium is, for example, a CD-ROM or a DVD-ROM. The computer programs may be stored in a recording medium owned by a predetermined server and installed via the network N.

[0155] The fifth processing device 520 operates based on a program stored in advance in the fifth storage device 510. The fifth processing device 520 is, for example, a CPU. A DSP, an LSI, an ASIC, an FPGA, etc. may be used as the fifth processing device 520. The fifth processing device 520 is connected to the fifth input device 501, the fifth display device 502, the fifth communication device 503, the fifth interface device 504, the fifth sensor 505, the fifth storage device 510, etc., and controls each device. The fifth processing device 520 acquires information measured by the fifth sensor 505 when the user walks and transmits the information to the first wearable device 100 via the fifth interface device 504.

[0156] The fifth processing device 520 reads the computer program stored in the fifth storage device 510 and operates in accordance with the read computer program. As a result, the fifth processing device 520 functions as a fifth acquisition unit 521 and a fifth transmission unit 522.

[0157] FIG. 11 is a sequence diagram showing an example of the operation of the estimation process in the walking posture state estimation system 2.

[0158] An example of the operation of the estimation process will be described below with reference to the flowchart shown in Fig. 11. The operation flow described below is executed mainly by each processing device of each device in cooperation with each element of each device, based on a program stored in advance in each storage device of each device included in the walking posture state estimation system 2.

[0159] First, the fifth acquisition unit 521 of each sole information acquisition device 500 acquires a measurement signal from the fifth sensor 505. The fifth acquisition unit 521 acquires sole data indicating the magnitude of pressure exerted on each position on the user's sole, as static information related to the user's sole (step S401). The static information indicates information related to static alignment. Alignment indicates the arrangement of bones or joints, particularly morphological characteristics such as posture, scoliosis, kyphosis, bow legs or knock-knees, femoral neck-shaft angle or anteversion angle, and tibia torsion. The static alignment indicates the alignment in a natural standing position (standing posture with the face facing forward, both upper limbs hanging down along the trunk, the radial edges of the forearms facing forward, and the lower limbs parallel with the toes facing forward). An abnormality in the static alignment affects the magnitude of pressure exerted on each position on the sole. The walking posture state estimation system 2 can estimate the static alignment with high accuracy by using the sole data as static information. The fifth acquisition unit 521 periodically acquires the static information.

[0160] Next, the fifth transmission unit 522 transmits the static information acquired by the fifth acquisition unit 521, together with the device ID of the sole information acquisition device 500 and the acquisition time when the static information was acquired, to the first wearable device 100 via the fifth interface device 504 (step S402). Every time the fifth acquisition unit 521 acquires static information, the fifth transmission unit 522 transmits the static information to the first wearable device 100. The fifth transmission unit 522 may transmit multiple pieces of static information acquired by the fifth acquisition unit 521 together to the first wearable device 100 at any timing.

[0161] Next, the first acquisition unit 121 of the first wearable device 100 acquires the static information, device ID, and acquisition time from each sole information acquisition device 500 by receiving them via the first interface device 104 (step S403).

[0162] Next, the second acquisition unit 221 of each second wearable device 200 acquires a measurement signal from the second sensor 205. The second acquisition unit 221 acquires the accelerations and angular velocities in three axes of the second wearable device 200 indicated by the measurement signal as second dynamic information obtained when the user walks (step S404). The second dynamic information indicates information related to dynamic alignment. The dynamic alignment indicates alignment during exercise. If an abnormality occurs in the dynamic alignment of each part of the body, the accelerations and angular velocities of each part are affected. The walking posture state estimation system 2 can accurately estimate the dynamic alignment of the part by using the accelerations and angular velocities of the part where the second wearable device 200 is worn as the second dynamic information. The second acquisition unit 221 periodically acquires the second dynamic information.

[0163] Next, the second transmission unit 222 transmits the second dynamic information acquired by the second acquisition unit 221 to the first wearable device 100 via the second interface device 204, together with the device ID of the second wearable device 200 and the acquisition time at which the second dynamic information was acquired (step S405). The second transmission unit 222 transmits the second dynamic information to the first wearable device 100 every time the second acquisition unit 221 acquires the second dynamic information. The second transmission unit 222 may collectively transmit multiple pieces of second dynamic information acquired by the second acquisition unit 221 to the first wearable device 100 at any timing.

[0164] Next, the first acquisition unit 121 of the first wearable device 100 acquires the second dynamic information, the device ID, and the acquisition time by receiving the second dynamic information from the second wearable device 200 via the first interface device 104 (step S406).

[0165] Next, the first acquisition unit 121 acquires a measurement signal from the first sensor 105. The first acquisition unit 121 acquires the acceleration and angular velocity in three axes, geomagnetic field, atmospheric pressure, or position of the first wearable device 100 indicated in the measurement signal as first dynamic information obtained when the user walks (step S407). The first dynamic information is an example of dynamic information. The first dynamic information is information different from the second dynamic information and indicates information related to dynamic alignment. By using the acceleration and angular velocity, geomagnetic field, atmospheric pressure, or position of the part of the body where the first wearable device 100 is worn as the first dynamic information, the walking posture state estimation system 2 can accurately estimate the dynamic alignment of that part. The first acquisition unit 121 periodically acquires the first dynamic information.

[0166] Next, the first transmission unit 122 transmits the static information, first dynamic information, and second dynamic information acquired by the first acquisition unit 121 to the information processing device 300 via the first communication device 103, along with the device ID and acquisition time corresponding to each piece of information (step S408). The first transmission unit 122 analyzes the multiple pieces of continuously measured first dynamic information and second dynamic information to detect maximum and minimum values ​​in the first dynamic information and second dynamic information arranged in time series. The first transmission unit 122 detects the vibration periods of the first wearable device 100 and the second wearable device 200 based on the detected maximum and minimum values. The first transmission unit 122 identifies the leg swing interval of the user wearing the first wearable device 100 and the second wearable device 200 based on the detected vibration periods. Then, the first transmission unit 122 transmits to the information processing device 300 the first dynamic information and the second dynamic information each corresponding to a predetermined number of steps (for example, 10 steps) of the user, and the static information acquired during the same period.

[0167] As a result, as will be described later, when the input information to be input to the trained model 311 is information for a predetermined number of steps, the information processing device 300 does not need to extract data for the predetermined number of steps from the acquired static information, first dynamic information, and second dynamic information. The walking posture state estimation system 2 can reduce the processing load on the information processing device 300, and can reduce the overall processing load.

[0168] Note that the first transmission unit 122 may transmit the static information, the first dynamic information, or the second dynamic information to the information processing device 300 every time the first acquisition unit 121 acquires the static information, the first dynamic information, or the second dynamic information. Furthermore, the first transmission unit 122 may transmit the plurality of pieces of static information, the first dynamic information, and the second dynamic information acquired by the first acquisition unit 121 together to the information processing device 300 at any timing.

[0169] Next, the third acquisition unit 321 of the information processing device 300 acquires the static information, the first dynamic information, the second dynamic information, the device ID, and the acquisition time by receiving the information from the first wearable device 100 via the third communication device 301 (step S409). The third acquisition unit 321 stores the acquired static information, the first dynamic information, and the second dynamic information in the data table 312 in association with the corresponding device ID and acquisition time.

[0170] Next, the estimation unit 322 estimates the walking posture of the user wearing the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200 based on the static information, the first dynamic information, and the second dynamic information acquired by the third acquisition unit 321 (step S410). The estimation unit 322 estimates the walking posture of the user using the trained model 311.

[0171] In the present embodiment, the trained model 311 includes one or more trained models. The trained model 311 is generated in advance by the information processing device 300, the server device 400, or the like. The trained model 311 is trained in advance by supervised learning such as deep learning or a support vector machine. The trained model 311 may also be trained in advance by a random forest, or the like. When input information is input, the trained model 311 is trained to output the walking posture state of a user wearing the sole information acquisition device 500, the first wearable device 100, or the second wearable device 200 that generated the input information.

[0172] The input information is static information acquired by the sole information acquisition device 500, first dynamic information acquired by the first wearable device 100, and / or second dynamic information acquired by the second wearable device 200. The input information may be information generated based on the static information, the first dynamic information, and / or the second dynamic information. For example, the input information is a set of static information, the first dynamic information, and / or the second dynamic information for a predetermined number of consecutive steps or a predetermined period (e.g., 5 seconds) (a one-dimensional or two-dimensional vector in which the magnitude of pressure measured by each pressure sensor, or the acceleration or angular velocity measured by each acceleration sensor or gyro sensor is used as an element). The input information may be a statistical value such as the average, median, maximum, or minimum value of the static information, the first dynamic information, and / or the second dynamic information for a predetermined number of consecutive steps or a predetermined period. Furthermore, the input information may be a quaternion calculated from the angular velocity, acceleration, or geomagnetic field included in the first dynamic information and / or the second dynamic information.

[0173] The input information may also be predetermined components (such as the first principal component and / or the second principal component) calculated by principal component analysis of the sole data or the angular velocity or acceleration in three axial directions included in the static information, the first dynamic information, and / or the second dynamic information. In this case, if the contribution rate of the first principal component is greater than a predetermined rate (e.g., 80%), the first principal component may be used as the input information, and if the contribution rate of the first principal component is equal to or less than the predetermined rate, the second principal component may be used as the input information. Furthermore, the principal component analysis may be performed after noise is removed using a known noise processing technique. In these cases, a model pre-trained by a random forest or the like may be used as the trained model 311. In these cases, a model pre-trained by a neural network or the like may be used as the trained model 311. This allows the walking posture state estimation system 2 to further improve the accuracy of the trained model 311.

[0174] The input information may also include spectral images obtained by transforming the angular velocities and / or accelerations in the three-axis directions included in the first dynamic information and / or the second dynamic information using a short-time Fourier transform or a wavelet transform. The spectral images obtained by transforming the angular velocities and / or accelerations in the three-axis directions included in the first dynamic information using a short-time Fourier transform or a wavelet transform are examples of first image data and image data. The spectral images obtained by transforming the angular velocities and / or accelerations in the three-axis directions included in the second dynamic information using a short-time Fourier transform or a wavelet transform are examples of second image data and image data. In these cases, a model pre-trained using a random forest or the like is used as the trained model 311. In these cases, a model pre-trained using a neural network or the like may be used as the trained model 311. This allows the walking posture state estimation system 2 to further improve the accuracy of the trained model 311.

[0175] The input information may also be a feature vector having elements of the gradation value (pressure) of each pixel of the sole image or the sole vector, and elements of angular velocity, acceleration, geomagnetism, atmospheric pressure and / or position.

[0176] The walking posture state output from the trained model 311 includes five elements. The first element indicates the state of the stride, particularly whether the user's stride is sufficient or insufficient. The second element indicates the state of the center of gravity position, particularly whether the user's center of gravity is normal, leaning forward, or leaning backward. The third element indicates the state of the swinging leg, particularly whether the user's outgoing leg is not kicking, dragging, or kicking. The fourth element indicates the state of the supporting leg, particularly whether the user's supporting leg is stiff or not stiff. The fifth element indicates the state of the hip joint, particularly whether the user's hip joint is open or not. The first element is an element related to the premise of the walking posture. The second and third elements are elements related to the front-to-back axis. The fourth and fifth elements are elements related to the rotational (left-to-right) axis. The walking posture state does not have to include all of the first to fifth elements, as long as it includes at least one element.

[0177] The walking posture state output from the trained model 311 is not limited to information directly indicating each of the first to fifth elements, but may also be a predetermined feature value related to each of the first to fifth elements.

[0178] The trained model 311 includes, for example, trained models corresponding to each sole information acquisition device 500, the first wearable device 100, and each second wearable device 200. That is, the trained model 311 includes a first trained model, a second trained model, and a third trained model. The first trained model is capable of outputting a first walking posture state estimated based on static information acquired from each sole information acquisition device 500. The second trained model is capable of outputting a second walking posture state estimated based on first dynamic information acquired from the first wearable device 100. The third trained model is capable of outputting a third walking posture state estimated based on second dynamic information acquired from each second wearable device 200.

[0179] Alternatively, the second trained model may be capable of outputting a first walking posture state estimated based on first image data based on first dynamic information acquired from the first wearable device 100. The third trained model may be capable of outputting a second walking posture state estimated based on second image data based on second dynamic information acquired from each second wearable device 200.

[0180] As a result, the walking posture state estimation system 2 can estimate the walking posture state for each part of the body to which the sole information acquisition device 500, the first wearable device 100 or the second wearable device 200 is worn, and can comprehensively estimate the user's walking posture state based on the walking posture state estimated for each part.

[0181] In these cases, the estimation unit 322 may acquire the walking posture state by inputting input information based on static information to the first layer of the first trained model, inputting input information based on the first dynamic information to the second layer of the second trained model, and inputting input information based on the second dynamic information to the third layer of the third trained model. This first layer is a specific layer of the input layer or the intermediate layer. This second layer is a layer different from the first layer and is a specific layer of the input layer or the intermediate layer. This third layer is a layer different from the first layer and / or the second layer and is a specific layer of the input layer or the intermediate layer. This allows the walking posture state estimation system 2 to flexibly change the number of processing layers (number of processing steps) in each trained model, thereby suppressing increases in processing time and processing load.

[0182] Furthermore, the estimation unit 322 may acquire first output information, second output information, and third output information as the walking posture state. The first output information is information output from the first layer of the first trained model when input information based on static information is input to the first trained model. The second output information is information output from the second layer of the second trained model when input information based on first dynamic information is input to the second trained model. The third output information is information output from the third layer of the third trained model when input information based on second dynamic information is input to the third trained model. The first layer is a specific layer of an intermediate layer or an output layer. The second layer is a layer different from the first layer and is a specific layer of an intermediate layer or an output layer. The third layer is a layer different from the first layer and / or the second layer and is a specific layer of an intermediate layer or an output layer. This allows the walking posture state estimation system 2 to flexibly change the number of processing layers (number of processing steps) in each trained model, thereby suppressing increases in processing time and processing load.

[0183] The trained model 311 may be generated to correspond to more than one of the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200. In particular, the trained model 311 may be generated to correspond to all of the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200. That is, the trained model 311 includes a single trained model that can output a walking posture state estimated based on the static information acquired from the sole information acquisition device 500, the first dynamic information acquired from the first wearable device 100, and the second dynamic information acquired from each second wearable device 200.

[0184] Alternatively, the trained model 311 includes a single trained model capable of outputting an estimated walking posture state based on each static information acquired from each sole information acquisition device 500 and image data based on the first dynamic information and each second dynamic information acquired from the first wearable device 100 and each second wearable device 200.

[0185] In these cases, the trained model 311 is trained to output a walking posture state when a vector whose elements are static information, first dynamic information, and second dynamic information obtained from multiple wearable devices at corresponding times (approximately the same times) is input.

[0186] This allows the walking posture state estimation system 2 to estimate the walking posture state in a comprehensive manner based on information acquired from each sole information acquisition device 500, the first wearable device 100, and each second wearable device 200.

[0187] Furthermore, the trained model 311 includes a trained model corresponding to each of the first to fifth elements. That is, the trained model 311 includes a plurality of trained models capable of outputting information relating to each of the first to fifth elements. The trained model 311 may also include a single trained model capable of collectively outputting information relating to each of the first to fifth elements.

[0188] The trained model 311 may also include an upstream trained model and a downstream trained model arranged in series. In this case, the upstream trained model is trained to output a predetermined feature when input information is input, and the downstream trained model is trained to output a walking posture state when a feature output from an intermediate layer or an output layer of the upstream trained model is input. The predetermined feature is a feature related to the input information or a feature related to the walking posture state.

[0189] The trained model 311 may also include any combination of the trained models described above. For example, the trained model 311 includes a first upstream trained model, a second upstream trained model, a third upstream trained model, and a downstream trained model. The first upstream trained model is trained to output a predetermined feature amount when a first principal component of sole data is input. The second upstream trained model is trained to output a predetermined feature amount when a second principal component of sole data is input. The third upstream trained model is trained to output a predetermined feature amount when a first principal component of angular velocity or acceleration is input. Alternatively, the third upstream trained model is trained to output a predetermined feature amount when a spectral image transformed by short-time Fourier transform or wavelet transform on angular velocity and / or acceleration is input. The downstream trained model is trained to output a walking posture state when a feature vector whose elements are data output from the output layer or intermediate layer of the first upstream trained model, the second upstream trained model, and the third upstream trained model is input. This allows the walking posture state estimation system 2 to further improve the accuracy of the trained model 311.

[0190] Alternatively, the trained model 311 includes a first upstream trained model, a second upstream trained model, and a downstream trained model. The first upstream trained model is trained to output a predetermined feature amount when sole data is input. The second upstream trained model is trained to output a predetermined feature amount when angular velocity or acceleration is input. The downstream trained model is trained to output a walking posture state when a feature vector whose elements are data output from the output layer or intermediate layer of the first upstream trained model and the second upstream trained model is input. This allows the walking posture state estimation system 2 to further improve the accuracy of the trained model 311.

[0191] In this way, the trained model 311 is configured to be able to output the estimated walking posture state of the user wearing the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200.

[0192] The training data for the trained model 311 includes a combination of input information generated in advance and the walking posture state of a user wearing the sole information acquisition device 500 that generated the input information, the first wearable device 100, and the second wearable device 200. Each training data is created by an expert who views video images of a user wearing the device that generated the static information, the first dynamic information, or the second dynamic information when the static information, the first dynamic information, or the second dynamic information is generated by each device.

[0193] Each training data set may be composed of, for example, a set of static information, first dynamic information, or second dynamic information for a predetermined number of consecutive steps or a predetermined period, combined with a walking posture state. In this case, each training data set may be generated so that part of the static information, first dynamic information, or second dynamic information included in each training data set overlaps. That is, the first training data set may include information for a predetermined number of steps or a predetermined period from a first time, and the second training data set may include data for a predetermined number of steps or a predetermined period from a second time immediately after the first time (e.g., one second later). This allows the walking posture state estimation system 2 to efficiently generate the trained model 311 from a small amount of sample data, thereby reducing the time and effort required to generate the trained model 311.

[0194] When the trained model 311 is pre-trained using a random forest or the like, each training data set is composed of, for example, a combination of static information (the grayscale values ​​of each pixel of the sole image or each element of the sole vector) for a predetermined number of consecutive steps or a predetermined period, a vector in which the first dynamic information or the second dynamic information is arranged in one dimension, and a walking posture state. Each training data set may include, for example, only the angular velocities in the three axial directions of the first dynamic information or the second dynamic information. The walking posture state estimation system 2 can improve the accuracy of the trained model 311 by generating the trained model 311 based on both the angular velocities and the accelerations. On the other hand, the walking posture state estimation system 2 can reduce the time and effort required to generate the trained model 311 by generating the trained model 311 based only on the angular velocities. In these cases, each training data set may be generated so that part of the static information, the first dynamic information, or the second dynamic information included in each training data set overlaps.

[0195] The estimation unit 322 converts the static information, the first dynamic information, and the second dynamic information acquired in step S409 into a format compatible with the trained model 311 and inputs the converted information to the trained model 311. The estimation unit 322 estimates the walking posture state of the user wearing the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200, based on the information output from the trained model 311. The estimation unit 322 estimates the walking posture state of the user for each element included in the walking posture state.

[0196] When the information output from the trained model 311 is a feature, the information processing device 300 stores in advance a table or formula indicating the relationship between the feature and the walking posture state in the third storage device 310. The estimation unit 322 refers to the table or formula stored in the third storage device 310, and identifies the walking posture state of the user corresponding to the feature output from the trained model 311.

[0197] Furthermore, if the trained model 311 includes multiple trained models corresponding to each sole information acquisition device 500, the first wearable device 100, and each second wearable device 200, the estimation unit 322 estimates that the walking posture state output from the most trained models among the walking posture states output from each trained model is the walking posture state of the user.

[0198] In this way, the estimation unit 322 estimates the walking posture state of the user based on the acquired static information, first dynamic information, and second dynamic information by using the trained model 311. That is, the estimation unit 322 uses the trained model 311 to acquire an estimated walking posture state based on the acquired static information, first dynamic information, and second dynamic information.

[0199] Next, the output control unit 323 outputs the estimated walking posture state of the user wearing the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200 by transmitting it to the first wearable device 100 via the third communication device 301 (step S411).

[0200] Next, the first acquisition unit 121 of the first wearable device 100 acquires the estimated walking posture state of the user by receiving it from the information processing device 300 via the first communication device 103 (step S412).

[0201] Next, the first acquisition unit 121 outputs the acquired walking posture state by displaying it on the first display device 102 to notify the user (step S413), and the series of steps ends. This allows the user to recognize their own walking posture state.

[0202] Alternatively, one of steps S404 to S406 and S407 may be omitted, and the first acquisition unit 121 may estimate the walking posture state based on only the static information and one of the first dynamic information and the second dynamic information. By estimating the walking posture state based on the static information and the first dynamic information, the first wearable device 100 can estimate the walking posture state at low cost and with low load without using the second wearable device 200. On the other hand, by estimating the walking posture state based on the static information and the second dynamic information, the first wearable device 100 can increase the degree of freedom of the body part from which the second dynamic information is acquired, thereby enabling highly accurate estimation of the walking posture state. Furthermore, in step S410, the static information used to estimate the walking posture state and each piece of dynamic information do not need to be acquired at the same time. In particular, the static information may be information acquired when the user is stationary (e.g., standing naturally) rather than when walking.

[0203] In step S411, the output control unit 323 of the information processing device 300 may transmit the estimated walking posture state of the user wearing the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200 to an administrator device owned by an administrator who manages the user's health. In this case, the administrator device notifies the administrator by displaying the received walking posture state. The administrator can recognize the user's walking posture state and suggest improvements to the user's walking posture state.

[0204] Furthermore, the timing of execution of the processes of steps S401, S404, and S407 may be arbitrary, and may be simultaneous. For example, when the first acquisition unit 121 of the first wearable device 100 receives an instruction to start measurement from the user using the first input device 101, it transmits a measurement request signal requesting the start of measurement to the sole of foot information acquisition device 500 and the second wearable device 200 via the first interface device 104, and acquires first dynamic information. On the other hand, when the fifth acquisition unit 521 of the sole of foot information acquisition device 500 receives a measurement request signal from the first wearable device 100 via the fifth interface device 504, it acquires static information and transmits it to the first wearable device 100 via the fifth interface device 504. Furthermore, when the second acquisition unit 221 of the second wearable device 200 receives a measurement request signal from the first wearable device 100 via the second interface device 204, it acquires second dynamic information and transmits it to the first wearable device 100 via the second interface device 204. The first acquisition unit 121 repeats the above process until it receives an instruction to end measurement from the user using the first input device 101, and then executes the process of step S408.

[0205] The inventors evaluated the trained model 311 for all of the first to fifth elements and confirmed that the accuracy rate of the information output from the trained model 311 was 93% or higher, indicating sufficiently high accuracy. In other words, it was confirmed that the walking posture state estimation system 2 can estimate the user's walking posture state with higher accuracy by using the trained model 311. Furthermore, by using the trained model 311, the walking posture state estimation system 2 can accurately and efficiently calculate walking posture states for various combinations of various types of parameters. Furthermore, by using the trained model 311, the walking posture state estimation system 2 can identify walking posture states without storing walking posture states corresponding to various combinations of various types of parameters, thereby reducing the storage capacity of the storage device. Furthermore, by using the trained model 311, the walking posture state estimation system 2 can identify walking posture states without performing complex determinations according to various combinations of various types of parameters, thereby reducing the processing time and processing load of the estimation process.

[0206] The technical significance of the walking posture state including the first to fifth elements is as explained in the description of the walking posture state estimation system 1.

[0207] It is generally believed that repeated dynamic alignment accumulates in static alignment, and diagnostic indices tend to be established solely based on static alignment. However, the equivalence between dynamic alignment and static alignment is only 5% to 60%. Furthermore, if a diagnostic indices is determined solely based on static alignment, it takes a long time for the improvement effects of instruction to become clear. By estimating the user's walking posture state using a combination of dynamic and static information, the walking posture state estimation system 2 can confirm whether both dynamic and static alignment have improved, allowing for more accurate confirmation of whether the user's alignment is normal. Furthermore, by estimating the user's walking posture state using a combination of dynamic and static information, the walking posture state estimation system 2 can confirm the improvement effects of instruction in a shorter period of time.

[0208] As described above in detail, the walking posture state estimation system 2 estimates the walking posture state of the user using the trained model 311 based on the static information acquired by the sole information acquisition device 500 and the first dynamic information acquired by the first wearable device 100. This enables the walking posture state estimation system 2 to estimate the walking posture state of the user with higher accuracy.

[0209] FIG. 12 is a sequence showing an example of the operation of the estimation process in the walking posture state estimation system 2 according to another embodiment.

[0210] An example of the operation of the estimation process according to this embodiment will be described below with reference to the flowchart shown in FIG. 12. The operation flow described below is executed mainly by the processing devices of each device in cooperation with the elements of each device, based on a program previously stored in the storage device of each device included in the walking posture state estimation system 2. In this embodiment, the trained model 311 is stored in the server device 400. The processes of steps S501 to S509 and S516 to S518 in FIG. 12 are similar to the processes of steps S401 to S409 and S411 to S413 in FIG. 11, so their description will be omitted; only steps S510 to S515 will be described below.

[0211] In step S510, the estimation unit 322 of the information processing device 300 transmits a request signal requesting transmission of the user's walking posture information to the server device 400 via the third communication device 301 (step S510). The request signal includes the static information, the first dynamic information, and the second dynamic information acquired in step S406 and converted into a format corresponding to the trained model 311.

[0212] Next, the server device 400 receives a request signal from the information processing device 300 (step S511). Next, the server device 400 inputs the static information, the first dynamic information, and the second dynamic information included in the received request signal to the trained model 311, and acquires output information output from the trained model 311 (step S512). Next, the server device 400 transmits the acquired output information to the information processing device 300 (step S513).

[0213] Next, the estimation unit 322 of the information processing device 300 acquires the output information by receiving it from the server device 400 via the third communication device 301 (step S514). Next, the estimation unit 322 estimates the walking posture state of the user wearing the first wearable device 100 and the second wearable device 200 based on the acquired output information (step S515). The estimation unit 322 estimates the walking posture state of the user in the same manner as the processing of step S410 in FIG. 11 .

[0214] As described above in detail, the walking posture state estimation system 2 is able to estimate the user's walking posture state with greater accuracy even when the trained model 311 is stored in a server device 400 other than the information processing device 300.

[0215] By utilizing the trained model 311 stored in the server device 400, the walking posture state estimation system 2 can estimate the walking posture state of the user using the latest trained model updated by the server device 400. Furthermore, by utilizing the trained model 311 stored in the server device 400, the walking posture state estimation system 2 can reduce the storage capacity of the information processing device 300. Meanwhile, by utilizing the trained model 311 stored in the information processing device 300, the information processing device 300 can estimate the walking posture state of the user even when communication with the server device 400 is disconnected. Furthermore, by utilizing the trained model 311 stored in the information processing device 300, the information processing device 300 can reduce the amount of communication between the information processing device 300 and the server device 400.

[0216] FIG. 13 is a sequence diagram showing an example of the operation of the estimation process in a walking posture state estimation system 2 according to yet another embodiment.

[0217] An example of the operation of the estimation process according to this embodiment will be described below with reference to the flowchart shown in FIG. 13 . The operation flow described below is executed primarily by each processing device of each device in cooperation with each element of each device, based on a program previously stored in each storage device of each device included in the walking posture state estimation system 2. In this embodiment, the trained model 311 and the data table 312 are stored in the first storage device 110 of the first wearable device 100. The first processing device 120 functions as a first acquisition unit 121 and a first transmission unit 122, as well as an estimation unit and an output control unit having functions similar to those of the estimation unit 322 and the output control unit 323. The processes of steps S601 to S606 and S609 in FIG. 13 are similar to those of steps S401 to S406 and S413 in FIG. 11 , and therefore will not be described again. Only steps S607 to S608 will be described below.

[0218] In step S607, the first acquirer 121 of the first wearable device 100 acquires first dynamic information obtained when the user walks, similar to the process of step S407 (step S607). The first acquirer 121 stores the acquired static information, first dynamic information, and second dynamic information in the data table 312 in association with the corresponding device ID and acquisition time.

[0219] Next, the estimation unit of the first wearable device 100 estimates the walking posture state of the user wearing the sole information acquisition device 500, the first wearable device 100, and the second wearable device 200 based on the static information, the first dynamic information, and the second dynamic information, similar to the processing of step S410 (step S608).

[0220] As described above in detail, the walking posture state estimation system 2 can estimate the walking posture state of the user with higher accuracy even when the first wearable device 100 estimates the walking posture state of the user.

[0221] Although the preferred embodiments have been described above, the embodiments are not limited thereto. For example, even when the first wearable device 100 estimates the walking posture state of the user, the first wearable device 100 may estimate the walking posture state of the user based on the output information of the trained model 311 stored in the server device 400.

[0222] Furthermore, the sole-of-foot information acquisition device 500 and / or the second wearable device 200 may transmit the static information and / or the second dynamic information directly to the information processing device 300, rather than transmitting the static information and / or the second dynamic information to the information processing device 300 via the first wearable device 100. This allows the walking posture state estimation system 2 to reduce the processing load on the first wearable device 100. On the other hand, the first wearable device 100 aggregates the static information from the sole-of-foot information acquisition device 500 and the second dynamic information from the second wearable device 200, allowing the walking posture state estimation system 2 to reduce the processing load on the information processing device 300.

[0223] Alternatively, the walking posture state estimation system 2 may estimate the walking posture state based on the sole data only initially, and thereafter estimate the walking posture state based on the acceleration and angular velocity.

[0224] The second wearable device 200 may be omitted from the walking posture state estimation system 2. In the walking posture state estimation system 2, multiple first wearable devices 100 and / or multiple information processing devices 300 may cooperate to share the respective steps of the above-described processes.

[0225] 1, 2 Walking posture state estimation system, 100 First wearable device, 105 First sensor, 110 First storage device, 121 First acquisition unit, 122 First transmission unit, 200 Second wearable device, 205 Second sensor, 300 Information processing device, 301 Third communication device, 310 Third storage device, 311 Trained model, 321 Third acquisition unit, 322 Estimation unit, 500 Sole information acquisition device, 505 Fifth sensor

Claims

1. A first wearable device that can be worn by a user and acquires first data obtained when the user walks, a second wearable device that can be worn by the user and acquires second data different from the first data obtained when the user walks, and an information processing device that receives the first data and the second data and can output an estimated walking posture state of the user wearing the first wearable device and the second wearable device, wherein the information processing device has a storage unit that stores a learned model capable of outputting the walking posture state estimated based on the first data and the second data. A walking posture state estimation system characterized by the above.

2. The first wearable device has a receiving unit that receives the second data from the second wearable device, and a transmitting unit that transmits the first data and the second data for a predetermined number of steps of the user to the information processing device. The walking posture state estimation system according to claim 1.

3. The walking posture state includes any one or more of a state of step width, a state of center of gravity position, a state of the swinging leg, a state of the supporting leg, and a state of the hip joint. The walking posture state estimation system according to claim 1 or 2.

4. The learned model includes a first learned model capable of outputting a first walking posture state estimated based on the first data, and a second learned model capable of outputting a second walking posture state estimated based on the second data. The walking posture state estimation system according to claim 1 or 2.

5. The learned model is a single learned model capable of outputting the walking posture state estimated based on the first data and the second data. The walking posture state estimation system according to claim 1 or 2.

6. The learned model includes a first learned model capable of outputting a first walking posture state estimated based on first image data based on the first data, and a second learned model capable of outputting a second walking posture state estimated based on second image data based on the second data. The walking posture state estimation system according to claim 1 or 2.

7. The learned model is a single learned model capable of outputting the walking posture state estimated based on image data based on the first data and the second data. The walking posture state estimation system according to claim 1 or 2.

8. The information processing apparatus inputs data based on the first data into a first layer of a first learned model included in the learned model, and inputs data based on the second data into a second layer different from the first layer of a second learned model included in the learned model, thereby obtaining the walking posture state. The walking posture state estimation system according to claim 1 or 2.

9. The walking posture state includes information output from a first layer of the learned model when data based on the first data is input into the learned model, and information output from a second layer different from the first layer of the learned model when data based on the second data is input into the learned model. The walking posture state estimation system according to claim 1 or 2.

10. A first wearable device wearable by a user acquires first data obtained when the user walks, a second wearable device wearable by the user acquires second data different from the first data obtained when the user walks, and a learned model capable of outputting a walking posture state of the user wearing the first wearable device and the second wearable device estimated based on the first data and the second data is used to estimate the walking posture state based on the acquired first data and the second data, and output the walking posture state. A walking posture state estimation method characterized by the above.

11. An acquisition unit that acquires the first data and the second data from a first sensor that measures the first data obtained when the user walks and a second sensor that measures the second data different from the first data obtained when the user walks, and a learned model capable of outputting a walking posture state estimated based on the first data and the second data is used to obtain a walking posture state estimated based on the acquired first data and the second data. An estimation unit, characterized by comprising:

12. A walking posture state estimation method, comprising: acquiring first data obtained when a user walks from a first sensor, and second data different from the first data obtained when the user walks from a second sensor; and acquiring a walking posture state estimated based on the first data and the second data by using a learned model capable of outputting the walking posture state estimated based on the first data and the second data.

13. A control program for a walking posture state estimation device, causing the walking posture state estimation device to acquire first data obtained when a user walks from a first sensor, and second data different from the first data obtained when the user walks from a second sensor, and acquire a walking posture state estimated based on the first data and the second data by using a learned model capable of outputting the walking posture state estimated based on the first data and the second data.

14. A walking posture state estimation device, comprising: a first sensor that measures first data obtained when a user walks; an acquisition unit that acquires the second data from a second sensor that measures second data different from the first data obtained when the user walks; a storage unit that stores a learned model capable of outputting a walking posture state estimated based on the first data and the second data; and an estimation unit that acquires a walking posture state estimated based on the first data and the second data acquired by using the learned model.

15. A walking posture state estimation method by a walking posture state estimation device having a first sensor that acquires first data obtained when a first user walks, the method comprising: acquiring the second data from a second sensor that acquires second data different from the first data obtained when the user walks; storing a learned model capable of outputting a walking posture state estimated based on the first data and the second data; and acquiring a walking posture state estimated based on the first data and the second data acquired by using the learned model.

16. A control program for a walking posture state estimation device having a first sensor that acquires first data obtained when a first user walks, the control program causing the walking posture state estimation device to: acquire the second data from a second sensor that acquires second data different from the first data obtained when the user walks; store a learned model capable of outputting a walking posture state estimated based on the first data and the second data; and acquire the walking posture state estimated based on the acquired first data and second data by using the learned model.

17. A walking posture state estimation system, comprising: a sole information acquisition device that acquires static information regarding the sole of a user; a wearable device that can be worn by the user and acquires dynamic information obtained when the user walks; and an information processing device connected to the sole information acquisition device and the wearable device, wherein the information processing device includes: a receiving unit that receives the static information and the dynamic information; a storage unit that stores a learned model capable of outputting a walking posture state of the user estimated based on the static information and the dynamic information; and an output unit that outputs the walking posture state.

18. The walking posture state estimation system according to claim 17, further comprising a second wearable device that can be worn by the user and acquires second dynamic information obtained when the user walks, wherein the wearable device includes: a second dynamic information receiving unit that receives the second dynamic information from the second wearable device; and a transmitting unit that transmits the dynamic information and the second dynamic information for a predetermined number of steps of the user to the information processing device.

19. The walking posture state estimation system according to claim 17 or 18, wherein the walking posture state includes any one or more of a state of step width, a state of center of gravity position, a state of swing leg, a state of support leg, and a state of hip joint.

20. The walking posture state estimation system according to claim 17 or 18, wherein the learned model includes a first learned model capable of outputting a first walking posture state estimated based on the static information and a second learned model capable of outputting a second walking posture state estimated based on the dynamic information.

21. The walking posture state estimation system according to claim 17 or 18, wherein the learned model is a single learned model capable of outputting the walking posture state estimated based on the static information and the dynamic information.

22. The information processing apparatus inputs information based on the static information to a first layer of a first learned model included in the learned model, and inputs information based on the dynamic information to a second layer different from the first layer of a second learned model included in the learned model, thereby obtaining the walking posture state. The walking posture state estimation system according to claim 17 or 18.

23. The walking posture state includes information output from a first layer of the learned model when information based on the static information is input to the learned model, and information output from a second layer different from the first layer of the learned model when information based on the dynamic information is input to the learned model. The walking posture state estimation system according to claim 17.

24. A static information acquisition device acquires static information regarding the soles of a user's feet, a wearable device wearable by the user acquires dynamic information obtained when the user walks, and uses a learned model capable of outputting the walking posture state of the user estimated based on the static information and the dynamic information to estimate the walking posture state based on the acquired static information and dynamic information, and outputs the walking posture state. A walking posture state estimation method characterized by the above.

25. An acquisition unit that acquires the static information and the dynamic information from a static information sensor that measures static information regarding the soles of a user's feet and a dynamic information sensor that measures dynamic information obtained when the user walks, and uses a learned model capable of outputting the walking posture state of the user estimated based on the static information and the dynamic information to obtain an estimated walking posture state estimated based on the acquired static information and dynamic information, and an output unit that outputs the walking posture state. A walking posture state estimation device characterized by having the above.

26. A walking posture state estimation method, comprising: acquiring the static information and the dynamic information from a static information sensor that measures static information regarding the sole of a user's foot and a dynamic information sensor that measures dynamic information obtained when the user walks; obtaining a walking posture state estimated based on the static information and the dynamic information by using a learned model capable of outputting the walking posture state of the user; and outputting the walking posture state.

27. A control program for a walking posture state estimation device, which causes the walking posture state estimation device to: acquire the static information and the dynamic information from a static information sensor that measures static information regarding the sole of a user's foot and a dynamic information sensor that measures dynamic information obtained when the user walks; obtain a walking posture state estimated based on the static information and the dynamic information by using a learned model capable of outputting the walking posture state of the user; and output the walking posture state.

28. A walking posture state estimation device, comprising: an acquisition unit that acquires the static information from a static information sensor that measures static information regarding the sole of a user's foot; a sensor that measures dynamic information obtained when the user walks; an estimation unit that obtains a walking posture state estimated based on the static information and the dynamic information by using a learned model capable of outputting the walking posture state of the user; and an output unit that outputs the walking posture state.

29. A walking posture state estimation method by a walking posture state estimation device having a sensor that measures dynamic information obtained when the user walks, the method comprising: acquiring the static information from a static information sensor that measures static information regarding the sole of a user's foot; obtaining a walking posture state estimated based on the static information and the dynamic information by using a learned model capable of outputting the walking posture state of the user; and outputting the walking posture state.

30. A control program for a walking posture state estimation device having a sensor that measures dynamic information obtained when a user walks, the program comprising: obtaining the static information from a static information sensor that measures static information regarding the sole of the user; using a learned model capable of outputting the walking posture state of the user estimated based on the static information and the dynamic information, obtaining the walking posture state estimated based on the obtained static information and the dynamic information; and causing the walking posture state estimation device to output the walking posture state. A control program characterized by the above.

Images