Information processing device, information processing method, program, information processing system, and generation method

Abstract

To determine properly an abnormality of a lower limb.SOLUTION: An information processing unit (10) has an acquisition part (15) for acquiring information based on a sensor (21) attached to a user's leg, a determination part (16) for determining an abnormality of user's walking based on acceleration in a walking direction of the user acquired by the acquisition part, and an output part (17) for outputting information based on a determination result by the determination part.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, a program, an information processing system, and a generation method. [Background technology]

[0002] BACKGROUND ART Techniques for diagnosing (determining) abnormalities in the lower limbs based on data measured by a sensor are known (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-261525 [Patent Document 2] Japanese Patent Application Publication No. 2019-150229 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the techniques described in Patent Documents 1 and 2 have a problem in that, for example, abnormalities in the lower limbs may not be properly determined.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, an information processing method, a program, and an information processing system that can appropriately determine abnormalities in the lower limbs. [Means for solving the problem]

[0006] In a first aspect of the present disclosure, an information processing device has an acquisition unit that acquires information based on a sensor attached to a user's foot, a determination unit that determines abnormalities in the user's walking based on the acceleration in the user's walking direction acquired by the acquisition unit, and an output unit that outputs information based on the determination result by the determination unit.

[0007] In addition, a second aspect of the present disclosure provides an information processing method that acquires information based on a sensor attached to a user's foot, determines abnormalities in the user's gait based on the acceleration in the user's walking direction acquired by the acquisition unit, and outputs information based on the determination result.

[0008] In addition, in a third aspect of the present disclosure, a program is provided that causes an information processing device to execute the following processes: acquiring information based on a sensor attached to a user's foot; determining abnormalities in the user's walking based on the acceleration in the user's walking direction acquired by the acquisition unit; and outputting information based on the determination result.

[0009] In addition, a fourth aspect of the present disclosure provides an information processing system including a sensor attached to a user's foot, an information processing device, and an information processing terminal, the information processing system including an acquisition unit that acquires information based on the acceleration in the user's walking direction measured by the sensor, a determination unit that determines whether the user's walking is abnormal based on the acceleration in the user's walking direction acquired by the acquisition unit, and an output unit that outputs information based on the determination result by the determination unit to the information processing terminal.

[0010] In addition, a fifth aspect of the present disclosure provides a generation method for acquiring a dataset including a combination of information based on acceleration in the user's walking direction measured by a sensor attached to the user's foot and information indicating abnormalities in the user's walking, generating a trained model that determines abnormalities in the user's walking from the information measured by the sensor attached to the user's foot based on the acceleration in the user's walking direction acquired by the acquisition unit, and outputting the generated trained model. [Effects of the Invention]

[0011] According to one aspect, abnormalities in the lower limbs can be appropriately determined. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device that performs generation processing according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device that performs a determination process according to an embodiment. [Figure 3] 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 5] FIG. 4 is a diagram illustrating an example of a position where a sensor according to an embodiment is attached. [Figure 6] FIG. 1 is a diagram illustrating an example of the configuration of a measurement device according to an embodiment. [Figure 7] 1 is a flowchart illustrating an example of a process for generating a trained model according to an embodiment. [Figure 8] FIG. 1 is a diagram illustrating an example of a dataset for machine learning according to an embodiment. [Figure 9] FIG. 10 is a diagram showing an example of the transition of acceleration in the walking direction while a healthy user takes one step. [Figure 10] FIG. 1 is a diagram illustrating an example of a walking motion of a healthy person. [Figure 11] FIG. 1 is a diagram illustrating an example of a walking motion of a healthy person. [Figure 12] FIG. 1 is a diagram illustrating an example of a walking motion of a healthy person. [Figure 13] 10 is a flowchart illustrating an example of a determination process of the information processing system according to the embodiment. [Figure 14] FIG. 10 is a diagram showing an example of the transition of acceleration in the walking direction while a healthy person and a patient take one step, based on data measured by a sensor according to an embodiment. [Figure 15] 10A and 10B are diagrams illustrating an example of frequency components of acceleration in the walking direction while a healthy person and a patient take one step, based on data measured by a sensor according to an embodiment. [Figure 16] 10A and 10B are diagrams showing an example of the transition of the angle of the walking direction between the sole of the foot and the ground while a healthy person and a patient take a step, based on data measured by a sensor according to an embodiment. [Figure 17] 10A and 10B are diagrams illustrating an example of frequency components of vertically upward acceleration during a step taken by a healthy subject and a patient, based on data measured by a sensor according to an embodiment. [Figure 18] FIG. 10 is a diagram showing an example of the ratio between acceleration A and acceleration B during a walking step by a healthy person and a patient, based on data measured by a sensor according to an embodiment. [Figure 19] FIG. 10 is a diagram showing an example of the ratio between acceleration A and acceleration C during a step taken by a healthy person and a patient, based on data measured by a sensor according to an embodiment. [Figure 20] 10A and 10B are diagrams showing examples of single oscillation components of acceleration in the walking direction while a healthy person and a patient take one step, based on data measured by a sensor according to an embodiment. [Figure 21] FIG. 10 is a diagram showing an example of the difference between the maximum and minimum values ​​of the angle of the walking direction between the sole of the foot and the ground while a healthy person and a patient take a step, based on data measured by a sensor according to an embodiment. [Figure 22] 10A and 10B are diagrams showing an example of the fifth oscillatory component of the vertical upward acceleration during a step taken by a healthy subject and a patient, based on data measured by a sensor according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] The principles of the present disclosure will be described with reference to some exemplary embodiments. It should be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the present disclosure, without implying any limitation on the scope of the disclosure. The disclosure described herein may be implemented in various ways other than those described below.

[0014] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (Embodiment 1) <Configuration> <<Information processing device 10A that performs generation processing>> The configuration of an information processing device 10A that performs generation processing according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information processing device 10A that performs generation processing according to an embodiment. The information processing device 10A (for example, a server 40 described later) has an acquisition unit 11, a generation unit 12, and an output unit 13. Each of these units may be realized by cooperation between one or more programs installed in the information processing device 10A and hardware such as a processor 101 and a memory 102 of the information processing device 10A.

[0016] The acquisition unit 11 acquires various types of information from a storage unit within the information processing device 10A or an external device. The acquisition unit 11 acquires, for example, a data set including a combination of information based on acceleration in the user's walking direction (direction of travel) measured by a sensor attached to the user's foot and information indicating abnormalities in the user's gait.

[0017] Based on the information acquired by the acquisition unit 11, the generation unit 12 generates a trained model that determines abnormalities in the user's gait from information measured by sensors attached to the user's feet.

[0018] The output unit 13 outputs (transmits, records) various pieces of information to a storage unit within the information processing device 10A or an external device. The output unit 13 outputs, for example, the trained model generated by the generation unit 12.

[0019] <<Information processing device 10B performing determination processing>> Next, the configuration of the information processing device 10B according to the embodiment will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of the configuration of the information processing device 10B that performs the determination process according to the embodiment. The information processing device 10B (for example, a server 40 described later) has an acquisition unit 15, a determination unit 16, and an output unit 17. These units may be realized by cooperation between one or more programs installed in the information processing device 10B and hardware such as a processor 101 and a memory 102 of the information processing device 10B. Note that the information processing device 10A and the information processing device 10B may be the same information processing device or may be different information processing devices. Note that hereinafter, when there is no need to distinguish between the information processing device 10A and the information processing device 10B, they will also be simply referred to as "information processing device 10."

[0020] The acquisition unit 15 acquires various information from a storage unit within the information processing device 10B or an external device. The acquisition unit 15 acquires information based on the acceleration in the user's walking direction, which is measured by a sensor attached to the user's foot, for example.

[0021] The determination unit 16 determines whether there is an abnormality in the user's gait based on the information acquired by the acquisition unit 15. The output unit 17 outputs (transmits, displays, notifies, records) various types of information to a storage unit within the information processing device 10B or an external device. The output unit 17 outputs, for example, information based on the determination result by the determination unit 16.

[0022] (Embodiment 2) Next, the configuration of the information processing system 1 according to the embodiment will be described with reference to FIG. <System configuration> FIG. 3 is a diagram showing an example of the configuration of an information processing system 1 according to an embodiment. In the example of FIG. 3, the information processing system 1 includes a measuring device 20A and a measuring device 20B (hereinafter, when there is no need to distinguish between them, they will also be simply referred to as "measuring devices 20"). The information processing system 1 also includes a user terminal 30, a server 40, and a hospital terminal 50. Note that the numbers of measuring devices 20, user terminals 30, servers 40, and hospital terminals 50 are not limited to those shown in the example of FIG. 3. Note that the measuring devices 20, user terminals 30, servers 40, and hospital terminals 50 are each an example of an information processing device 10.

[0023] In the following, as an example, a description will be given of a case in which the server 40 performs a generation (learning) process and a judgment (estimation, inference) process of a trained model. The generation process may be executed by one or more of the measurement device 20, the user terminal 30, the server 40, and the hospital terminal 50. The judgment process may be executed by one or more of the measurement device 20, the user terminal 30, the server 40, and the hospital terminal 50. The generation process and the judgment process may be executed by the same device or by different devices.

[0024] The measurement device 20 and the user terminal 30 may be connected to each other so as to be able to communicate with each other via short-range wireless communication such as BLE (Bluetooth (registered trademark) Low Energy) or a cable.

[0025] 3, the user terminal 30, the server 40, and the hospital terminal 50 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include the Internet, a mobile communication system, a wireless LAN (Local Area Network), a short-range wireless communication such as BLE, a LAN, and a bus. Examples of the mobile communication system include a fifth-generation mobile communication system (5G), a fourth-generation mobile communication system (4G), a third-generation mobile communication system (3G), and the like.

[0026] The measuring device 20 has a sensor 21 attached to the user's foot. The measuring device 20 outputs data measured using the sensor 21 to an external device such as a user terminal 30.

[0027] The user terminal 30 may be, for example, a smartphone, a tablet, a personal computer, an IoT (Internet of Things) communication device, a mobile phone, etc. The user terminal 30 transmits the data acquired from the measurement device 20 to the server 40.

[0028] The server 40 is, for example, a device such as a server, a cloud, a personal computer, or a smartphone. The server 40 determines whether there is an abnormality in the user's gait based on the data measured by the measuring device 20. The server 40 also transmits information based on the determination result to the hospital terminal 50.

[0029] The hospital terminal 50 may be, for example, a device such as a smartphone, a tablet, a personal computer, or a mobile phone. The hospital terminal 50 may be a terminal used in a facility such as a hospital. The hospital terminal 50 displays the information received from the server 40 on a screen.

[0030] <Hardware configuration> Fig. 4 is a diagram showing an example of the hardware configuration of an information processing device 10 according to an embodiment. In the example of Fig. 4, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These components may be connected via a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface required for communication with other network elements.

[0031] When the program 104 is executed by the processor 101, memory 102, and other components in cooperation with each other, the computer 100 performs at least some of the processing of the embodiments of the present disclosure. The memory 102 may be of any type suitable for a local technology network. The memory 102 may be, by way of non-limiting example, a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. While only one memory 102 is shown in the computer 100, several physically distinct memory modules may be present in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and, by way of non-limiting example, a processor based on a multi-core processor architecture. The computer 100 may have multiple processors, such as application-specific integrated circuit chips time-slaved to a clock that synchronizes the main processor.

[0032] Embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.

[0033] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor or device to perform the processes or methods of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or divided among program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.

[0034] The program code for executing the methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus. When the program code is executed by the processor or controller, the functions / acts in the flowcharts and / or implementing block diagrams are performed. The program code may be executed entirely on the machine, partly on the machine, as a standalone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0035] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROMs (Read Only Memory), CD-Rs (Recordable), and CD-RWs (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0036] <Regarding the measuring device 20> Next, an example of the measuring device 20 according to the embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing an example of a position where the sensor 21 according to the embodiment is attached. Fig. 6 is a diagram showing an example of the configuration of the measuring device 20 according to the embodiment.

[0037] 5, the measuring device 20 is housed (placed) in a recess 502 in an insole 501 of a shoe worn by a user. Note that the sensor 21 of the measuring device 20 may be attached to a position on the sole of the user's foot, anywhere between the arch and the heel.

[0038] In the example of FIG. 6, the measurement device 20 includes a sensor 21, a control device 22, and a communication device 23. The sensor 21 measures, for example, acceleration and angular velocity. The sensor 21 may be, for example, an inertial measurement unit (IMU) having a three-directional acceleration sensor and a three-axis gyro sensor. The control device 22 outputs data measured using the sensor 21 to an external device using the communication device 23. The control device 22 may have a configuration similar to that of the computer 100 shown in FIG. 4. In this case, the control device 22 may be, for example, a microcontroller.

[0039] <Processing> <<Generation process>> Next, an example of a process (learning process) for generating a trained model of the information processing device 10A according to the embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a flowchart showing an example of a process for generating a trained model according to the embodiment. Fig. 8 is a diagram showing an example of a data set for learning according to the embodiment. Fig. 9 is a diagram showing an example of a transition in acceleration in the walking direction while a healthy user takes one step. Figs. 10 to 12 are diagrams showing an example of the walking motion of a healthy user.

[0040] In step S101, the acquisition unit 11 of the information processing device 10A acquires a training dataset 801. In the example of FIG. 8, each record of the training dataset 801 includes data for one or more items indicating features related to walking and a correct answer label. The data for each item may be data calculated based on data on at least one of acceleration and angular velocity measured by the sensor 21. The data for each item may be calculated by any device, such as the measuring device 20, the user terminal 30, or the server 40, by processing similar to steps S202 and S203 in FIG. 13, which will be described later.

[0041] The correct label is a correct label in supervised learning. The correct label may include, for example, information indicating the presence or absence of an abnormality. The correct label may also include, for example, information indicating the severity of the abnormality. In this case, for example, the severity of the abnormality may be expressed as a value on a 10-point scale from 0 to 9, with severity "0" indicating no abnormality and severity "9" indicating the highest severity. The correct label may be designated (set, registered) by, for example, an operator of the server 40, a user of the hospital terminal 50 (for example, a doctor, etc.), or a user of the user terminal 30, etc.

[0042] ((About the walking movement of healthy people)) 9 shows an example of a transition 901 of acceleration in the walking direction while a healthy user takes one step, from time 0 to time T. In addition, in FIGS. 10 to 12, the walking direction is indicated as the Y direction, the vertical upward direction is indicated as the Z direction, and the angle between the sole of the foot and the ground in the walking direction is indicated as θ.

[0043] In the example of FIG. 9, at time point 911, as shown in FIG. 10, thigh muscles 953 are not exerted with force, and lower leg muscles 952 are exerted with force and contract, causing the user to lift their heel off the ground as indicated by arrow 951. Also, at time point 912, as shown in FIG. 11, thigh muscles 953 and lower leg muscles 952 are not exerted with force, and the toes of the user's feet leave the ground (Toe off). Note that hereinafter, the time point at which the toes of the user's feet leave the ground as shown in FIG. 11 is also referred to as "time point B" as appropriate. Also, the value of the acceleration in the walking direction at time point B is also referred to as "acceleration B" (an example of "first acceleration") as appropriate.

[0044] At time point 913, as shown in Fig. 12, force is applied to thigh muscles 953, causing them to contract, while no force is applied to lower leg muscles 952, causing the user's entire lower leg to swing forward. Note that hereinafter, the time point at which the user's entire lower leg swings forward as shown in Fig. 12 will also be referred to as "time point A" where appropriate. The value of the acceleration in the walking direction at time point A will also be referred to as "acceleration A" (an example of "second acceleration") where appropriate.

[0045] 9, when a healthy person is walking, the transition 901 of acceleration in the walking direction forms a W-shaped waveform with two peaks at time 912 when the toes of the user's feet leave the ground and time 913 when the entire lower leg of the user is swung forward. This is thought to be because at time 912 when the toes of the user's feet leave the ground, sufficient forefoot rocker and knee flexion are performed to ensure smooth acceleration.

[0046] At time point 914, before the foot lands, the user rapidly decelerates by returning the knee from a flexed state to an extended state. Hereinafter, the time point when the user's foot lands will also be referred to as "time point C" as appropriate. The value of the acceleration in the walking direction at time point C will also be referred to as "acceleration C" (an example of "third acceleration") as appropriate.

[0047] (Example using acceleration B) The data items included in the records of the learning dataset 801 may include, for example, a data item based on acceleration B. This is because a user (patient) with osteoarthritis (OA) or the like is unable to sufficiently bend the knee, and therefore the acceleration in the walking direction when the toes of the feet leave the ground is thought to be degenerate (reduced) compared to a healthy person.

[0048] In this case, the item may be based on acceleration B and at least one of acceleration A and the user's walking speed. In this case, the item may include an item based on the ratio between acceleration A and acceleration B (for example, the ratio between acceleration A and acceleration B, or its reciprocal). This is because there is no significant difference in acceleration A between healthy people and patients with OA or the like, and therefore the influence of differences in walking speed on the ratio between acceleration A and acceleration B, or its reciprocal, is reduced. Furthermore, the item of data based on acceleration B may include an item based on the ratio between walking speed and acceleration B (for example, walking speed / acceleration B, or its reciprocal). Note that walking speed may be calculated based on the transition of acceleration in the walking direction measured by sensor 21. Note that the symbol " / " indicates division. Therefore, for example, the notation X / Y indicates the ratio between X and Y.

[0049] (Example using acceleration C) The data items included in the records of the learning dataset 801 may include, for example, a data item based on acceleration C. This is because patients with OA or the like have insufficient knee motor ability and therefore gradually reduce their walking speed during the swing phase, which is thought to result in a degenerate (decreased) acceleration C compared to healthy individuals.

[0050] In this case, the item may be based on acceleration C and / or acceleration A and the user's walking speed. In this case, the item may include an item based on the ratio of acceleration A to acceleration C (for example, acceleration A / acceleration C, or its reciprocal). This is because there is no significant difference in acceleration A between healthy people and patients with OA or the like, and therefore the influence of differences in walking speed on the ratio value of acceleration A to acceleration C or its reciprocal is reduced. Furthermore, the item of data based on acceleration C may include an item based on the ratio of walking speed to acceleration C (for example, walking speed / acceleration C, or its reciprocal).

[0051] Furthermore, the data items included in the records of the training dataset 801 may include, for example, an item for the magnitude of the single oscillation component of acceleration in the user's walking direction while the user takes a step. Furthermore, the data items included in the records of the training dataset 801 may include, for example, an item based on the angle between the sole of the foot and the ground in the walking direction while the user takes a step. This is because patients with OA or the like are unable to flex their ankles (dorsiflexion and plantar flexion) sufficiently, and therefore the angle between the sole of the foot and the ground in the walking direction is thought to be degenerate (reduced) compared to healthy people.

[0052] Furthermore, the data items included in the records of the training data set 801 may include, for example, the magnitude of the fifth oscillation component of the user's vertical acceleration while the user takes one step.

[0053] Next, the generation unit 12 of the information processing device 10A generates a trained model using a supervised machine learning method based on the machine learning dataset 801 (step S102). Here, the information processing device 10A may use, for example, a weighted K-Nearest Neighbor Algorithm. In this case, the information processing device 10A may add, for example, a weight inversely proportional to the Euclidean distance and generate a trained model with a K value of 10 (10 neighbors).

[0054] Furthermore, the information processing device 10A may generate a trained model using any supervised learning machine learning method, such as a neural network (NN), a random forest, a linear regression, or a support vector machine.

[0055] Next, the output unit 13 of the information processing device 10A outputs the trained model generated by the generation unit 12 (step S103). Here, the information processing device 10A may record the generated trained model in an internal storage unit. Alternatively, the information processing device 10A may transmit (distribute) the generated trained model to an external device and record it therein.

[0056] <<Determination process>> Next, an example of the determination process of the information processing device 10B according to the embodiment will be described with reference to FIG. 13. FIG. 13 is a flowchart showing an example of the determination process of the information processing system 1 according to the embodiment. FIG. 14 is a diagram showing an example of the transition of acceleration in the walking direction while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 15 is a diagram showing an example of the frequency component of acceleration in the walking direction while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 16 is a diagram showing an example of the transition of the angle in the walking direction between the sole of the foot and the ground while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 17 is a diagram showing an example of the frequency component of acceleration in the vertical upward direction while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment.

[0057] In step S201, the acquisition unit 15 of the information processing device 10B acquires data related to the user's walking measured by the sensor 21. Subsequently, the determination unit 16 of the information processing device 10B extracts data related to the user's walking while the user takes one step (step S202). Here, the information processing device 10B may determine the start and end points of one step based on, for example, the transition of the angle between the sole of the user's foot and the ground in the walking direction. In this case, the information processing device 10B may determine the start and end points of one step based on, for example, the time period during which the angle between the sole of the user's foot and the ground in the walking direction reaches a maximum value, then a minimum value, and then a maximum value again.

[0058] Furthermore, the information processing device 10B may determine the start and end points of one step taken by the user based on, for example, the transition of the acceleration in the walking direction. In this case, the information processing device 10B may determine the start and end points of one step taken by the user based on, for example, the time period from when the acceleration in the walking direction reaches a maximum value to when it reaches a minimum value and then reaches a maximum value again.

[0059] The information processing device 10B may then normalize the time length (walking cycle) during which the user takes one step, as shown on the horizontal axis of FIG. 14. This makes it possible to determine gait abnormalities, for example, based on data measured at an arbitrary walking speed. The information processing device 10B then extracts acceleration A, acceleration B, and acceleration C from the data during which the user takes one step, with the time length normalized. FIG. 14 shows an example of a transition 1401 of acceleration in the walking direction during a step taken by a healthy person, and a transition 1402 of acceleration in the walking direction during a step taken by a patient with OA or the like. In the example of FIG. 14, time point 1411 corresponds to time point B, time point 1412 corresponds to time point A, and time point 1413 corresponds to time point C.

[0060] For example, when two local maximum values ​​of the acceleration in the walking direction are adjacent within a predetermined range after normalizing the duration of a user's walking step to a time-independent value, the information processing device 10B may determine the time point at which the second local maximum value is reached as time point A. This allows, for example, the time point A for a healthy person to be appropriately determined. Furthermore, for example, the information processing device 10B may determine, as time point A, the time point at which the acceleration in the walking direction during the user's walking step is at its maximum value.

[0061] Furthermore, for example, when two local maximum values ​​of acceleration in the walking direction are adjacent within a predetermined range after the length of time the user takes to walk one step is normalized to a time-independent value, the information processing device 10B may determine the time point at which the first local maximum value is obtained as time point B. This allows, for example, the information processing device 10B to appropriately determine time point B for a healthy person. Furthermore, for example, the information processing device 10B may determine, as time point B, a time point that is a predetermined length (for example, 5% of the walking cycle of one step) before the maximum value of acceleration in the walking direction after the length of time the user takes to walk one step is normalized to a time-independent value.

[0062] Furthermore, the information processing device 10B may determine as time point C, for example, the time point at which the acceleration in the walking direction within a predetermined range while the user takes one step becomes the minimum value.

[0063] Next, the determination unit 16 of the information processing device 10B calculates a feature amount related to walking based on data related to the user's walking while the user takes one step (step S203). Here, the information processing device 10B calculates data of one or more items that are the same as each item of data included in the records of the learning dataset 801 in Fig. 8. Therefore, the information processing device 10B may calculate, for example, at least one of the ratio between acceleration A and acceleration B and the ratio between acceleration A and acceleration C.

[0064] Furthermore, the information processing device 10B may calculate the magnitude of a single oscillation component of acceleration in the user's walking direction while the user takes one step, as shown in FIG. 15 . In this case, the information processing device 10B may first convert the acceleration transition from a time-dependent waveform to a frequency-dependent waveform (frequency) by, for example, Fourier transforming a waveform showing the transition of acceleration in the user's walking direction while the user takes one step. Then, the information processing device 10B may extract, as the single oscillation component, the value of the waveform at a frequency (frequency of 1) at which the user vibrates once while taking one step. FIG. 15 shows an example of a frequency component 1501 of acceleration in the walking direction of a healthy person taking one step and a frequency component 1502 of acceleration in the walking direction of a patient with OA or the like taking one step. FIG. 15 also shows an example of a single oscillation component 1511 of the frequency component 1501 of the healthy person and a single oscillation component 1512 of the frequency component 1502 of the patient with OA or the like.

[0065] The information processing device 10B may also calculate an item based on, for example, the angle of the walking direction between the sole of the foot and the ground while the user takes a step. In this case, the item may include, for example, the difference between the maximum and minimum values ​​of the angle of the walking direction between the sole of the foot and the ground while the user takes a step. FIG. 16 shows an example of a transition 1601 of the angle of the walking direction between the sole of the foot and the ground while a healthy person takes a step, and a transition 1602 of the angle of the walking direction between the sole of the foot and the ground while a patient with OA or the like takes a step. FIG. 16 also shows an example of a maximum value 1611 and a minimum value 1613 of the transition 1601 for the healthy person, and a maximum value 1612 and a minimum value 1614 of the transition 1602 for the patient with OA or the like.

[0066] Furthermore, the information processing device 10B may calculate, for example, the magnitude of a fifth-frequency component of the user's vertical acceleration while the user takes one step. In this case, the information processing device 10B may first convert the transition of the acceleration from a time-dependent waveform to a frequency-dependent waveform (vibration frequency) by, for example, Fourier transforming a waveform showing the transition of the user's vertical acceleration while the user takes one step. Then, the information processing device 10B may extract, as the fifth-frequency component, the value of the waveform at a frequency at which the user vibrates five times while taking one step. FIG. 17 shows an example of a frequency component 1701 of the vertical upward acceleration of a healthy person taking one step and a frequency component 1702 of the vertical upward acceleration of a patient with OA or the like taking one step. FIG. 17 also shows an example of a fifth-frequency component 1711 of the frequency component 1701 of the healthy person and a fifth-frequency component 1712 of the frequency component 1702 of the patient with OA or the like.

[0067] Next, the determination unit 16 of the information processing device 10B determines (estimates, infers) the user's gait abnormality based on the calculated feature data and the trained model generated by the generation unit 12 (step S204). Here, the information processing device 10B uses the calculated feature data and the trained model as explanatory variables to infer a correct label (objective variable). This makes it possible to, for example, evaluate the progression of the patient's condition, compare before and after surgery, and evaluate the degree of recovery.

[0068] Next, the output unit 17 of the information processing device 10B outputs information based on the determination result by the determination unit 16 (step S205). Here, the information processing device 10B may, for example, display a message indicating the presence or absence of an abnormality and its severity. The information processing device 10B may also notify the presence or absence of an abnormality and its severity by, for example, sound or lighting a lamp. The information processing device 10B may also record, for example, transitions in the value indicating the likelihood of the presence or absence of an abnormality for a specific user calculated in step S204 at multiple dates and times (for example, every week). Then, the information processing device 10B may output a message of rehabilitation advice for the user based on the transitions.

[0069] (About the effect of each item used in inference) Next, with reference to FIGS. 18 to 22, an example of the effect of each item (feature amount, explanatory variable) used in inference will be described. FIG. 18 is a diagram showing an example of the ratio between acceleration A and acceleration B while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 19 is a diagram showing an example of the ratio between acceleration A and acceleration C while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 20 is a diagram showing an example of the single-oscillary component of acceleration in the walking direction while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 21 is a diagram showing an example of the difference between the maximum and minimum values ​​of the angle between the sole of the foot and the ground in the walking direction while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment. FIG. 22 is a diagram showing an example of the fifth-oscillary component of acceleration in the vertical upward direction while a healthy person and a patient take one step, based on data measured by the sensor 21 according to the embodiment.

[0070] Figures 18 to 22 show examples of measurement data for each item in nine healthy elderly people and 16 OA patients. The values ​​for each item in each group of healthy people and each group of OA patients were extracted from the normalized waveform for each step, and the average and variance for all steps for all users in each group were calculated. Comparison of the values ​​for each item was performed using a two-sided test using the t-distribution. In the following, the P value indicates the probability that a test statistic will take that value under the null hypothesis in a statistical hypothesis test. The smaller the P value, the less likely it is that the test statistic will take that value. A P value of less than 5% (P<0.05, significance level 5%) is considered to indicate a difference between the two groups.

[0071] FIG. 18 shows an example of a ratio value 1801 between acceleration A and acceleration B in a group of healthy subjects, and an example of a ratio value 1802 between acceleration A and acceleration B in a group of OA patients. The P value in the case of FIG. 18 is approximately 0.004. FIG. 19 shows an example of a ratio value 1901 between acceleration A and acceleration C in a group of healthy subjects, and an example of a ratio value 1902 between acceleration A and acceleration C in a group of OA patients. The P value in the case of FIG. 19 is approximately 0.002.

[0072] Figure 20 shows an example of a single oscillation component 2001 of acceleration in the walking direction during a step taken by a group of healthy subjects, and an example of a single oscillation component 2002 of acceleration in the walking direction during a step taken by a group of OA patients. The P value in Figure 20 is less than 0.001. Figure 21 shows an example of a difference 2101 between the maximum and minimum values ​​of the angle between the sole of the foot and the ground in the walking direction during a step taken by a group of healthy subjects, and an example of a difference 2102 between the maximum and minimum values ​​of the angle between the sole of the foot and the ground in the walking direction during a step taken by a group of OA patients. The P value in Figure 21 is less than 0.001.

[0073] Figure 22 shows an example of a five-oscillary component 2201 of the vertical acceleration of a user during a step in the healthy subject group and a five-oscillary component 2202 of the vertical acceleration of a user during a step in the OA patient group. The P value in Figure 22 is less than 0.001.

[0074] <Modification> The information processing device 10 may be a device contained in a single housing, but the information processing device 10 of the present disclosure is not limited to this. Each unit of the information processing device 10 may be realized, for example, by cloud computing configured with one or more computers. Furthermore, each unit of the information processing device 10 may be realized by multiple devices, for example, the measurement device 20, the user terminal 30, the server 40, and the hospital terminal 50. Such information processing devices 10 are also included as examples of the "information processing device" of the present disclosure.

[0075] The present invention is not limited to the above-described embodiment, and can be modified as appropriate within the scope of the invention.

[0076] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an acquisition unit that acquires information based on sensors attached to the user's feet; a determination unit that determines whether the user has an abnormality in walking based on the acceleration in the user's walking direction acquired by the acquisition unit; an output unit that outputs information based on the determination result by the determination unit; An information processing device having the above. (Appendix 2) The acquisition unit acquiring information based on a sensor attached to a position anywhere between the arch and the heel of the user's foot; 10. The information processing device according to claim 1. (Appendix 3) The determination unit a first acceleration in the walking direction when the toe of the foot leaves the ground; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; 3. The information processing device according to claim 1 or 2. (Appendix 4) The determination unit a third acceleration in the walking direction when the foot lands; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; 4. The information processing device according to claim 1. (Appendix 5) The determination unit determining whether the user's walking is abnormal based on the magnitude of a single oscillation component of acceleration in the user's walking direction while the user takes one step; 5. The information processing device according to any one of claims 1 to 4. (Appendix 6) The determination unit determining an abnormality in the user's walking based on at least one of an angle of a walking direction between the sole of the foot and the ground while the user takes a step and a magnitude of a five-oscillating component of the user's vertical acceleration while the user takes a step; 6. An information processing device according to any one of appendices 1 to 5. (Appendix 7) Acquire information based on sensors attached to the user's feet; determining an abnormality in the user's walking based on the acquired acceleration in the user's walking direction; Output information based on the judgment result, An information processing method that performs processing. (Appendix 8) In the information processing device, acquiring information based on sensors attached to the user's feet; A process of determining an abnormality in the user's walking based on the acquired acceleration in the user's walking direction; A process of outputting information based on the determination result; A program that executes the following. (Appendix 9) In the information processing device, execute a process for determining an abnormality in the user's gait based on acceleration in the user's walking direction measured by a sensor attached to the user's foot; Trained model. (Appendix 10) An information processing system having a sensor attached to a user's foot, an information processing device, and an information processing terminal, The information processing device includes: an acquisition unit that acquires information based on the acceleration in the user's walking direction measured by the sensor; a determination unit that determines whether the user has an abnormality in walking based on the acceleration in the user's walking direction acquired by the acquisition unit; an output unit that outputs information based on a determination result by the determination unit to the information processing terminal; An information processing system having: (Appendix 11) The acquisition unit acquiring information based on a sensor attached to a position anywhere between the arch and the heel of the user's foot; 11. The information processing system of claim 10. (Appendix 12) Acquire a data set including a combination of information based on acceleration in the user's walking direction measured by a sensor attached to the user's foot and information indicating an abnormality in the user's gait; Based on the acquired information, a trained model is generated that determines abnormalities in the user's gait from information measured by sensors attached to the user's feet; Output the generated trained model. Generation method. (Appendix 13) an acquisition unit that acquires a data set including a combination of information based on acceleration in the user's walking direction measured by a sensor attached to the user's foot and information indicating abnormalities in the user's walking; a generation unit that generates a trained model that determines abnormalities in the user's gait from information measured by sensors attached to the user's feet, based on the acceleration in the user's walking direction acquired by the acquisition unit; and an output unit that outputs the trained model generated by the generation unit; An information processing device having the above. (Appendix 14) The acquisition unit Acquire a data set including a combination of information based on acceleration in the user's walking direction measured by a sensor attached to the user at a position between the arch and the heel of the foot, and information indicating abnormalities in the user's walking. 14. The information processing device according to claim 13. (Appendix 15) In the information processing device, obtaining a data set including a combination of information based on acceleration in the user's walking direction measured by a sensor attached to the user's foot and information indicative of an abnormality in the user's gait; A process of generating a trained model that determines abnormalities in the user's gait from information measured by sensors attached to the user's feet based on the acquired information; and A process of outputting the generated trained model; A program that executes the following. [Explanation of symbols]

[0077] 1. Information Processing Systems 10. Information processing equipment 11 Acquisition Department 12 Generation part 13 Output section 15 Acquisition Department 16 Judgment section 17 Output section 20 Measuring Equipment 21 Sensors 22 Control device 23 Communication equipment 30 User terminals 40 servers 50 Hospital terminals

Claims

1. an acquisition unit that acquires information based on sensors attached to the user's feet; a determination unit that determines whether the user has an abnormality in walking based on the acceleration in the user's walking direction acquired by the acquisition unit; an output unit that outputs information based on the determination result by the determination unit; and The determination unit a first acceleration in the walking direction when the toe of the foot leaves the ground; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; Information processing device.

2. The acquisition unit acquiring information based on a sensor attached to a position anywhere between the arch and the heel of the user's foot; The information processing device according to claim 1 .

3. The determination unit a third acceleration in the walking direction when the foot lands; determining an abnormality in the user's walking based on at least one of the second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and the walking speed of the user; 3. The information processing device according to claim 1.

4. The determination unit determining whether the user's walking is abnormal based on the magnitude of a single oscillation component of acceleration in the user's walking direction while the user takes one step; The information processing device according to claim 1 .

5. The determination unit The acceleration in the walking direction of the user; determining an abnormality in the user's walking based on at least one of an angle of a walking direction between the sole of the foot and the ground while the user takes a step and a magnitude of a five-oscillating component of the user's vertical acceleration while the user takes a step; The information processing device according to claim 1 .

6. An acquisition unit that acquires information based on a sensor attached to a user's foot; a determination unit that determines whether the user has an abnormality in walking based on the acceleration in the user's walking direction acquired by the acquisition unit; an output unit that outputs information based on the determination result by the determination unit; and The determination unit a third acceleration in the walking direction when the foot lands; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; Information processing device.

7. An information processing device, Acquire information based on sensors attached to the user's feet; determining an abnormality in the user's walking based on the acquired acceleration in the user's walking direction; Output information based on the judgment result, Execute the process, The process of determining abnormalities in the user's gait includes: a first acceleration in the walking direction when the toe of the foot leaves the ground; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; Information processing methods.

8. In the information processing device, acquiring information based on sensors attached to the user's feet; A process of determining an abnormality in the user's walking based on the acquired acceleration in the user's walking direction; A process of outputting information based on the determination result; Execute The process of determining abnormalities in the user's gait includes: a first acceleration in the walking direction when the toe of the foot leaves the ground; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; program.

9. An information processing system having a sensor attached to a user's foot, an information processing device, and an information processing terminal, The information processing device includes: an acquisition unit that acquires information based on the acceleration in the user's walking direction measured by the sensor; a determination unit that determines whether the user has an abnormality in walking based on the acceleration in the user's walking direction acquired by the acquisition unit; an output unit that outputs information based on a determination result by the determination unit to the information processing terminal; and The determination unit a first acceleration in the walking direction when the toe of the foot leaves the ground; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; Information processing system.

10. An information processing device, Acquire a data set including a combination of information based on acceleration in the user's walking direction measured by a sensor attached to the user's foot and information indicating an abnormality in the user's gait; Based on the acquired information, a trained model is generated that determines abnormalities in the user's gait from information measured by sensors attached to the user's feet; Output the generated trained model, The trained model is a first acceleration in the walking direction when the toe of the foot leaves the ground; determining an abnormality in the user's walking based on at least one of a second acceleration in the walking direction when the foot is swung in the walking direction due to contraction of the user's thigh muscles and a walking speed of the user; Generation method.

Images