Fall risk determination device, fall risk determination method, and program

Abstract

A fall risk determination device (40) is provided with: an acquisition unit (42a) that acquires a moving image showing a current person (1); an estimation unit (42b) that estimates, on the basis of the moving image, the skeleton of the pedestrian (1) reflected in the moving image; a classification unit (42c) that, on the basis of the skeleton, classifies a period during which the pedestrian (1) walks in the moving image into a plurality of periods; an extraction unit (42d) that extracts one or more first feature quantities relating to the walking of the pedestrian (1) on the basis of the skeleton for each of the plurality of periods; a determination unit (42e) that determines the risk of falling of the pedestrian (1) on the basis of the one or more first feature quantities; and an output unit (42f) that outputs the determination result of the determination unit (42e).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a fall risk determination device, a fall risk determination method, and a program. BACKGROUND

[0002] In the past, the number of fall accidents of workers is increasing, and countermeasures against falls are being strengthened. If the subject can be notified of the fall risk and physical ability of the subject, the subject's behavior change can be promoted by deepening the subject's self-understanding to prevent the subject's fall accidents.

[0003] A method of evaluating the fall risk of a subject who is walking is disclosed in Patent Literature 1.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2021-30051 SUMMARY

[0007] PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] The present disclosure provides a fall risk determination device and the like that can determine the fall risk of a pedestrian with high precision.

[0009] SOLUTION TO PROBLEM

[0010] A fall risk determination device according to one embodiment of the present disclosure includes: an acquisition unit that acquires a motion image in which a pedestrian is imaged; an estimation unit that estimates a skeleton of the pedestrian imaged in the motion image on the basis of the motion image; a classification unit that classifies a period during which the pedestrian walks in the motion image into a plurality of periods on the basis of the skeleton; an extraction unit that extracts one or more first feature amounts related to the walking of the pedestrian on the basis of the skeleton for each of the plurality of periods; a determination unit that determines a fall risk of the pedestrian on the basis of the one or more first feature amounts; and an output unit that outputs a determination result of the determination unit.

[0011] The fall risk determination method according to one embodiment of the present disclosure is a fall risk determination method executed by a computer, the fall risk determination method including: an acquisition step of acquiring a moving image of a person; an estimation step of estimating a skeleton of the person imaged in the moving image on the basis of the moving image; a classification step of classifying a period during which the person walks in the moving image into a plurality of periods on the basis of the skeleton; an extraction step of extracting one or more first feature amounts relating to walking of the person on the basis of the skeleton for each of the plurality of periods; a determination step of determining a fall risk of the person on the basis of the one or more first feature amounts; and an output step of outputting a determination result in the determination step.

[0012] In addition, the program according to one embodiment of the present disclosure is a program for causing the computer to execute the fall risk determination method.

[0013] In addition, the fall risk determination device according to one embodiment of the present disclosure includes: a walking action acquisition unit that acquires an action of a person walking; a classification unit that classifies a period during which the person walks into a plurality of periods; an extraction unit that extracts one or more first feature amounts relating to walking of the person on the basis of a movement of a skeleton of the action of the person walking for each of the plurality of periods; a determination unit that determines a fall risk of the person on the basis of the one or more first feature amounts; and an output unit that outputs a determination result of the determination unit.

[0014] Effects of Invention

[0015] The fall risk determination device according to one embodiment of the present disclosure and the like can determine a fall risk of a person with high precision. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a block diagram showing a functional structure of the display system according to Embodiment 1.

[0017] Figure 2 is a diagram for explaining a skeleton model of a person according to Embodiment 1.

[0018] Figure 3 is a diagram for explaining a skeleton model of a person according to Embodiment 1.

[0019] Figure 4 is a diagram showing specific examples of the plurality of periods according to Embodiment 1.

[0020] Figure 5 is a diagram showing a first example of an image displayed by the display unit according to Embodiment 1.

[0021] Figure 6FIG. 2 is a diagram showing a second example of an image displayed by the display section according to Embodiment 1.

[0022] Figure 7 FIG. 3 is a diagram for explaining a specific example of physical ability.

[0023] Figure 8 FIG. 4 is a diagram showing a specific example of a characteristic quantity for calculating physical ability.

[0024] Figure 9 FIG. 5 is a diagram showing a specific example of a characteristic quantity for calculating a walking style.

[0025] Figure 10 FIG. 6 is a timing chart showing a processing procedure of the display system according to Embodiment 1.

[0026] Figure 11 FIG. 7 is a flowchart showing a processing procedure of the fall risk determination device according to Embodiment 1.

[0027] Figure 12 FIG. 8 is a block diagram showing a functional structure of the fall risk determination device according to Embodiment 2.

[0028] Figure 13 FIG. 9 is a flowchart showing a processing procedure of the fall risk determination device according to Embodiment 2. DETAILED DESCRIPTION

[0029] Hereinafter, each embodiment will be specifically described with reference to the accompanying drawings. Furthermore, each embodiment to be described below indicates a general or specific example. The numerical values, shapes, materials, component elements, arrangement positions and connection modes of the component elements, steps, and order of the steps shown in each embodiment below are one example, and are not intended to limit the present disclosure. In addition, regarding the component elements in each embodiment below for which the component elements not recited in the independent claim are not recited, the component elements are described as arbitrary component elements.

[0030] Furthermore, each drawing is a schematic diagram, and the illustration is not necessarily strict. In addition, in each drawing, the same reference numerals are assigned to substantially the same structures, and repeated description is sometimes omitted or simplified.

[0031] In addition, in each embodiment, the vertical direction is referred to as the Z-axis direction or the up-down direction, one of the directions in a plane perpendicular to the vertical direction is referred to as the Y-axis direction or the front-rear direction, and the direction perpendicular to the Y-axis direction in the perpendicular plane is referred to as the X-axis direction, the left-right direction, or the lateral direction. In addition, in each embodiment, the positive side of the Z-axis direction is upward or up, and the negative side of the Z-axis direction is downward or down. In addition, in each embodiment, the positive side of the Y-axis direction is the front side or front, and the negative side of the Y-axis direction is the rear side or rear. In addition, in the present disclosure, the positive side of the X-axis direction is the right side or right, and the negative side of the X-axis direction is the left side or left.

[0032] (Embodiment 1)

[0033] [Structure]

[0034] Figure 1 is a block diagram showing a functional structure of the display system 10 according to Embodiment 1.

[0035] The display system 10 is a system that determines a fall risk based on a moving image containing a subject who walks (hereinafter, referred to as a pedestrian 1) as a subject, that is, a moving image that represents the pedestrian 1.

[0036] The display system 10 includes an information terminal 30 and a fall risk determination device 40.

[0037] A user, for example, captures the pedestrian 1 from the start of walking to the stop of walking by operating the information terminal 30. A moving image generated thereby is transmitted to the fall risk determination device 40. In the fall risk determination device 40, a fall risk of the pedestrian 1 is determined based on the received moving image, and the determination result is transmitted to the information terminal 30. The information terminal 30 displays the received determination result.

[0038] Here, the pedestrian 1 refers to a person whose fall risk is determined.

[0039] In addition, the user, for example, refers to a user of the information terminal 30 such as a physical therapist, an occupational therapist, a nurse, or a rehabilitation professional staff.

[0040] Further, the user of the information terminal 30 can be either any person or the pedestrian 1.

[0041] The information terminal 30 is a computer that instructs the pedestrian 1 to start walking, acquires a moving image (moving image data) containing the pedestrian 1 as a subject, which is generated by capturing the pedestrian 1 by a camera 20, and transmits the acquired moving image to the fall risk determination device 40. Specifically, the information terminal 30 generates a moving image by capturing the pedestrian 1, and transmits the generated moving image to the fall risk determination device 40.

[0042] The information terminal 30 is, for example, a portable computer device such as a smartphone or a tablet terminal used by the user. The information terminal 30 can also be a stationary computer device such as a personal computer.

[0043] The information terminal 30 includes the camera 20, a communication section 31, a control section 32, a storage section 33, a reception section 34, a display section 35, and an instruction section 36.

[0044] The video camera 20 is a video camera that generates a moving image containing the pedestrian 1 walking as a subject by photographing the pedestrian 1 walking. In the present embodiment, the video camera 20 generates a moving image that reflects the pedestrian 1 by photographing the pedestrian 1 walking.

[0045] The video camera 20 can be either a video camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a video camera using a CCD (Charge Coupled Device) image sensor.

[0046] Further, the video camera 20 can also be an external video camera attached to the information terminal 30. In this case, the information terminal 30 can not have the video camera 20 as long as it has a communication interface for connecting to the video camera 20 in a communicable manner.

[0047] For example, the video camera 20 can be one or a plurality of video cameras. The video camera 20 can photograph the pedestrian 1 from the front or from the side. In addition, the video camera 20 can have a function of motion capture.

[0048] The communication section 31 is a communication interface that communicates with the fall risk determination device 40. Specifically, in the communication section 31, the information terminal 30 communicates with the fall risk determination device 40 via the network 5 such as the Internet. The communication section 31 is realized by, for example, a wireless communication circuit for wireless communication with the fall risk determination device 40.

[0049] Further, the communication standard of the communication by the communication section 31 is not particularly limited.

[0050] In addition, the communication section 31 can be connected to the fall risk determination device 40 in a communicable manner by wireless communication or in a communicable manner by wired communication. For example, the communication section 31 is realized by a connector or the like connected to a communication line in the case of being connected to the fall risk determination device 40 in a communicable manner by wired communication.

[0051] The control section 32 is a processing section that performs various information processing in the information terminal 30. The control section 32 outputs, for example, a moving image generated by the video camera 20 to the fall risk determination device 40 via the communication section 31. For example, the control section 32 outputs, in the case of outputting a moving image, time information of each image in association with a plurality of images constituting the moving image.

[0052] Further, for example, the control section 32 acquires, from the fall risk determination device 40 via the communication section 31, a determination result of the fall risk of the pedestrian 1 determined by the fall risk determination device 40. Further, for example, the control section 32 acquires, from the fall risk determination device 40 via the communication section 31, an evaluation result (evaluation result information) of the fall risk determination device 40 (more specifically, the determination section 42e). For example, the control section 32 causes the display section 35 to display information indicating the acquired evaluation result. Further, the control section 32 performs various processes, for example, based on an operation input received by the reception section 34. The control section 32 is realized by, for example, a microcomputer. Alternatively, the control section 32 can be realized by a processor such as a CPU (Central Processing Unit). The functions of the control section 32 are realized, for example, by the microcomputer or the processor that configures the control section 32 executing a dedicated application program stored in the storage section 33.

[0053] The storage section 33 is a storage device that stores a dedicated application program or the like for execution by the control section 32. The storage section 33 is realized by, for example, a semiconductor memory or an HDD (Hard Disk Drive).

[0054] The reception section 34 is an input interface that receives an operation input by a user of the information terminal 30. For example, the reception section 34 receives an input operation of the user such as a shooting instruction of a moving image or a transmission instruction of a moving image to the fall risk determination device 40. The reception section 34 is realized by, for example, a touch panel display or the like. For example, in a case where the reception section 34 is realized by a touch panel display, the touch panel display functions as the display section 35 and the reception section 34.

[0055] Further, the reception section 34 is not limited to a touch panel display, and can be, for example, a keyboard, a pointing device such as a touch pen or a mouse, or a hardware button or the like. Further, the reception section 34 can be a microphone in a case where a sound-based input is received. Further, the reception section 34 can be a camera in a case where a gesture-based input is received. In this case, the reception section 34 can be realized by the camera 20 or by a camera different from the camera 20.

[0056] The display section 35 is a display device that displays an evaluation result of the fall risk determination device 40 or the like. The display section 35 can be realized by, for example, a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel, or by a sound device such as a speaker or a headphone, or by a display panel and a sound device.

[0057] The instruction section 36 is an instruction device that instructs the pedestrian 1 to walk. The instruction section 36 instructs the pedestrian 1 to walk, for example, by a video and a sound, and the like. That is, the instruction section 36 can instruct the user by a video or by a sound.

[0058] The instruction section 36 instructs, for example, "please walk" and "please stop walking" or the like by a video and / or a sound. The instruction section 36 can be realized by a display panel such as a liquid crystal panel or an organic EL panel, or can be realized by a sound device such as a speaker or a headphone, or can be realized by a display panel and a sound device.

[0059] In the present embodiment, the instruction section 36 instructs, for example, "please walk 10 m from a stopped state and then stop" or the like by a sound. The user captures, using the video camera 20, the pedestrian 1 from when the pedestrian 1 starts walking from a stopped state until the pedestrian 1 stops after walking 10 m. The information terminal 30 transmits the moving image captured like this to the fall risk determination device 40.

[0060] Further, the instruction section 36 and the display section 35 can be realized by the same display panel and / or sound device, or the like.

[0061] In addition, video information and / or sound information for the instruction section 36 to instruct the pedestrian 1 to perform walking can be stored in the storage section 33 in advance.

[0062] The fall risk determination device 40 is a computer that acquires a moving image transmitted from the information terminal 30 and edits at least one of the acquired moving images.

[0063] The fall risk determination device 40 includes a communication section 41, an information processing section 42, and a storage section 43.

[0064] The communication section 41 is a communication interface that communicates with the information terminal 30. Specifically, in the communication section 41, the fall risk determination device 40 communicates with the information terminal 30 via a network 5 such as the Internet. The communication section 41 is realized by, for example, a wireless communication circuit for wireless communication with the information terminal 30.

[0065] Further, the communication standard of the communication by the communication section 41 is not particularly limited.

[0066] In addition, the communication section 41 can be connected to the information terminal 30 in a manner capable of wireless communication, or can be connected to the information terminal 30 in a manner capable of wired communication. For example, the communication section 41 is realized by a connector or the like connected to a communication line or the like, in the case of being connected to the information terminal 30 in a manner capable of wired communication.

[0067] The information processing unit 42 is a processing unit that performs various information processing in the fall risk assessment device 40. The information processing unit 42 is implemented, for example, by a microcomputer. Alternatively, the information processing unit 42 may also be implemented by a processor such as a CPU. The functions of the information processing unit 42 are implemented, for example, by the microcomputer or processor constituting the information processing unit 42 executing computer programs stored in the storage unit 43.

[0068] The information processing unit 42 includes an acquisition unit 42a, an estimation unit 42b, a classification unit 42c, an extraction unit 42d, a determination unit 42e, and an output unit 42f.

[0069] The acquisition unit 42a is a processing unit that acquires a moving image of the pedestrian 1. Specifically, the acquisition unit 42a acquires the moving image output (transmitted) from the information terminal 30 via the communication unit 41. In this embodiment, the moving image includes the pedestrian 1 as the subject.

[0070] The estimation unit 42b is a processing unit that estimates the skeleton of pedestrian 1 reflected in the motion image based on the motion image acquired by the acquisition unit 42a. Specifically, the estimation unit 42b estimates (calculates) a skeleton model representing the skeleton of pedestrian 1 in the motion image based on the acquired motion image.

[0071] Figure 2 and Figure 3 These are diagrams illustrating the skeletal model of the pedestrian 1 involved in Embodiment 1. Specifically, Figure 2 The figure is a diagram showing a two-dimensional skeletal model of the skeleton of pedestrian 1 estimated by the estimation unit 42b superimposed on an image reflecting pedestrian 1. Figure 3 This is a diagram showing a three-dimensional skeletal model of the pedestrian 1 estimated by the estimation unit 42b. Specifically, Figure 3 This is a diagram showing a three-dimensional skeletal model of a pedestrian 1 walking sideways in the Y-axis direction at a certain moment.

[0072] A skeletal model is a model generated by connecting multiple skeletal points, such as joints, of pedestrian 1 in a motion image using links (lines). Specifically, a skeletal model refers to the coordinate data of multiple skeletal points, etc. For example, the estimation unit 42b estimates the positions (more specifically, coordinates) of multiple skeletal points representing the estimated joint positions, etc., by estimating the positions of pedestrian 1's joints, etc. More specifically, the estimation unit 42b estimates the positions of multiple predetermined skeletal points of pedestrian 1, including skeletal points of the neck, elbow, and wrist, in each image included in the motion image by performing image parsing, etc. Furthermore, the estimation unit 42b connects the estimated multiple skeletal points to each other using lines, for example, according to prescribed conditions. Thus, the estimation unit 42b estimates the skeletal model of pedestrian 1.

[0073] Furthermore, in the estimation of the skeletal model, estimation can be performed using any method, such as existing pose and skeletal estimation algorithms.

[0074] Furthermore, the estimation unit 42b can estimate both the two-dimensional and three-dimensional skeletal models of pedestrian 1. That is, the estimation unit 42b can estimate both the two-dimensional and three-dimensional coordinates of the skeletal points of pedestrian 1. For example, based on the motion image acquired by the acquisition unit 42a, the estimation unit 42b estimates the two-dimensional skeletal model of pedestrian 1 (that is, the coordinates of each skeletal point in a two-dimensional orthogonal coordinate system), and based on the estimated two-dimensional skeletal model, uses a learned machine learning model (i.e., a fully learned model) to estimate the three-dimensional skeletal model of pedestrian 1 (that is, the coordinates of each skeletal point in a three-dimensional orthogonal coordinate system).

[0075] The learned model is a pre-built recognizer that uses a two-dimensional skeletal model with known 3D coordinate data of each skeletal point (more specifically, the skeletal model) as learning data and the 3D coordinate data as training data for machine learning. The learned model takes the two-dimensional skeletal model as input and outputs the corresponding 3D coordinate data, i.e., the 3D skeletal model. The learned model is, for example, pre-stored in storage unit 43.

[0076] In this way, the estimation unit 42b can also estimate the three-dimensional skeletal model of pedestrian 1 in the motion image acquired by the acquisition unit 42a.

[0077] In addition, the learned model is a machine learning model that uses neural networks (such as convolutional neural networks (CNN)) such as deep learning, but it can also be other machine learning models.

[0078] Alternatively, the estimation unit 42b may estimate only the skeletal points in the skeletal model used in the feature extraction of the extraction unit 42d described later. That is, the estimation unit 42b may estimate only a portion of the skeletal model of pedestrian 1.

[0079] The classification unit 42c is a processing unit that classifies the period during which pedestrian 1 walks in a motion image into multiple periods based on the skeleton of pedestrian 1 estimated by the estimation unit 42b. Specifically, the classification unit 42c classifies the period from when pedestrian 1 starts walking until he stops in the motion image into multiple periods based on the skeletal model of pedestrian 1.

[0080] Figure 4This is a diagram illustrating specific examples of the various periods involved in Implementation 1.

[0081] Multiple periods include, for example, the acceleration period when pedestrian 1 begins to walk and accelerates, the constant speed period, and the deceleration period. The constant speed period is the period during which pedestrian 1 walks steadily and is the next period after the acceleration period. The deceleration period is the period during which pedestrian 1 decelerates until it stops and is the next period after the constant speed period. In other words, the classification unit 42c classifies the periods of pedestrian 1 walking in the motion image into three periods: the acceleration period, the constant speed period, and the deceleration period.

[0082] The acceleration phase is the period during which pedestrian 1 begins to walk and accelerates. Specifically, the acceleration phase is the period during which a stationary pedestrian 1 begins to walk and accelerates until the walking speed roughly reaches a certain fixed speed.

[0083] The constant speed period is the period during which pedestrian 1 walks steadily, and it is the period following the acceleration period. Specifically, the constant speed period is the period during which pedestrian 1's walking speed roughly reaches a certain fixed speed, that is, the period during which pedestrian 1 walks steadily, and it is the period before pedestrian 1 begins to decelerate in order to stop.

[0084] The deceleration period is the time during which pedestrian 1 slows down until it comes to a stop, and it is the period following the constant speed period. Specifically, the deceleration period is the time from when pedestrian 1, who is walking at approximately a fixed speed, begins to slow down until it comes to a stop.

[0085] The classification unit 42c, for example, calculates the velocity (movement speed) of the bone point representing the heel in the skeletal model estimated by the estimation unit 42b as the velocity (movement speed) of pedestrian 1 based on the time change of the coordinates of the bone point representing the heel. For example, the classification unit 42c calculates the velocity of the bone point representing the heel based on the distance (movement distance) and time (one cycle time) along the Y-axis direction (that is, the direction of pedestrian 1's movement) from the point of contact with the ground to the point of contact with the ground again on the right or left heel. For example, the classification unit 42c calculates the velocity of pedestrian 1 at each moment by calculating (movement time) / (one cycle time). The classification unit 42c, for example, classifies the period of pedestrian 1's walking in the motion image into three periods: acceleration period, constant speed period, and deceleration period based on the time change of the velocity of pedestrian 1 calculated in this way.

[0086] For example, classification unit 42c may define the period from when pedestrian 1 begins walking until their walking speed reaches or exceeds a first speed as an acceleration period. Alternatively, classification unit 42c may define the period from when the walking speed reaches or exceeds the first speed until it falls below a second speed as a constant speed period. Furthermore, classification unit 42c may define the period from when the walking speed falls below the second speed until pedestrian 1 stops (i.e., when the walking speed reaches 0) as a deceleration period. Classification unit 42c may also define the period after reaching or falling below the second speed as a deceleration period.

[0087] Furthermore, the first and second speeds can be arbitrarily determined beforehand without any particular limitation. For example, the first speed can be a speed that is faster than the second speed. The first and second speeds can also be the same, for example.

[0088] Furthermore, the skeletal points used in the classification of the period in the classification section 42c can be arbitrarily determined as skeletal points of the waist, etc., without any particular limitation.

[0089] Furthermore, the classification unit 42c may calculate the acceleration of pedestrian 1 (e.g., acceleration in the Y-axis direction) based on predetermined skeletal points such as the heel bone point, and separate periods based on the calculated acceleration. For example, during the acceleration period, the acceleration of pedestrian 1 reaches or exceeds a first acceleration. Additionally, for example, during the uniform speed period, the acceleration of pedestrian 1 is less than the first acceleration but reaches or exceeds a second acceleration. Additionally, for example, during the deceleration period, the acceleration of pedestrian 1 is less than the second acceleration. For example, the classification unit 42c may determine a transition from the acceleration period to the uniform speed period when pedestrian 1 reaches a speed less than the first acceleration after the start of walking, and determine a transition from the uniform speed period to the deceleration period when pedestrian 1 reaches a speed less than the second acceleration. The first acceleration is, for example, a positive number, and the second acceleration is, for example, a negative number. The first and second accelerations can be arbitrarily determined without particular limitation.

[0090] These threshold information, such as the first velocity and the first acceleration, are, for example, pre-stored in the storage unit 43.

[0091] In addition, the classification unit 42c can also determine the start and stop of pedestrian 1's walking based on pedestrian 1's posture (that is, the posture of the skeletal model).

[0092] Alternatively, the classification unit 42c may, for example, classify the period of pedestrian 1 walking in the motion image into two periods, such as the first half and the second half of pedestrian 1's walking. Or, the classification unit 42c may, for example, classify the period of pedestrian 1 walking in the motion image into four or more periods, such as the acceleration period, the first half of the constant speed period, the second half of the constant speed period, and the deceleration period.

[0093] Extraction unit 42d is a processing unit that extracts one or more feature quantities (also called first feature quantities) related to the walking of pedestrian 1 based on the skeleton of pedestrian 1 in the motion image estimated by estimation unit 42b for each of the multiple periods classified by classification unit 42c. In this embodiment, extraction unit 42d extracts one or more feature quantities for the acceleration period based on the skeletal model of pedestrian 1 in the acceleration period of the motion image. Furthermore, in this embodiment, extraction unit 42d extracts one or more feature quantities for the uniform speed period based on the skeletal model of pedestrian 1 in the uniform speed period of the motion image. Furthermore, in this embodiment, extraction unit 42d extracts one or more feature quantities for the deceleration period based on the skeletal model of pedestrian 1 in the deceleration period of the motion image.

[0094] Characteristic quantities are numerical values ​​that quantitatively represent the characteristics of pedestrian 1's walking. Examples of characteristic quantities include trunk movement, upper limb movement, lower limb movement, and gait.

[0095] Examples of trunk movements include the up-and-down movement of the chest, the amplitude of chest oscillation, the angle of the head's forward and backward swaying, the left-and-right tilting of the trunk, and the amplitude of the left-and-right swaying of the trunk.

[0096] Examples of upper limb movements include the forward and backward swinging range of the wrist, the vertical tilting angle of the shoulder, and the left and right swinging angle of the elbow.

[0097] Examples of lower limb movements include the vertical swing range of the knee, the knee opening range during the leg swing phase, the ankle opening range during the leg swing phase, the range of motion of the hip joint, the range of motion of the knee joint, and the range of motion of the foot joint.

[0098] As a gait, it can be used to represent step size, stride width (stride length), step width, walking angle, heel height, and toe height.

[0099] In addition, as characteristic quantities, examples include steps (total steps), walking time, walking distance, walking speed, walking rate (steps per second), cadence (steps per minute), walking ratio (average stride length / cadence), average stride length / average stride length, and average stride length / average stride length, etc.

[0100] In addition, the feature quantity can also be a feature quantity other than those mentioned above.

[0101] For example, the first feature quantity extracted by the extraction unit 42d includes any one of the following features of pedestrian 1 during walking: features related to joint angles, features related to joint movement speed, features related to joint movement acceleration, features related to walking time, and features related to distance based on joint movement distance. In other words, in order to evaluate the fall risk of pedestrian 1, the extraction unit 42d extracts any one of the following features of pedestrian 1 during walking: features related to joint angles, features related to joint movement speed, features related to joint movement acceleration, features related to walking time, and features related to distance based on joint movement distance, as features for at least one of multiple periods.

[0102] Characteristic quantities related to the angle of a joint include, for example, the range of motion (degree of motion) of a specific joint such as the hip, ankle, or knee joint. Furthermore, specific joints can be arbitrarily determined without particular limitation.

[0103] In addition, the joints used to extract features related to movement distance, movement speed, and movement acceleration can be arbitrary and are not particularly limited.

[0104] Furthermore, the joints (skeletal points) used when calculating the characteristics related to walking time can be arbitrarily determined without particular limitation.

[0105] Additionally, for example, the extraction unit 42d may extract at least one of the following features of pedestrian 1 during walking: average toe speed, walking speed, leg support time, trunk sway amplitude, maximum toe speed, stride length, waist height variation, hip joint range of motion, heel contact angle, chest vertical movement, ankle joint range of motion, knee height variation, and knee joint angle swing amplitude. In other words, the extraction unit 42d extracts at least one of the following features of pedestrian 1 during walking: average toe speed, walking speed, leg support time, trunk sway amplitude, maximum toe speed, stride length, waist height variation, hip joint range of motion, heel contact angle, chest vertical movement, ankle joint range of motion, knee height variation, and knee joint angle swing amplitude, as features of at least any one of multiple periods.

[0106] The average speed of the toes refers to the average speed of movement of pedestrian 1 in the direction of travel.

[0107] Walking speed refers to the speed at which pedestrian 1 moves in the direction of travel.

[0108] The time of double-leg support refers to the time that pedestrian 1's two legs are in contact with the ground.

[0109] The amplitude of the torso's forward and backward sway refers to the amplitude of the torso (e.g., upper body) of pedestrian 1 in the forward and backward direction. The amplitude of the torso's forward and backward sway is, for example, the angle formed by a line segment passing through the skeletal point of the waist and parallel to the vertical direction when pedestrian 1 is viewed from the side (X-axis direction in this embodiment) and the line segment connecting the skeletal point of the waist and the skeletal points of both shoulders.

[0110] The maximum speed of the toes refers to the maximum speed of movement of pedestrian 1 in the direction of travel.

[0111] Stride length refers to the distance between the front and back feet in the direction of travel during a pedestrian's walk.

[0112] The change in waist height (up and down movement of the waist) refers to the amplitude of the waist in the up and down direction (Z-axis direction in this embodiment) during a walking cycle.

[0113] The anteroposterior range of motion of the hip joint refers to the range of motion of the hip joint in the anteroposterior direction during a walking cycle. For example, the anteroposterior range of motion of the hip joint is the angle formed by the vertical line (the normal to the ground where pedestrian 1 is walking) – waist – knee during a walking cycle. Specifically, the anteroposterior range of motion of the hip joint is the angle formed by the vertical line through the waist and the line segment connecting the waist and knee during a walking cycle when pedestrian 1 is viewed laterally.

[0114] The heel-to-ground angle refers to the maximum angle between the sole of the foot and the ground when the heel touches the ground. For example, the heel-to-ground angle (ground contact angle) is the angle between the toes and the ground with the heel as the base point when the heel contacts the ground. In other words, the heel-to-ground angle is the angle between the line segment connecting the heel (the bony point of the heel) and the toes (the bony point of the toes) and the ground when the heel contacts the ground, as seen from a lateral perspective.

[0115] The up-and-down movement of the chest refers to the amplitude of the chest's vertical movement during one walking cycle.

[0116] The range of motion of the ankle joint is the area of ​​motion of the ankle joint during a walking cycle. The range of motion of the ankle joint angle is, for example, the angle formed by the knee-ankle-toe during a walking cycle. Specifically, the range of motion of the ankle joint angle is the angle formed by the line segment connecting the knee and ankle and the line segment connecting the toe and ankle during a walking cycle when viewed from the side of pedestrian 1.

[0117] The change in knee height (the vertical movement of the knee) refers to the amplitude of the knee's vertical oscillation during a walking cycle.

[0118] The range of motion of the knee joint refers to the range of motion (angle) of the knee joint during flexion and extension in one walking cycle.

[0119] The inventors of this application conducted in-depth research through correlation analysis of people's fall history records and found that, in particular, the average speed of the toes, walking speed, leg support time, trunk sway amplitude, maximum speed of the toes, stride length, waist height variation (up and down movement of the waist), hip joint sway amplitude, heel contact angle (ground contact angle), chest up and down movement, ankle joint range of motion, knee height variation, and knee joint angle swing amplitude are highly correlated with fall history records (fall risk).

[0120] For example, the slower the walking speed, the greater the risk of falling. In other words, the more likely a person has a history of falls—that is, the more times they have fallen within a specified period since the time of the fall risk assessment—the slower their walking speed. Additionally, for example, the less vertical movement of the knees, the higher the risk of falling. Also, for example, the smaller the ground contact angle, the greater the risk of falling, the greater the risk of falling. Furthermore, for example, the longer the legs bear weight, the worse the balance, and the greater the risk of falling. Additionally, for example, the smaller the forward and backward sway of the torso, the greater the risk of falling. Furthermore, for example, the smaller the vertical movement of the waist, the greater the risk of falling.

[0121] Therefore, by using these characteristic quantities in the assessment of fall risk, it is possible to more accurately determine fall risk.

[0122] Furthermore, the inventors of this application, through in-depth research, discovered that by using walking speed (the first two steps), hip joint mobility (the first step), and knee joint mobility (the first step) as characteristic quantities during the acceleration phase, walking speed (central value), leg support time (central value), and body sway amplitude (central value) as characteristic quantities during the constant speed phase, and one walking cycle time (the last step) and toe-off angle (the last two steps) as characteristic quantities during the deceleration phase, the risk of falling can be determined more accurately. Therefore, in this embodiment, the extraction unit 42d extracts walking speed (the first two steps), hip joint mobility (the first step), and knee joint mobility (the first step) as characteristic quantities during the acceleration phase. Additionally, in this embodiment, the extraction unit 42d extracts walking speed (central value), leg support time (central value), and body sway amplitude (central value) as characteristic quantities during the constant speed phase. Furthermore, in this embodiment, the extraction unit 42d extracts a walking cycle time (last step) and toe-off angle (last two steps) as feature quantities during the deceleration period.

[0123] In addition, a walking cycle time (last step) refers to the time involved in one step when pedestrian 1 stops.

[0124] As described above, for example, the extraction unit 42d extracts more than one feature quantity in a manner that is different from each other in multiple periods. Of course, the extraction unit 42d may also extract the same feature quantity in each of the multiple periods. For example, if the extraction unit 42d extracts the average speed of the toe as a feature quantity in each of the multiple periods, it may extract the average speed of the toe as a feature quantity for each period separately.

[0125] In addition, the extraction unit 42d can also extract features related to the walking of pedestrian 1 during the walking period based on the estimated skeleton of pedestrian 1.

[0126] The determination unit 42e is a processing unit that determines the fall risk of pedestrian 1 based on one or more feature values ​​from each of the multiple periods extracted by the extraction unit 42d. For example, the determination unit 42e determines the fall risk corresponding to the acceleration period based on one or more feature values ​​extracted during the acceleration period. Additionally, the determination unit 42e determines the fall risk corresponding to the constant speed period based on one or more feature values ​​extracted during the constant speed period. Furthermore, the determination unit 42e determines the fall risk corresponding to the deceleration period based on one or more feature values ​​extracted during the deceleration period. Finally, the determination unit 42e determines (evaluates) the fall risk of pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period. For example, the determination unit 42e ultimately evaluates the fall risk of pedestrian 1 by adding the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period (overall reasoning).

[0127] For example, the determination unit 42e uses a machine learning model, which is learned from features related to a person’s walking and a history of falls indicating whether the person has fallen within a specified period, to determine the risk of a fall for pedestrian 1.

[0128] The machine learning model is a pre-built recognizer that extracts one or more features from motion images of a walking person as learning data and uses the person's fall history (whether there have been falls within a specified period since the motion image was taken) as training data. The machine learning model takes one or more features as input and outputs the person's fall risk. The machine learning model is, for example, pre-stored in storage unit 43.

[0129] In addition, machine learning models can be machine learning models that use neural networks (such as convolutional neural networks (CNNs)) such as deep learning, but they can also be other machine learning models.

[0130] In addition, the learned model can also be a statistical model or method that uses the history of falls as the target variable and the feature quantities as explanatory variables (multiple regression model or logistic regression model).

[0131] Furthermore, the specified period can be arbitrarily determined without any particular limitation. For example, the specified period could be set to the past year.

[0132] Alternatively, the output of the machine learning model could be, for example, whether there is a risk of falling (i.e., whether it is 0 or 1). Or, the output could be, for example, the degree of fall risk (e.g., any value between 0 and 1). In this case, for example, the higher the fall risk, the more likely the machine learning model will output a value closer to 1, such as 0.8; the lower the fall risk, the more likely the machine learning model will output a value closer to 0, such as 0.2. The decision unit 42e, for example, inputs one or more feature values ​​into the machine learning model, and as a result, uses the output of the machine learning model as the decision result. The decision unit 42e can also evaluate the fall risk based on the value output from the machine learning model. For example, if the value output from the machine learning model is 0 or higher and less than 0.5, the decision unit 42e may determine the fall risk as low; if it is 0.5 or higher and less than 0.8, the decision unit 42e may determine the fall risk as moderate; and if it is 0.8 or higher, the decision unit 42e may determine the fall risk as high.

[0133] The method for determining these thresholds and the risk of falling can be arbitrarily decided without particular limitation. For example, if the value output from the machine learning model is less than 0.5, the determination unit 42e may determine the risk of falling as low, and if it is greater than 0.5, the determination unit 42e may determine the risk of falling as high.

[0134] For example, such machine learning models are set up to correspond to each of the multiple periods. For example, the determination unit 42e uses a first machine learning model that is input with one or more features extracted during the acceleration period, a second machine learning model that is input with one or more features extracted during the constant speed period, and a third machine learning model that is input with one or more features extracted during the deceleration period to determine the fall risk corresponding to each period.

[0135] The inventors of this application have conducted in-depth research and discovered that by using walking speed (the first two steps), hip joint range of motion (the first step), and knee joint range of motion (the first step) as inputs (features), the AUC (Area Under Curve) of the first machine learning model becomes 0.78.

[0136] Furthermore, AUC refers to an indicator of the degree to which a binary value can be accurately distinguished. In this embodiment, for example, the first machine learning model is an indicator of the degree to which it correctly determines a person's risk of falling.

[0137] Furthermore, the inventors of this application have conducted in-depth research and discovered that by using walking speed (central value), leg support time (central value), and body sway amplitude (central value) as inputs (features), the AUC of the second machine learning model becomes 0.80.

[0138] Furthermore, the inventors of this application have conducted in-depth research and discovered that by using a walking cycle time (the last step) and the toe-off angle (the last two steps) as inputs (features), the AUC of the third machine learning model becomes 0.58.

[0139] As described above, the inventors of this application have discovered that by classifying the period of walking into multiple periods and using appropriate feature quantities for each period to determine the risk of falling, it is possible to determine the risk of falling with high accuracy.

[0140] Furthermore, for example, the decision unit 42e may use majority voting or weighting the outputs of each machine learning model when adding the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period. For example, the fall risk may be determined using only the output of the machine learning model with an AUC of a predetermined value or higher. That is, in determining the fall risk, it is not necessary to use all the periods classified by the classification unit 42c; only one period or two or more periods may be used. The predetermined value can be arbitrarily determined without particular limitation. Alternatively, for example, the weighting may be performed by multiplying the output of the machine learning model by a coefficient with a larger AUC value.

[0141] Alternatively, a machine learning model can be a regression model that uses a person's fall history (their fall history over the past year) as the target variable and includes the feature value in the explanatory variables.

[0142] Alternatively, the decision unit 42e can also use methods such as decision trees or random forests to determine the risk of falling.

[0143] The output unit 42f is a processing unit that outputs the determination result of the determination unit 42e. For example, the output unit 42f sends information indicating the determination result to the information terminal 30 via the communication unit 41. As a result, the determination result is output (e.g., displayed) to the information terminal 30.

[0144] In addition, the information processing unit 42 may also include a computing unit 42g.

[0145] The calculation unit 42g is a processing unit that calculates auxiliary indicators used to interpret the fall risk of pedestrian 1.

[0146] Ancillary indicators are, for example, indicators (information) used to determine factors and / or recommendations for quantified fall risk. Specifically, ancillary indicators are information representing at least one of pedestrian 1's physical capabilities and walking style.

[0147] For example, the calculation unit 42g calculates the angle (joint angle) between two links connected to a specified skeletal point of pedestrian 1 as a feature quantity, based on the skeletal model estimated by the estimation unit 42b. Alternatively, the calculation unit 42g calculates the distance between the specified skeletal point and the distal end, as well as the range of positional variation of the specified skeletal point, as feature quantities. For example, the calculation unit 42g calculates pedestrian 1's physical ability as an auxiliary indicator based on whether the calculated values ​​are above or below a specified threshold or within a specified range.

[0148] Accordingly, for example, a training program can be provided for pedestrians 1 who are at risk of falling (e.g., the determined risk of falling is higher than a predetermined threshold), based on physical abilities such as muscle strength, to maintain or improve their physical capabilities. The predetermined bone points, predetermined thresholds, and predetermined ranges can also be arbitrarily determined. This information can also be pre-stored in the storage unit 43.

[0149] In addition, information related to auxiliary indicators of pedestrian 1 can be either pre-stored in storage unit 43, or received by user through receiving unit 34 and obtained from information terminal 30 by obtaining unit 42a.

[0150] Alternatively, the calculation unit 42g can also generate a training plan based on the calculation results. In this case, for example, the calculation unit 42g can also generate a training plan based not only on the calculation results but also on the decision results of the decision unit 42e.

[0151] For example, the extraction unit 42d extracts one or more features (second features) related to the walking of pedestrian 1 based on the skeleton of pedestrian 1 estimated by the estimation unit 42b for each of the multiple periods. In this case, for example, the calculation unit 42g calculates the physical ability of pedestrian 1 as an auxiliary indicator based on the one or more features (second features) extracted by the extraction unit 42d. For example, the calculation (estimation) of physical ability is performed for each walking stage (start of walking (acceleration period), steady walking (uniform speed period), end of walking (deceleration period)), and features associated with physical ability (e.g., five items) are designed to construct an estimation model.

[0152] The feature quantity extracted here for calculating physical ability includes any one of the following features of pedestrian 1 during walking: features related to joint angles, features related to joint movement speed, features related to joint movement acceleration, features related to walking movement time, and features related to distance based on joint movement distance. In other words, in order to calculate the physical ability of pedestrian 1, extraction unit 42d extracts any one of the following features of pedestrian 1 during walking: features related to joint angles, features related to joint movement speed, features related to joint movement acceleration, features related to walking movement time, and features related to distance based on joint movement distance, as features for at least one of multiple periods.

[0153] Furthermore, the feature quantities extracted to evaluate pedestrian 1's fall risk can be the same as or different from the feature quantities extracted to calculate pedestrian 1's physical abilities.

[0154] Additionally, for example, during the walking period of pedestrian 1, the extraction unit 42d extracts one or more feature quantities (third feature quantities) related to the walking of pedestrian 1 based on the skeleton of pedestrian 1 estimated by the estimation unit 42b. In this case, for example, the calculation unit 42g calculates the walking pattern of pedestrian 1 as an auxiliary indicator based on one or more feature quantities (third feature quantities). For example, the calculation (estimation) of the walking pattern is performed using feature quantities extracted from walking in all stages of walking.

[0155] The feature quantities extracted here for calculating the walking mode include at least one of the following during pedestrian 1's walking: walking speed, body sway, hip joint range of motion, knee joint range of motion, leg support time, standing time, heel height, heel contact angle, heel lift-off angle, waist movement, chest movement, stride width, head sway, hand sway, elbow sway, stride length, and step length. In other words, in order to calculate the walking mode of pedestrian 1, extraction unit 42d extracts at least one of the following during pedestrian 1's walking: walking speed, body sway, hip joint range of motion, knee joint range of motion, leg support time, standing time, heel height, heel contact angle, heel lift-off angle, waist movement, chest movement, stride width, head sway, hand sway, elbow sway, stride length, and step length, as feature quantities during pedestrian 1's walking.

[0156] Furthermore, the walking period of pedestrian 1 refers to the time from the start of walking to the end of walking. In this embodiment, the walking period of pedestrian 1 includes an acceleration period, a constant speed period, and a deceleration period.

[0157] The forward and backward swaying of the body refers to the angle (amplitude) of the body's swaying in the forward and backward direction during a walking cycle.

[0158] The anteroposterior range of motion of the knee joint refers to the range of motion of the knee joint during a walking cycle. The anteroposterior range of motion of the knee angle is, for example, the angle formed by the waist-knee-ankle during a walking cycle. Specifically, the range of motion of the knee angle is the angle formed by the line segment connecting the waist and knee and the line segment connecting the ankle and knee during a walking cycle, viewed laterally from the perspective of pedestrian 1.

[0159] Standing time refers to the time that the foot remains in contact with the ground during one walking cycle.

[0160] Heel height refers to the maximum distance between the heel and the ground during the leg swing phase.

[0161] The angle of heel off the ground refers to the angle between the sole of the foot and the ground during the leg swing.

[0162] Stride width refers to the lateral distance (in this embodiment, the X-axis direction) between the heels of both feet when one foot touches the ground and the other foot touches the ground.

[0163] The forward and backward swaying of the head refers to the amplitude (angle) of the head's forward and backward movement during one walking cycle.

[0164] The back-and-forth swing of the hand refers to the amplitude (angle) of the hand's movement in the forward and backward direction during one walking cycle.

[0165] The forward and backward swing of the elbow refers to the amplitude (angle) of the elbow's forward and backward movement during one walking cycle.

[0166] Stride length is the distance from the point where the foot touches the ground to the point where the same foot touches the ground again after one walking cycle.

[0167] For example, the computing unit 42g uses a machine learning model, which is learned based on features related to a person's walking and the person's auxiliary indicators (specifically, physical function and walking style), to determine the auxiliary indicators of pedestrian 1.

[0168] The machine learning model used in the calculation of the auxiliary indicator is a pre-built recognizer that extracts one or more features from a motion image of a walking person as learning data and uses the person's auxiliary indicator as training data. The machine learning model takes one or more features as input and outputs the auxiliary indicator. The machine learning model is, for example, pre-stored in storage unit 43.

[0169] In addition, machine learning models can be machine learning models that use neural networks (such as convolutional neural networks (CNNs)) such as deep learning, but they can also be other machine learning models.

[0170] Furthermore, the type of machine learning model used by the determination unit 42e (e.g., the method used by the machine learning model) can be the same as or different from the type of machine learning model used by the calculation unit 42g.

[0171] For example, such a machine learning model is set up corresponding to each of the multiple periods in the calculation of pedestrian 1's physical ability. For example, the calculation unit 42g uses a first machine learning model that is input with one or more features extracted during the acceleration period, a second machine learning model that is input with one or more features extracted during the constant speed period, and a third machine learning model that is input with one or more features extracted during the deceleration period to determine the physical ability corresponding to each period. For example, the calculation unit 42g finally evaluates pedestrian 1's physical ability by adding the physical ability corresponding to the acceleration period, the physical ability corresponding to the constant speed period, and the physical ability corresponding to the deceleration period.

[0172] Additionally, for example, such a machine learning model is set to correspond to the duration of pedestrian 1's walking (e.g., the entire walking period) in the calculation of pedestrian 1's walking pattern.

[0173] The output of the machine learning model can, for example, evaluate whether each item, such as muscle strength and foot movement, included in the auxiliary indicators described later, is high or low; in other words, whether it is adequate compared to the average person (e.g., 1 or 0). Alternatively, the output of the machine learning model can be an output at multiple stages (e.g., any value between 0 and 1). In this case, for example, the higher the ability (or the more appropriately one can perform), the more likely the machine learning model is to output a value closer to 1, such as 0.8; the lower the ability (or the less appropriately one can perform), the more likely the machine learning model is to output a value closer to 0, such as 0.2. The computation unit 42g, for example, inputs one or more feature quantities into the machine learning model, and as a result, uses the output of the machine learning model as the computation result. The computation unit 42g can also evaluate the auxiliary indicators based on the values ​​output from the machine learning model. For example, when the auxiliary indicator is muscle strength in body function, if the value output from the machine learning model is 0 or higher and less than 0.5, the calculation unit 42g will determine the muscle strength as low; if it is 0.5 or higher and less than 0.8, the calculation unit 42g will determine the muscle strength as moderate; and if it is 0.8 or higher, the calculation unit 42g will determine the muscle strength as high.

[0174] The determination of these thresholds and auxiliary indicators can be arbitrary and not particularly limited. For example, if the value output from the machine learning model is less than 0.5, the calculation unit 42g may determine the muscle strength as low, and if it is greater than 0.5, the calculation unit 42g may determine the muscle strength as high.

[0175] In addition, for example, when the auxiliary indicators include multiple items, the calculation unit 42g can also calculate a comprehensive evaluation of physical ability and walking style based on the sum of multiple items.

[0176] Furthermore, for example, the calculation unit 42g can use majority voting or weight the outputs of each machine learning model when adding the physical abilities corresponding to the acceleration period, the uniform speed period, and the deceleration period. For example, physical abilities can be determined using only the outputs of machine learning models with an AUC of a predetermined value or higher. That is, in determining physical abilities, it is not necessary to use all the periods classified by the classification unit 42c; only one period or two or more periods need to be used. The predetermined value can also be arbitrarily determined without particular limitation. In addition, for example, the weighting can be performed by multiplying the output of the machine learning model by a coefficient with a larger AUC value.

[0177] Alternatively, a machine learning model can also be a regression model that uses physical ability or walking style as the target variable and includes the feature quantity in the explanatory variables.

[0178] In addition, the computing unit 42g can also use methods such as decision trees or random forests to determine physical abilities or walking patterns.

[0179] Furthermore, the feature quantities used in the assessment of fall risk can be the same as or different from those used to calculate auxiliary indicators. Similarly, the feature quantities used in the assessment of physical ability can be the same as or different from those used to calculate gait patterns. Additionally, for example, when both physical ability and gait patterns include multiple items, the more than one feature quantity used in the assessment of each item can be the same or different.

[0180] In addition, the output unit 42f may also output the calculation results of the calculation unit 42g (specifically, auxiliary information representing the physical function and walking style of pedestrian 1, etc.) and / or training plans, along with the result of the judgment of the fall risk of pedestrian 1.

[0181] Figure 5 This is a diagram illustrating a specific example of the determination result displayed by the display unit 35 according to Embodiment 1. Specifically, Figure 5 This is a specific example of an image showing the determination result of the determination unit 42e, which is output by the output unit 42f and displayed on the display unit 35.

[0182] The control unit 32, for example, causes the display unit 35 to display the acquired determination result (specifically, an image representing the determination result).

[0183] For example, in Figure 5 The image shown contains information indicating the risk of pedestrian 1 falling, as well as information indicating the physical functions associated with pedestrian 1's fall. Figure 5 In the example shown, the image representing the assessment result of pedestrian 1's fall risk includes information such as "Fall risk is moderate." Alternatively, the image representing the assessment result may also include information such as "Fall risk: ★★☆" where the level of fall risk is indicated by filling in ☆, etc., to represent the stage (level) of fall risk. Furthermore, as information representing the physical functions associated with pedestrian 1's fall, the image includes assessment results of physical functions such as "left-right balance" and "muscle strength," as well as information encouraging pedestrian 1 to train, such as "Muscle strength is slightly weakened, therefore training is recommended." The image may also include an image representing a training plan.

[0184] Figure 6 This is a diagram showing a second example of an image displayed by the display unit 35 according to Embodiment 1.

[0185] exist Figure 6The image shown contains, for example, display information 100, 101, and 102.

[0186] Display information 100 is an image representing the fall risk of pedestrian 1, representing the judgment result of judgment unit 42e.

[0187] Display information 101 is an image representing the calculation results of pedestrian 1's walking style, and it represents the calculation results of pedestrian 1's walking style in the calculation unit 42g. Display information 101 is an example of auxiliary information. For example, display information 101 includes the calculation results of four items (foot movement, torso, arm swing, and pelvis) as items representing pedestrian 1's walking style. Display information 101 includes, for example, the calculation results of these items and a comprehensive result obtained by integrating these calculation results. Based on these calculation results, the comprehensive result is displayed using, for example, an image of an animal's walking style as a metaphorical representation of the characteristic walking style, such as "Your walking style type is penguin type level 2".

[0188] Display information 102 is an image representing the calculation results of pedestrian 1's physical abilities, specifically the calculation results of pedestrian 1's physical abilities within the calculation unit 42g. Display information 102 is another example of auxiliary information. For example, display information 102 includes calculation results for five items (muscle strength, trunk function, work ability, balance ability, and reflex ability) as items representing pedestrian 1's physical abilities. Display information 102 may include, for example, the calculation results for these items and a comprehensive result obtained by integrating these calculation results. For example, the comprehensive result may be displayed as "Physical Score 65 points."

[0189] In addition, for example, suggestions for these purposes (e.g., recommendations for recommended exercises, etc.) can be displayed when the user selects "view walking suggestions" or "view physical ability suggestions" using an operating device such as a mouse, keyboard and / or touch panel.

[0190] Figure 7 This is a diagram used to illustrate specific examples of physical abilities. Specifically, Figure 7 This shows specific examples of physical abilities calculated by the 42g calculation unit, the measurement methods (measurement items) for each physical ability, and the measurement content for each item.

[0191] The physical abilities being calculated include, for example, muscle strength, reflexes, balance, trunk function, and work capacity. These physical abilities are calculated, for example, through the two-step test, walking start reaction time, standing on one leg with eyes closed, the Functional Reach Test (FRT), and the Timed Up & Go Test (TUG). Pedestrian 1's physical abilities can also be calculated by performing these measurements.

[0192] In this embodiment, the calculation unit 42g uses characteristic quantities to calculate physical capabilities.

[0193] Figure 8 This is a diagram illustrating specific examples of characteristic quantities used to calculate physical capabilities. Specifically, Figure 8 The diagram shows specific examples of physical abilities calculated by the calculation unit 42g, as well as specific examples of characteristic quantities used in the calculation of physical abilities for each item during each of the multiple periods.

[0194] For example, the computing unit 42g uses Figure 8 The physical abilities of each item corresponding to each period are calculated by using the characteristic quantities shown in each of the multiple periods, and the physical abilities are calculated by adding the calculation results of the multiple periods together.

[0195] Figure 9 This is a diagram illustrating a specific example of characteristic quantities used to calculate walking patterns. Specifically, Figure 9 The diagram shows specific examples of walking patterns calculated by the calculation unit 42g, as well as specific examples of characteristic quantities used in the calculation of walking patterns for each item.

[0196] The calculations include walking patterns such as foot movements (rocker function), arm swings, pelvic tilt (head and torso tilt), and torso movement (swaying and balance). For example, the 42g calculation unit uses... Figure 9 The walking pattern for each item is calculated using the various characteristic quantities shown. For example, the calculation unit 42g calculates the appropriate degree of foot movement. Additionally, for example, the calculation unit 42g calculates the appropriate degree of arm swing during walking.

[0197] Furthermore, the aforementioned items related to physical abilities and walking styles, as well as the characteristic quantities used to calculate them, are merely examples and can be arbitrarily determined. Additionally, the number of physical abilities and walking styles being calculated can be one or more.

[0198] The acquisition unit 42a, estimation unit 42b, classification unit 42c, extraction unit 42d, determination unit 42e, output unit 42f, and calculation unit 42g are each implemented, for example, by a memory storing a control program and a processor executing the control program. The processor for each processing unit can be implemented using a single memory and processor, or they can be implemented using separate memories and processors.

[0199] The storage unit 43 is a storage device that stores the motion image (motion image data) acquired by the acquisition unit 42a, the learned model described above, the database described above, and various threshold information described above. The storage unit 43 is implemented, for example, by a semiconductor memory or an HDD.

[0200] As described above, in the fall risk assessment device 40, the fall risk of pedestrian 1 is determined (estimated) based on the characteristic quantities (walking characteristic quantities) of pedestrian 1 (e.g., the degree to which they are likely to fall). Additionally, for example, the fall risk assessment device 40 calculates (estimates) the walking style of pedestrian 1 (e.g., the state and / or tendency of their walking). Furthermore, for example, the fall risk assessment device 40 calculates the physical abilities of pedestrian 1 (e.g., the level of ability related to the current walking style and fall risk). In this way, by determining the information related to the fall risk of pedestrian 1, in other words, by quantifying it, it is possible to estimate the degree to which pedestrian 1 is likely to fall, and to estimate the information needed based on considering the causes and future countermeasures. For example, using the walking characteristic quantities, estimates are made of physical abilities (e.g., the five items mentioned above) considered to be related to falls, and walking styles are estimated, and recommendations corresponding to these results are output.

[0201] For example, in estimating physical abilities, we estimate which of the five physical abilities of pedestrian 1 is high and which is low. Here, for example, we output a recommendation for the item that is estimated to be low.

[0202] Additionally, for example, in the estimation of walking patterns, the posture and movement of pedestrian 1 during walking are estimated. Based on the estimation results, a recommendation regarding walking patterns is output.

[0203] For example, in the estimation of physical ability based on walking, the movement used as the true value of physical strength measurement is also universal, and the characteristic quantities of each walking interval become important so that differences in characteristics can be easily detected for each item of physical ability for each individual, for example, during the period of starting to walk. On the other hand, in the estimation of walking style (gait), the evaluation of the overall walking posture, etc., becomes important.

[0204] As mentioned above, for example, based on the determination of walking style and the estimation of physical ability, the best recommendation is output. For example, it may display statements such as "People with walking style ○○ have this tendency, so pay attention to walking in ××", or "The ability of △△ in physical ability has decreased, so there is a tendency to □□, and this kind of exercise is better".

[0205] [Processing Procedure]

[0206] Next, the processing procedure of display system 10 will be explained.

[0207] Figure 10 This is a timing diagram illustrating the processing flow of the display system 10 according to Embodiment 1.

[0208] First, the instruction unit 36 ​​instructs pedestrian 1 to start walking (S201). For example, when the receiving unit 34 receives an instruction from the user to make pedestrian 1 start walking, the instruction unit 36 ​​gives an instruction such as "Please start walking".

[0209] Alternatively, upon receiving an instruction, the control unit 32 may acquire a moving image captured by the camera 20 and identify the pedestrian 1 in the acquired moving image. The identification of the pedestrian 1 in the moving image may utilize known image analysis techniques such as pattern matching.

[0210] Next, the camera 20 generates a motion image containing the pedestrian 1 as the subject by capturing the pedestrian 1 walking as the subject (S202).

[0211] Next, the control unit 32 outputs the motion image generated by the camera 20 to the fall risk assessment device 40 via the communication unit 31 (S203). At this time, the control unit 32 can also send the motion image to the fall risk assessment device 40 after anonymizing it. This protects the privacy data of pedestrian 1.

[0212] Through the processing of the information terminal 30, the fall risk assessment device 40 acquires motion images and determines the fall risk of pedestrian 1 based on the acquired motion images (S100). The fall risk assessment device 40 outputs the assessment result to the information terminal 30.

[0213] Next, the control unit 32 obtains the determination result via the communication unit 31 (S204).

[0214] Next, the display unit 35 displays the determination result obtained by the control unit 32 (S205).

[0215] Figure 11 This is a flowchart illustrating the processing procedure of the fall risk assessment device 40 according to Embodiment 1. Specifically, Figure 11 This is a flowchart showing the details of the process in step S100.

[0216] First, the acquisition unit 42a acquires a motion image of the pedestrian 1 (S110). Specifically, the acquisition unit 42a acquires a motion image of the pedestrian 1 from the start of walking until it stops.

[0217] Next, the estimation unit 42b estimates the skeleton of the pedestrian 1 reflected in the motion image based on the motion image acquired by the acquisition unit 42a (S120). Specifically, the estimation unit 42b calculates a skeleton model representing the skeleton of the pedestrian 1 reflected in the motion image based on the motion image acquired by the acquisition unit 42a.

[0218] Next, the classification unit 42c classifies the period of pedestrian 1 walking in the motion image acquired by the acquisition unit 42a into multiple periods based on the skeleton estimated by the estimation unit 42b (S130). For example, the classification unit 42c classifies the period of pedestrian 1 walking in the motion image acquired by the acquisition unit 42a into three periods: acceleration period, constant speed period, and deceleration period, based on the skeletal model calculated by the estimation unit 42b. For example, the classification unit 42c calculates the velocity (movement speed) of the skeletal point representing the heel in the skeletal model based on the time change of the coordinates of the skeletal point representing the heel as the velocity (movement speed) of pedestrian 1. For example, the classification unit 42c classifies the period of pedestrian 1 walking into acceleration period, constant speed period, and deceleration period based on the velocity of pedestrian 1.

[0219] Next, the extraction unit 42d extracts one or more features related to the walking of pedestrian 1 based on the skeleton of pedestrian 1 estimated by the estimation unit 42b for each of the multiple periods classified by the classification unit 42c (S140). For example, the extraction unit 42d extracts one or more features for each of the acceleration, constant speed, and deceleration periods. For example, the extraction unit 42d extracts a first feature, a second feature, and a third feature for the acceleration period, a fourth feature, a fifth feature, and a sixth feature for the constant speed period, and a seventh feature and an eighth feature for the deceleration period. For example, each feature is a feature of a different type. Alternatively, for example, the first feature and the third feature may be features of the same type, and some of the features extracted in each period may be features of the same type.

[0220] Next, the determination unit 42e determines the fall risk of pedestrian 1 based on one or more feature values ​​for each of the multiple periods extracted by the extraction unit 42d (S150). For example, the determination unit 42e determines the fall risk corresponding to the acceleration period based on one or more feature values ​​extracted during the acceleration period. Additionally, the determination unit 42e determines the fall risk corresponding to the constant speed period based on one or more feature values ​​extracted during the constant speed period. Furthermore, the determination unit 42e determines the fall risk corresponding to the deceleration period based on one or more feature values ​​extracted during the deceleration period. Finally, the determination unit 42e evaluates (determines) the fall risk of pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period.

[0221] Next, the output unit 42f outputs the determination result of the determination unit 42e to the information terminal 30 (S160). As a result, the determination result of the determination unit 42e is displayed on the display unit 35 of the information terminal 30.

[0222] (Implementation Method 2)

[0223] Next, the fall risk assessment device according to Embodiment 2 will be described. Furthermore, in the description of Embodiment 2, some or all of the descriptions that overlap with those in Embodiment 1 may be omitted or simplified.

[0224] Figure 12 This is a block diagram illustrating the functional structure of the fall risk assessment device 400 according to Embodiment 2.

[0225] Fall risk assessment device 400, for example Figure 1 The display system 10 shown is a computer provided as an alternative to the fall risk assessment device 40. The fall risk assessment device 400, for example, assesses (evaluates) the fall risk of pedestrian 1 using the information terminal 30.

[0226] In this embodiment, the fall risk assessment device 400 assesses the fall risk without using a moving image containing the pedestrian 1 as the subject. In other words, the display system equipped with the fall risk assessment device 400 is a system that assesses the fall risk without using a moving image containing the pedestrian 1 as the subject.

[0227] For example, in Embodiment 1 described above, the acquisition unit 42a acquires motion images of pedestrian 1 captured by camera 20, and the estimation unit 42b estimates the skeleton of pedestrian 1.

[0228] Here, in addition to the method using camera 20, the estimation of the skeleton of pedestrian 1 can also be achieved using radio waves. For example, the skeleton (pose) of pedestrian 1 can be estimated based on the propagation and reflection states of radio waves. For example, the propagation and reflection states of radio waves can be obtained by sending radio waves to pedestrian 1 and receiving the reflected waves of the sent radio waves.

[0229] For example, in estimating the skeleton of pedestrian 1, multipath propagation can be utilized using radio wave reflections accompanying WiFi (trademark). For instance, a pre-learned machine learning model that has learned the correlation between multipath propagation and walking motion is generated and pre-stored in the fall risk assessment device 400. Furthermore, in a WiFi environment, pedestrian 1 walks, multipath propagation is detected, and the fall risk assessment device 400 acquires information representing the detected multipath propagation. Based on the information representing changes in the multipath propagation detected in this way, the fall risk assessment device 400 uses the machine learning model to estimate the skeleton (posture) of pedestrian 1. For example, a WiFi access point is set up at the location where pedestrian 1 walks, and multipath propagation is detected using multiple receivers. The fall risk assessment device 400 estimates the skeleton of pedestrian 1 based on the information thus detected.

[0230] In this way, the skeleton of pedestrian 1 can be estimated without using the images captured by camera 20, but by using other means to detect the skeleton of pedestrian 1.

[0231] Furthermore, the intensity and wavelength of the radio waves used in estimating the skeleton of pedestrian 1, as well as the communication standards used for multipath detection, can be arbitrarily determined without any particular limitation.

[0232] For example, a user walks in such a place. The information generated (motion information) is sent to the fall risk assessment device 400. In the fall risk assessment device 400, based on the received motion information, the fall risk of pedestrian 1 is assessed, and the assessment result is sent to the information terminal 30. The information terminal 30 displays the received assessment result.

[0233] Furthermore, in this embodiment, the display system may also include a WiFi access point and multiple receivers. Additionally, in this embodiment, the information terminal 30 used by pedestrian 1 may not include a camera 20. For example, in this embodiment, the information terminal 30 may not need to capture images of pedestrian 1 using the camera 20. Furthermore, the number of multiple receivers can be arbitrarily determined without particular limitation. Moreover, the configuration layout of the access point and multiple receivers can also be arbitrarily set.

[0234] The fall risk assessment device 400 includes a communication unit 41, an information processing unit 420, and a storage unit 43.

[0235] The communication unit 41 is a communication interface for communicating with the information terminal 30. Specifically, in the communication unit 41, the fall risk assessment device 400 communicates with the information terminal 30 via a network 5 such as the Internet. The communication unit 41 is implemented, for example, by a wireless communication circuit for wireless communication with the information terminal 30.

[0236] The information processing unit 420 is a processing unit that performs various information processing in the fall risk assessment device 400. The information processing unit 420 is implemented, for example, by a microcomputer. Alternatively, the information processing unit 420 may also be implemented by a processor such as a CPU. The functions of the information processing unit 420 are implemented, for example, by the microcomputer or processor constituting the information processing unit 420 executing computer programs stored in the storage unit 43.

[0237] Storage unit 43 is a storage device for storing various types of information. In this embodiment, for example, storage unit 43 stores a machine learning model that has been pre-learned to correlate multipaths with walking movements. Of course, storage unit 43 may also store the learned model, database, and various threshold information described in Embodiment 1 above.

[0238] The information processing unit 420 includes a walking motion acquisition unit 420a, a classification unit 420c, an extraction unit 420d, a judgment unit 420e, and an output unit 420f.

[0239] Like the acquisition unit 42a, the walking motion acquisition unit 420a is a processing unit that acquires various types of information. The walking motion acquisition unit 420a acquires the walking motion of pedestrian 1. Specifically, the walking motion acquisition unit 420a acquires motion information representing the walking motion of pedestrian 1. This motion information may be, for example, information representing the aforementioned multipath. For instance, the walking motion acquisition unit 420a acquires information representing radio waves received by multiple receivers (not shown) via the communication unit 41 as motion information.

[0240] Classification unit 420c, like classification unit 42c, is a processing unit that classifies the period of pedestrian 1's walking into multiple periods. Based on the motion information obtained by walking motion acquisition unit 420a, classification unit 420c classifies the period from when pedestrian 1 starts walking to when he stops walking into multiple periods.

[0241] The classification unit 420c, for example, estimates the movement of the skeleton (skeleton model) of the pedestrian 1's walking action based on the machine learning model stored in the storage unit 43. Similarly to the processing performed by the classification unit 42c, it calculates the speed (movement speed) of the skeleton point representing the heel as the speed (movement speed) of the pedestrian 1 based on the time change of the coordinates of the skeleton point representing the heel in the skeleton model, and classifies the period of the pedestrian 1's walking into three periods: acceleration period, constant speed period, and deceleration period.

[0242] Extraction unit 420d and extraction unit 42d are both processing units that extract one or more features (first features) related to pedestrian 1's walking for each of the multiple periods. Extraction unit 420d extracts one or more first features related to pedestrian 1's walking for each of the multiple periods classified by classification unit 420c, based on the skeletal motion of pedestrian 1's walking action estimated by a machine learning model using motion information.

[0243] Both determination unit 420e and determination unit 42e are processing units that determine the fall risk of pedestrian 1 based on one or more feature quantities (for the first feature). Determination unit 420e determines the fall risk of pedestrian 1 based on one or more feature quantities for each of the multiple periods extracted by extraction unit 420d. For example, determination unit 420e determines the fall risk corresponding to the acceleration period based on one or more feature quantities extracted during the acceleration period. Additionally, determination unit 420e determines the fall risk corresponding to the constant speed period based on one or more feature quantities extracted during the deceleration period. Furthermore, determination unit 420e determines the fall risk corresponding to the deceleration period based on one or more feature quantities extracted during the deceleration period. Finally, determination unit 420e determines (evaluates) the fall risk of pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period. For example, the determination unit 420e evaluates the fall risk of pedestrian 1 by adding the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period.

[0244] Output unit 420f, like output unit 42f, is a processing unit that outputs the determination result. Output unit 420f outputs the determination result of determination unit 420e.

[0245] [Processing Procedure]

[0246] Next, the processing procedure of the fall risk assessment device 400 will be explained.

[0247] Figure 13 This is a flowchart illustrating the processing procedure of the fall risk assessment device 400 according to Embodiment 2.

[0248] First, the walking motion acquisition unit 420a acquires the walking motion of pedestrian 1. Specifically, the walking motion acquisition unit 420a acquires motion information representing the walking motion of pedestrian 1 from the start of walking until it stops (S310).

[0249] Next, the classification unit 420c classifies the walking period of pedestrian 1 into multiple periods (S320). For example, based on the motion information obtained by the walking motion acquisition unit 420a, the classification unit 420c classifies the walking period of pedestrian 1 into three periods: acceleration period, constant speed period, and deceleration period. For example, the classification unit 420c calculates the speed (movement speed) of the bone point representing the heel in the skeletal model based on the time change of the bone point representing the heel, and uses this speed as the speed (movement speed) of pedestrian 1. For example, based on the speed of pedestrian 1, the classification unit 420c classifies the walking period of pedestrian 1 into acceleration period, constant speed period, and deceleration period.

[0250] Next, the extraction unit 420d extracts one or more features related to the walking of pedestrian 1 based on the skeletal movement of the pedestrian 1's walking motion for each of the multiple periods classified by the classification unit 420c (S330). For example, the extraction unit 420d extracts one or more features for each of the acceleration, constant speed, and deceleration periods. For example, the extraction unit 420d extracts a first feature, a second feature, and a third feature for the acceleration period, a fourth feature, a fifth feature, and a sixth feature for the constant speed period, and a seventh feature and an eighth feature for the deceleration period. For example, each feature is a feature of a different type. Alternatively, for example, the first feature and the third feature may be features of the same type, and some of the features extracted in each period may be features of the same type.

[0251] Next, the determination unit 420e determines the fall risk of pedestrian 1 based on one or more feature values ​​for each of the multiple periods extracted by the extraction unit 420d (S340). For example, the determination unit 420e determines the fall risk corresponding to the acceleration period based on one or more feature values ​​extracted during the acceleration period. Additionally, the determination unit 420e determines the fall risk corresponding to the constant speed period based on one or more feature values ​​extracted during the constant speed period. Furthermore, the determination unit 420e determines the fall risk corresponding to the deceleration period based on one or more feature values ​​extracted during the deceleration period. Finally, the determination unit 420e evaluates (determines) the fall risk of pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant speed period, and the fall risk corresponding to the deceleration period.

[0252] Next, the output unit 420f outputs the determination result of the determination unit 420e (S350). For example, the output unit 420f outputs the determination result of the determination unit 420e to the information terminal 30. As a result, the determination result of the determination unit 420e is displayed on the display unit 35 of the information terminal 30.

[0253] [Effects, etc.]

[0254] The following describes the techniques obtained from the disclosure of this specification and explains the effects obtained from the illustrated techniques.

[0255] Technology 1 is a fall risk determination device 40, comprising: an acquisition unit 42a that acquires a motion image reflecting a pedestrian 1; an estimation unit 42b that estimates the skeleton of the pedestrian 1 reflected in the motion image based on the motion image; a classification unit 42c that classifies the period during which the pedestrian 1 walks in the motion image into multiple periods based on the skeleton; an extraction unit 42d that extracts one or more first features related to the pedestrian 1's walking based on the skeleton for each of the multiple periods; a determination unit 42e that determines the fall risk of the pedestrian 1 based on one or more first features; and an output unit 42f that outputs the determination result of the determination unit 42e.

[0256] Accordingly, the fall risk assessment device 40 extracts one or more feature quantities for each period (each walking phase) based on the estimated skeleton (skeleton model) of pedestrian 1. The inventors of this application have discovered that by extracting one or more feature quantities in each period, such as the acceleration period, the constant speed period, and the deceleration period, to determine the fall risk, for example, by adding the fall risks in each period to determine (calculate) the fall risk of pedestrian 1, the fall risk of pedestrian 1 can be accurately, that is, with high precision. Therefore, according to the fall risk assessment device 40, the fall risk of pedestrian 1 can be determined with high precision. Furthermore, according to the fall risk assessment device 40, the fall risk of pedestrian 1 can be calculated based on the walking state of pedestrian 1, thus allowing for the calculation of the fall risk of pedestrian 1 through simple processing.

[0257] Technique 2 is the fall risk determination device 40 according to Technique 1, wherein the determination unit 42e uses a machine learning model to determine the fall risk of pedestrian 1. The machine learning model is learned based on features related to a person's walking and a fall history indicating whether the person has fallen within a specified period.

[0258] Therefore, it is possible to determine the risk of pedestrian 1 falling in a simple and highly accurate manner.

[0259] Technique 3 is the fall risk assessment device 40 according to Technique 1 or 2, wherein one or more first feature quantities include any one of the following: a feature quantity related to the angle of the joint during the pedestrian 1's walking, a feature quantity related to the speed of joint movement, a feature quantity related to the acceleration of joint movement, a feature quantity related to the walking time, and a feature quantity related to the distance of joint movement.

[0260] The inventors of this application have discovered that by using these feature quantities to determine the fall risk of pedestrian 1, the fall risk of pedestrian 1 can be determined with high accuracy. Therefore, the fall risk of pedestrian 1 can be determined with high accuracy. Furthermore, for example, the fall risk of pedestrian 1 can be determined without extracting feature quantities that can also be used for simple and high-accuracy fall risk determination, thus reducing the processing workload for fall risk determination while still achieving high-accuracy fall risk determination.

[0261] Technique 4 is a fall risk assessment device 40 according to any one of Techniques 1 to 3, wherein one or more first characteristic quantities include at least one of the following: average speed of toes, walking speed, support time of both legs, amplitude of forward and backward sway of the torso, maximum speed of toes, stride length, change in waist height, range of motion of hip joint, heel contact angle, up and down movement of chest, range of motion of ankle joint, change in knee height, and range of motion of knee joint angle.

[0262] The inventors of this application have discovered that, in particular, by using these feature quantities to determine the fall risk of pedestrian 1, the fall risk of pedestrian 1 can be determined with even higher accuracy. Therefore, the fall risk of pedestrian 1 can be determined with even higher accuracy. Furthermore, for example, by determining the fall risk of pedestrian 1 without extracting feature quantities that can also be used for simple and high-accuracy fall risk determination, the processing workload for fall risk determination can be reduced while achieving even higher accuracy in fall risk determination.

[0263] Technique 5 is a fall risk assessment device 40 according to any one of Techniques 1 to 4, wherein the multiple periods include: an acceleration period in which pedestrian 1 begins to walk and accelerates; a constant speed period, which is the period in which pedestrian 1 walks steadily and is the next period after the acceleration period; and a deceleration period, which is the period in which pedestrian 1 decelerates until it stops and is the next period after the constant speed period.

[0264] The acceleration period, constant velocity period, and deceleration period can be easily divided based on the pedestrian 1's speed and / or acceleration. Therefore, by defining multiple periods as acceleration, constant velocity, and deceleration periods, each period can be easily divided.

[0265] Technology 6 is that the fall risk assessment device 40 according to any one of technologies 1 to 5 further includes a calculation unit 42g, which calculates auxiliary indicators for interpreting the fall risk of pedestrian 1, and an output unit 42f outputs auxiliary information representing the auxiliary indicators.

[0266] Therefore, by outputting auxiliary information along with the judgment result, it is possible to easily determine what countermeasures to take for a pedestrian who is judged to be at high risk of falling.

[0267] Technology 7 is the fall risk assessment device 40 according to Technology 6, wherein the extraction unit 42d extracts one or more second feature quantities related to the walking of pedestrian 1 based on the skeleton for each of multiple periods, and the calculation unit 42g calculates the physical ability of pedestrian 1 as an auxiliary indicator based on the one or more second feature quantities. The one or more second feature quantities include any one of the following in the walking of pedestrian 1: feature quantity related to joint angle, feature quantity related to joint movement speed, feature quantity related to joint movement acceleration, feature quantity related to walking movement time, and feature quantity related to joint movement distance.

[0268] The inventors of this application have discovered that by using these feature quantities to calculate the physical abilities of pedestrian 1, it is possible to determine the physical abilities of pedestrian 1 with high accuracy. Therefore, it is possible to determine the physical abilities of pedestrian 1 with high accuracy. Furthermore, for example, by determining the physical abilities of pedestrian 1 without extracting feature quantities that can be used for simple and high-accuracy assessment of physical abilities, the processing load for determining physical abilities can be reduced while still achieving high-accuracy assessment. Additionally, for example, by outputting auxiliary information such as pedestrian 1's lower than average physical abilities and / or declining physical abilities, it is possible to easily determine what countermeasures should be taken for pedestrian 1 who is deemed to have a high risk of falling.

[0269] Technical 8 is a fall risk assessment device 40 according to technical 6 or 7, wherein the extraction unit 42d extracts one or more third feature quantities related to the walking of pedestrian 1 based on the skeleton during the walking period of pedestrian 1, and the calculation unit 42g calculates the walking mode of pedestrian 1 as an auxiliary indicator based on one or more third feature quantities. The one or more third feature quantities include at least one of the following during the walking of pedestrian 1: walking speed, back and forth sway of the body, back and forth range of motion of the hip joint, back and forth range of motion of the knee joint, support time of both legs, standing time, heel height, heel contact angle, heel lift angle, waist up and down movement, chest up and down movement, stride width, back and forth sway of the head, back and forth sway of the hands, back and forth sway of the elbows, stride length, and step length.

[0270] The inventors of this application have discovered that by using these feature quantities to calculate the walking pattern of pedestrian 1, the walking pattern of pedestrian 1 can be determined with high accuracy. Therefore, the walking pattern of pedestrian 1 can be determined with high accuracy. Furthermore, for example, the walking pattern of pedestrian 1 can be determined without extracting feature quantities that can be used for simple and high-accuracy walking pattern determination, thus reducing the processing load for walking pattern determination while achieving high accuracy. Additionally, for example, by outputting the walking pattern of pedestrian 1 as auxiliary information, it is possible to easily determine what countermeasures should be taken for pedestrian 1 who is deemed to have a high risk of falling.

[0271] Technical 9 is a fall risk assessment method executed by a computer, the fall risk assessment method comprising: an acquisition step (S110) of acquiring a motion image reflecting a pedestrian 1; an estimation step (S120) of estimating the skeleton of the pedestrian 1 reflected in the motion image based on the motion image; a classification step (S130) of classifying the period of the pedestrian 1 walking in the motion image into multiple periods based on the skeleton; an extraction step (S140) of extracting one or more first features related to the walking of the pedestrian 1 based on the skeleton for each of the multiple periods; a determination step (S150) of determining the fall risk of the pedestrian 1 based on one or more first features; and an output step (S160) of outputting the determination result in the determination step.

[0272] Therefore, it achieves the same effect as the fall risk assessment device 40.

[0273] Technique 10 is a program for enabling a computer to execute the fall risk assessment method according to Technique 9.

[0274] Therefore, it achieves the same effect as the fall risk assessment device 40.

[0275] Technical 11 is a fall risk determination device 400, comprising: a walking motion acquisition unit 420a, which acquires the walking motion of a pedestrian 1; a classification unit 420c, which classifies the walking period of the pedestrian 1 into multiple periods; an extraction unit 420d, which extracts one or more first feature quantities related to the walking of the pedestrian 1 based on the skeletal movement of the walking motion of the pedestrian 1 for each of the multiple periods; a determination unit 420e, which determines the fall risk of the pedestrian 1 based on one or more first feature quantities; and an output unit 420f, which outputs the determination result of the determination unit 420e.

[0276] Accordingly, the fall risk assessment device 400 extracts one or more feature quantities for each period (each walking phase) based on the movement of the skeleton (skeleton model) of pedestrian 1's walking action. The inventors of this application have discovered that by extracting one or more feature quantities in each period, such as the acceleration period, the constant speed period, and the deceleration period, to determine the fall risk, for example, by adding the fall risks in each period to determine (calculate) the fall risk of pedestrian 1, the fall risk of pedestrian 1 can be accurately, that is, with high precision. Therefore, according to the fall risk assessment device 400, the fall risk of pedestrian 1 can be determined with high precision. Furthermore, according to the fall risk assessment device 400, the fall risk of pedestrian 1 can be calculated based on the walking state of pedestrian 1, thus allowing for the calculation of the fall risk of pedestrian 1 through simple processing.

[0277] (Other implementation methods)

[0278] The above describes various implementation methods, but this disclosure is not limited to the above implementation methods.

[0279] For example, the instruction to start walking for pedestrian 1 can also be given by the user. In this case, the information terminal 30 may not have an instruction unit.

[0280] In addition, the information terminal 30 can also send the motion image of the pedestrian 1 while walking to the fall risk assessment device 40.

[0281] Alternatively, for example, the information terminal 30 may send an instruction to the fall risk assessment device 40, indicating that the assessment of the pedestrian 1's fall risk has begun, based on an instruction received by the receiving unit 34 from the user. In this case, for example, the fall risk assessment device 40 may also send information indicating an instruction to the information terminal 30 to cause the pedestrian 1 to begin walking. In this case, the information terminal 30 may also, based on the received information, cause the pedestrian 1 to begin walking via the instruction unit 36 ​​and capture an image of the pedestrian 1 on the camera 20.

[0282] Furthermore, the first feature, the second feature, and the third feature can refer to the same feature or different feature values.

[0283] Furthermore, for example, in the embodiments described above, the processing performed by the specific processing unit may also be executed by other processing units. Additionally, the order of multiple processes can be changed, or multiple processes can be executed in parallel.

[0284] Furthermore, for example, in the embodiments described above, each component of the processing unit, such as the information processing unit 42 and 420, can also be implemented by executing software programs suitable for each component. Each component can also be implemented by reading and executing software programs recorded in a recording medium such as a hard disk or semiconductor memory by a program execution unit such as a CPU or processor.

[0285] Furthermore, each component can be implemented using hardware. Each component can also be a circuit (or integrated circuit). These circuits can either form a single circuit as a whole, or they can be separate circuits. Moreover, these circuits can be either general-purpose or specialized circuits.

[0286] Furthermore, there are no particular limitations on the communication methods between the devices in the above embodiments. Additionally, relay devices (such as broadband routers, not shown) may be used in the communication between the devices.

[0287] Furthermore, the general or specific embodiments of this disclosure can also be implemented by non-transitory recording media such as systems, apparatuses, methods, integrated circuits, computer programs, or computer-readable CD-ROMs. Additionally, it can be implemented by any combination of systems, apparatuses, methods, integrated circuits, computer programs, and recording media.

[0288] Alternatively, this disclosure can be implemented as a fall risk assessment method, as a program for causing a computer to execute the fall risk assessment method, or as a computer-readable, non-transitory recording medium containing such a program.

[0289] Furthermore, in Embodiment 1 described above, an example was shown where the display system 10 includes an information terminal 30 and a fall risk assessment device 40. However, the display system disclosed herein can be implemented as a single device such as an information processing device, or it can be implemented using multiple devices. For example, the display system can also be implemented as a client-server system. When the display system is implemented using multiple devices, the constituent elements of the display system described in Embodiment 1 can be arbitrarily distributed among the multiple devices. For example, in Embodiment 1, the fall risk assessment device 40 extracts feature quantities. The extraction of feature quantities can also be performed by the information terminal 30. Additionally, for example, the information terminal 30 may include an estimation unit. In this case, for example, the fall risk assessment device can also acquire information representing the skeleton (specifically, the skeleton model) estimated by the information terminal, and perform the aforementioned classification unit, extraction unit, assessment unit, and output unit processing based on the acquired information. When the display system disclosed herein includes a fall risk assessment device 400, the components of the display system can be arbitrarily assigned to multiple devices, similar to the fall risk assessment device 40.

[0290] Furthermore, the display system disclosed herein may include either the fall risk assessment device 40 or the fall risk assessment device 400, or both. Additionally, the fall risk assessment device disclosed herein may be implemented by arbitrarily combining the functional structures of the fall risk assessment device 40 and the fall risk assessment device 400. For example, the fall risk assessment device disclosed herein may include both an acquisition unit 42a and a walking motion acquisition unit 420a. For example, the estimation unit, classification unit, extraction unit, determination unit, and output unit included in the fall risk assessment device disclosed herein may perform predetermined processing based on the information acquired by the acquisition unit 42a or the walking motion acquisition unit 420a, as described in the above embodiments.

[0291] Furthermore, the fall risk assessment device 40 may or may not include a calculation unit 42g. Similarly, the fall risk assessment device 400 may or may not include a calculation unit 42g.

[0292] Furthermore, this disclosure also includes various modifications to the embodiments that would be conceived by those skilled in the art, or implementations by arbitrarily combining the constituent elements and functions of the embodiments without departing from the spirit of this disclosure.

[0293] Explanation of reference numerals in the attached figures

[0294] 1: Pedestrian; 40, 400: Fall risk assessment device; 42a: Acquisition unit; 42b: Estimation unit; 42c, 420c: Classification unit; 42d, 420d: Extraction unit; 42e, 420e: Judgment unit; 42f, 420f: Output unit; 42g: Calculation unit; 420a: Walking motion acquisition unit.

Claims

1. A fall risk assessment device, comprising: The acquisition department acquires and reflects images of human motion. An estimation unit that estimates the skeleton of the pedestrian as reflected in the motion image based on the motion image; A classification department, which classifies the period of the pedestrian's walking in the motion image into multiple periods based on the skeleton; An extraction unit, for each of the plurality of periods, extracts one or more first feature quantities related to the pedestrian's walking based on the skeleton; The determination unit determines the pedestrian's risk of falling based on one or more of the first feature quantities; as well as The output unit outputs the determination result of the determination unit.

2. The fall risk assessment device according to claim 1, wherein, The determination unit uses a machine learning model to determine the pedestrian's risk of falling. The machine learning model is learned based on features related to a person's walking and a history of falls indicating whether the person has fallen within a specified period.

3. The fall risk assessment device according to claim 1, wherein, The one or more first characteristic quantities include any one of the following during the pedestrian's walking: characteristic quantity related to joint angle, characteristic quantity related to joint movement speed, characteristic quantity related to joint movement acceleration, characteristic quantity related to walking movement time, and characteristic quantity related to joint movement distance.

4. The fall risk assessment device according to claim 1, wherein, The first characteristic quantity includes at least one of the following during the pedestrian's walking: average speed of the toes, walking speed, duration of support of both legs, amplitude of forward and backward sway of the torso, maximum speed of the toes, stride length, change in waist height, range of motion of the hip joint, angle of heel contact with the ground, vertical movement of the chest, range of motion of the ankle joint, change in knee height, and range of motion of the knee joint angle.

5. The fall risk assessment device according to claim 1, wherein, The plurality of periods include: an acceleration period in which the pedestrian begins to walk and accelerates; a constant speed period, which is the period in which the pedestrian walks steadily and is the next period after the acceleration period; and a deceleration period, which is the period in which the pedestrian decelerates until he or she stops and is the next period after the constant speed period.

6. The fall risk assessment device according to any one of claims 1 to 5, wherein, It also includes a calculation unit that calculates auxiliary indicators to explain the pedestrian's fall risk. The output unit also outputs auxiliary information representing the auxiliary indicators.

7. The fall risk assessment device according to claim 6, wherein, The extraction unit, for each of the plurality of periods, extracts one or more second feature quantities related to the pedestrian's walking based on the skeleton. The calculation unit calculates the pedestrian's physical capabilities as an auxiliary indicator based on one or more second feature quantities. The one or more second characteristic quantities include any one of the following during the pedestrian's walking: characteristic quantity related to joint angle, characteristic quantity related to joint movement speed, characteristic quantity related to joint movement acceleration, characteristic quantity related to walking movement time, and characteristic quantity related to joint movement distance.

8. The fall risk assessment device according to claim 6, wherein, The extraction unit extracts one or more third features related to the pedestrian's walking based on the skeleton during the pedestrian's walking period. The calculation unit calculates the pedestrian's walking pattern as the auxiliary indicator based on one or more third feature quantities. The one or more third characteristic quantities include at least one of the following during the pedestrian's walking: walking speed, body sway, hip joint range of motion, knee joint range of motion, leg support time, standing time, heel height, heel contact angle, heel lift angle, waist movement, chest movement, stride width, head sway, hand sway, elbow sway, stride length, and step length.

9. A fall risk assessment method, executed by a computer, the fall risk assessment method comprising: The acquisition step involves obtaining motion images that reflect the current human body. The estimation step involves estimating the skeleton of the pedestrian as reflected in the motion image based on the motion image; The classification step classifies the period of the pedestrian's walking in the motion image into multiple periods based on the skeleton; The extraction step involves extracting one or more first features related to the pedestrian's walking based on the skeleton for each of the plurality of periods. The determination step involves assessing the pedestrian's fall risk based on one or more of the first feature quantities. as well as The output step outputs the determination result from the determination step.

10. A program for causing the fall risk assessment method according to claim 9 to be executed by the computer.

11. A fall risk assessment device, comprising: The walking motion acquisition unit acquires the walking motion of pedestrians; The classification department categorizes the pedestrian's walking period into multiple periods; An extraction unit, for each of the plurality of periods, extracts one or more first feature quantities related to the pedestrian's walking based on the motion of the skeleton of the pedestrian's walking action; The determination unit determines the pedestrian's risk of falling based on one or more of the first feature quantities; as well as The output unit outputs the determination result of the determination unit.

Images