Fall risk determination device, fall risk determination method, and program

Abstract

A fall risk determination device (40) comprises: an acquisition unit (42a) that acquires a moving image in which a pedestrian (1) appears; an estimation unit (42b) that, on the basis of the moving image, estimates the skeleton of the pedestrian (1) appearing in the moving image; a classification unit (42c) that, on the basis of the skeleton, classifies the period during which the pedestrian (1) is walking in the moving image into a plurality of periods; an extraction unit (42d) that, for each of the plurality of periods, extracts, on the basis of the skeleton, one or more first feature amounts pertaining to the walking action of the pedestrian (1); a determination unit (42e) that determines the fall risk of the pedestrian (1) on the basis of the one or more first feature amounts; and an output unit (42f) that outputs the result of the determination by the determination unit (42e).

Description

Fall risk determination device, fall risk determination method, and program

[0001] The present disclosure relates to a fall risk determination device, a fall risk determination method, and a program.

[0002] In the past, the number of accidents involving falls among workers has been increasing, and measures to prevent falls have been strengthened. If workers can be informed of their own fall risk and physical abilities, they can deepen their self-understanding, which can encourage behavioral changes and lead to the prevention of fall accidents.

[0003] Patent Document 1 discloses a method for assessing the risk of a person walking to fall.

[0004] Japanese Patent Application Laid-Open No. 2021-30051

[0005] The present disclosure provides a fall risk determination device and the like that can accurately determine the fall risk of a pedestrian.

[0006] A fall risk assessment device according to one aspect of the present disclosure includes an acquisition unit that acquires a video image showing a pedestrian; an estimation unit that estimates the skeleton of the pedestrian shown in the video image based on the video image; a classification unit that classifies periods in which the pedestrian is walking in the video image into multiple periods based on the skeleton; an extraction unit that extracts, for each of the multiple periods, one or more first feature amounts related to the pedestrian's walking based on the skeleton; a determination unit that assesses the pedestrian's risk of falling based on the one or more first feature amounts; and an output unit that outputs the assessment result of the determination unit.

[0007] A fall risk assessment method according to one aspect of the present disclosure is a fall risk assessment method executed by a computer, and includes an acquisition step of acquiring a video image showing a pedestrian; an estimation step of estimating the skeleton of the pedestrian shown in the video image based on the video image; a classification step of classifying periods in which the pedestrian is walking in the video image into multiple periods based on the skeleton; an extraction step of extracting one or more first feature amounts related to the pedestrian's walking for each of the multiple periods based on the skeleton; a determination step of determining the pedestrian's fall risk based on the one or more first feature amounts; and an output step of outputting the determination result from the determination step.

[0008] Furthermore, a program according to one aspect of the present disclosure is a program for causing the computer to execute the fall risk assessment method.

[0009] Furthermore, a fall risk assessment device according to one aspect of the present disclosure includes a walking motion acquisition unit that acquires the walking motion of a pedestrian, a classification unit that classifies a period during which the pedestrian is walking into a plurality of periods, an extraction unit that extracts one or more first feature amounts related to the pedestrian's walking for each of the plurality of periods based on skeletal movement of the pedestrian's walking motion, a determination unit that assesses the pedestrian's risk of falling based on the one or more first feature amounts, and an output unit that outputs the assessment result of the determination unit.

[0010] According to a fall risk determination device according to one aspect of the present disclosure, the fall risk of a pedestrian can be determined with high accuracy.

[0011] FIG. 1 is a block diagram showing a functional configuration of a display system according to Embodiment 1. FIG. 2 is a diagram for explaining a skeletal model of a pedestrian according to Embodiment 1. FIG. 3 is a diagram for explaining a skeletal model of a pedestrian according to Embodiment 1. FIG. 4 is a diagram showing a specific example of a plurality of periods according to Embodiment 1. FIG. 5 is a diagram showing a first example of an image displayed by a display unit according to Embodiment 1. FIG. 6 is a diagram showing a second example of an image displayed by a display unit according to Embodiment 1. FIG. 7 is a diagram for explaining a specific example of physical ability. FIG. 8 is a diagram showing a specific example of feature amounts for calculating physical ability. FIG. 9 is a diagram showing a specific example of feature amounts for calculating a gait. FIG. 10 is a sequence diagram showing a processing procedure of a display system according to Embodiment 1. FIG. 11 is a flowchart showing a processing procedure of a fall risk assessment device according to Embodiment 1. FIG. 12 is a block diagram showing a functional configuration of a fall risk assessment device according to Embodiment 2. FIG. 13 is a flowchart showing a processing procedure of a fall risk assessment device according to Embodiment 2.

[0012] Each embodiment will be described in detail below with reference to the drawings. Note that each embodiment described below is a comprehensive or specific example. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in each embodiment below are examples and are not intended to limit the present disclosure. Furthermore, among the components in each embodiment below, components not recited in an independent claim will be described as optional components.

[0013] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0014] In each embodiment, the vertical direction is referred to as the Z-axis direction or up-down direction, a direction in a plane perpendicular to the vertical direction is referred to as the Y-axis direction or front-to-back direction, and a direction perpendicular to the Y-axis direction in that perpendicular plane is referred to as the X-axis direction, left-to-right direction, or lateral direction. In each embodiment, the positive side of the Z-axis direction is upward or up, and the negative side of the Z-axis direction is downward or down. In each embodiment, the positive side of the Y-axis direction is forward or front, and the negative side of the Y-axis direction is backward or back. In this disclosure, the positive side of the X-axis direction is right side or right, and the negative side of the X-axis direction is left side or left.

[0015] First Embodiment [Configuration] FIG. 1 is a block diagram showing the functional configuration of a display system 10 according to a first embodiment.

[0016] The display system 10 is a system that determines the risk of falling based on a moving image that includes a walking subject (hereinafter referred to as a pedestrian 1) as a subject (i.e., a moving image that shows the pedestrian 1).

[0017] The display system 10 includes an information terminal 30 and a fall risk assessment device 40 .

[0018] For example, the user operates the information terminal 30 to capture images of the pedestrian 1 from when he or she starts walking until when he or she stops. The video images generated in this way are transmitted to the fall risk assessment device 40. The fall risk assessment device 40 assesses the risk of the pedestrian 1 falling based on the received video images and transmits the assessment result to the information terminal 30. The information terminal 30 displays the received assessment result.

[0019] Here, the pedestrian 1 is a person whose risk of falling is determined.

[0020] Furthermore, the user is a person who uses the information terminal 30, such as a physical therapist, an occupational therapist, a nurse, or a rehabilitation specialist.

[0021] The user of the information terminal 30 may be anyone, including the pedestrian 1 .

[0022] The information terminal 30 is a computer that instructs the pedestrian 1 to start walking, acquires a moving image (moving image data) including the pedestrian 1 as a subject, which is generated by capturing an image of the pedestrian 1 by the camera 20, and transmits the acquired moving image to the fall risk assessment device 40. Specifically, the information terminal 30 generates a moving image by capturing an image of the pedestrian 1, and transmits the generated moving image to the fall risk assessment device 40.

[0023] The information terminal 30 is, for example, a portable computer device such as a smartphone or a tablet terminal used by a user, or may be a stationary computer device such as a personal computer.

[0024] The information terminal 30 includes a camera 20, a communication unit 31, a control unit 32, a storage unit 33, a reception unit 34, a display unit 35, and an instruction unit 36.

[0025] Camera 20 is a camera that captures an image of walking pedestrian 1 to generate a moving image including walking pedestrian 1 as a subject. In this embodiment, camera 20 captures an image of walking pedestrian 1 to generate a moving image in which pedestrian 1 is captured.

[0026] The camera 20 may be a camera using a complementary metal oxide semiconductor (CMOS) image sensor, or may be a camera using a charge coupled device (CCD) image sensor.

[0027] The camera 20 may be an external camera attached to the information terminal 30. In this case, the information terminal 30 does not need to include the camera 20, but only needs to include a communication interface for connecting to the camera 20 so as to be able to communicate with it.

[0028] For example, the number of cameras 20 may be one or more. The camera 20 may capture an image of the front of the pedestrian 1 or may capture an image of the pedestrian 1 from the side. The camera 20 may also have a motion capture function.

[0029] The communication unit 31 is a communication interface that communicates with the fall risk assessment device 40. Specifically, the communication unit 31 enables the information terminal 30 to communicate with the fall risk assessment device 40 via a network 5 such as the Internet. The communication unit 31 is realized, for example, by a wireless communication circuit for wirelessly communicating with the fall risk assessment device 40.

[0030] The communication standard used by the communication unit 31 for communication is not particularly limited.

[0031] Furthermore, the communication unit 31 may be connected to the fall risk assessment device 40 so as to be able to communicate wirelessly, or may be connected to the fall risk assessment device 40 so as to be able to communicate via wires. For example, when the communication unit 31 is connected to the fall risk assessment device 40 so as to be able to communicate via wires, the communication unit 31 is realized by a connector or the like that is connected to a communication line.

[0032] The control unit 32 is a processing unit that performs various types of information processing in the information terminal 30. The control unit 32 outputs, for example, a moving image generated by the camera 20 to the fall risk assessment device 40 via the communication unit 31. For example, when outputting a moving image, the control unit 32 links the multiple images that make up the moving image with time information at which each image was generated and outputs the images.

[0033] Furthermore, for example, the control unit 32 acquires the assessment result of the fall risk of the pedestrian 1 assessed by the fall risk assessment device 40 from the fall risk assessment device 40 via the communication unit 31. Furthermore, for example, the control unit 32 acquires the assessment result (assessment result information) by the fall risk assessment device 40 (more specifically, the assessment unit 42e) from the fall risk assessment device 40 via the communication unit 31. For example, the control unit 32 displays information indicating the acquired assessment result on the display unit 35. Furthermore, the control unit 32 performs various processes based on, for example, operation input accepted by the acceptance unit 34. The control unit 32 is realized, for example, by a microcomputer. Alternatively, the control unit 32 may be realized by a processor such as a CPU (Central Processing Unit). The functions of the control unit 32 are realized, for example, by the microcomputer or processor constituting the control unit 32 executing a dedicated application program stored in the storage unit 33.

[0034] The storage unit 33 is a storage device that stores dedicated application programs and the like to be executed by the control unit 32. The storage unit 33 is realized by, for example, a semiconductor memory or a hard disk drive (HDD).

[0035] The reception unit 34 is an input interface that receives operational inputs from the user of the information terminal 30. For example, the reception unit 34 receives user input operations such as an instruction to capture a moving image or an instruction to transmit a moving image to the fall risk assessment device 40. The reception unit 34 is realized, for example, by a touch panel display. For example, when the reception unit 34 is realized by a touch panel display, the touch panel display functions as the display unit 35 and the reception unit 34.

[0036] Note that the reception unit 34 is not limited to a touch panel display, and may be, for example, a keyboard, a pointing device such as a touch pen or a mouse, or a hardware button. Furthermore, the reception unit 34 may be a microphone when receiving voice input. Furthermore, the reception unit 34 may be a camera when receiving gesture input. In this case, the reception unit 34 may be realized by the camera 20, or may be realized by a camera different from the camera 20.

[0037] The display unit 35 is a display device that displays the evaluation results and the like obtained by the fall risk assessment device 40. The display unit 35 may be realized, for example, by a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel, or may be realized by an audio device such as a speaker or earphones, or may be realized by a display panel and an audio device.

[0038] The instruction unit 36 ​​is an instruction device that instructs the walker 1 to walk. The instruction unit 36 ​​instructs the walker 1 to walk, for example, by video and audio. That is, the instruction unit 36 ​​may instruct the user by video or audio.

[0039] The instruction unit 36 ​​issues instructions such as "Please walk" and "Please stop walking" by video and / or audio. The instruction unit 36 ​​may be realized by a display panel such as a liquid crystal panel or an organic EL panel, by an audio device such as a speaker or earphones, or by a display panel and an audio device.

[0040] In this embodiment, the instruction unit 36 ​​issues an instruction by voice, such as "Walk 10 meters from a stopped state and then stop." The user uses the camera 20 to capture images of the pedestrian 1 starting to walk from a stopped state, walking 10 meters, and then stopping. The information terminal 30 transmits the video captured in this manner to the fall risk assessment device 40.

[0041] The instruction unit 36 ​​and the display unit 35 may be realized by the same device such as a display panel and / or an audio device.

[0042] Furthermore, the video information and / or audio information used by the instruction unit 36 ​​to instruct the walker 1 to walk may be stored in advance in the storage unit 33 .

[0043] The fall risk assessment device 40 is a computer that acquires moving images transmitted from the information terminal 30 and edits at least one of the acquired moving images.

[0044] The fall risk assessment device 40 includes a communication unit 41 , an information processing unit 42 , and a memory unit 43 .

[0045] The communication unit 41 is a communication interface that communicates with the information terminal 30. Specifically, the communication unit 41 enables the fall risk assessment device 40 to communicate with the information terminal 30 via a network 5 such as the Internet. The communication unit 41 is realized, for example, by a wireless communication circuit for wirelessly communicating with the information terminal 30.

[0046] The communication standard for communication performed by the communication unit 41 is not particularly limited.

[0047] Furthermore, the communication unit 41 may be connected to the information terminal 30 so as to be able to communicate wirelessly, or may be connected to the information terminal 30 so as to be able to communicate via wires. For example, when the communication unit 41 is connected to the information terminal 30 so as to be able to communicate via wires, the communication unit 41 is realized by a connector or the like that is connected to a communication line or the like.

[0048] The information processing unit 42 is a processing unit that performs various types of information processing in the fall risk assessment device 40. The information processing unit 42 is realized, for example, by a microcomputer. Alternatively, the information processing unit 42 may be realized by a processor such as a CPU. The functions of the information processing unit 42 are realized, for example, by the microcomputer or processor that constitutes the information processing unit 42 executing a computer program stored in the storage unit 43.

[0049] The information processing unit 42 includes an acquisition unit 42a, an estimation unit 42b, a classification unit 42c, an extraction unit 42d, a determination unit 42e, and an output unit 42f.

[0050] The acquisition unit 42a is a processing unit that acquires moving images that show the pedestrian 1. Specifically, the acquisition unit 42a acquires moving images output (transmitted) from the information terminal 30 via the communication unit 41. In this embodiment, the acquisition unit 42a acquires moving images that include the pedestrian 1 as a subject.

[0051] The estimation unit 42b is a processing unit that estimates the skeleton of the pedestrian 1 appearing in the moving image based on the moving image acquired by the acquisition unit 42a. Specifically, the estimation unit 42b estimates (calculates) a skeleton model that indicates the skeleton of the pedestrian 1 in the moving image based on the acquired moving image.

[0052] 2 and 3 are diagrams illustrating a skeletal model of the pedestrian 1 according to the first embodiment. Specifically, Fig. 2 is a diagram illustrating a two-dimensional skeletal model indicating the skeleton of the pedestrian 1 estimated by the estimation unit 42b superimposed on an image showing the pedestrian 1. Fig. 3 is a diagram illustrating a three-dimensional skeletal model indicating the skeleton of the pedestrian 1 estimated by the estimation unit 42b. Specifically, Fig. 3 is a diagram illustrating the three-dimensional skeletal model at a certain time when the pedestrian 1 is walking toward the positive side of the Y-axis direction.

[0053] The skeletal model is a model generated by connecting multiple skeletal points, which are specific positions such as joints of the walker 1 in the moving image, with links (lines). Specifically, the skeletal model is coordinate data of the multiple skeletal points. For example, the estimation unit 42b estimates the positions of the joints of the walker 1, thereby estimating the positions (more specifically, coordinates) of multiple skeletal points indicating the estimated joint positions. More specifically, the estimation unit 42b performs image analysis or the like to estimate the positions of multiple predetermined skeletal points of the walker 1, including the neck skeletal points, elbow skeletal points, and wrist skeletal points, in each image included in the moving image. Furthermore, the estimation unit 42b, for example, connects the estimated multiple skeletal points with lines according to predetermined conditions. In this way, the estimation unit 42b estimates the skeletal model of the pedestrian 1.

[0054] The skeleton model may be estimated by any method, as long as an existing posture and skeleton estimation algorithm is used, for example.

[0055] Furthermore, the estimation unit 42b may estimate a two-dimensional skeletal model of the walker 1, or may estimate a three-dimensional skeletal model of the walker 1. That is, the estimation unit 42b may estimate two-dimensional coordinates of skeleton points of the walker 1, or may estimate three-dimensional coordinates of skeleton points of the walker 1. For example, the estimation unit 42b estimates a two-dimensional skeletal model of the walker 1 (i.e., coordinates of each skeleton point in a two-dimensional orthogonal coordinate system) based on the moving image acquired by the acquisition unit 42a, and estimates a three-dimensional skeletal model of the walker 1 (i.e., coordinates of each skeleton point in a three-dimensional orthogonal coordinate system) based on the estimated two-dimensional skeletal model using a trained model, which is a trained machine learning model.

[0056] The trained model is a classifier constructed in advance by machine learning using a two-dimensional skeletal model in which the three-dimensional coordinate data of each skeletal point (more specifically, the skeletal model) is known as training data and the three-dimensional coordinate data as training data. The trained model takes the two-dimensional skeletal model as input and outputs three-dimensional coordinate data corresponding to the two-dimensional skeletal model, i.e., a three-dimensional skeletal model. The trained model is stored in advance in, for example, the storage unit 43.

[0057] In this way, the estimation unit 42b may estimate a three-dimensional skeletal model of the pedestrian 1 in the moving image acquired by the acquisition unit 42a.

[0058] The trained model is, for example, a machine learning model using a neural network such as deep learning (for example, a convolutional neural network (CNN)), but may be another machine learning model.

[0059] Alternatively, the estimation unit 42b may estimate only skeleton points of the skeleton model that are used for extracting feature quantities by the extraction unit 42d (described later). In other words, the estimation unit 42b may estimate only a part of the skeleton model of the pedestrian 1.

[0060] The classification unit 42c is a processing unit that classifies a period in which the walker 1 is walking in the moving image into a plurality of periods based on the skeleton of the walker 1 estimated by the estimation unit 42b. Specifically, the classification unit 42c classifies a period from when the walker 1 starts walking to when the walker 1 stops walking in the moving image into a plurality of periods based on a skeleton model of the walker 1.

[0061] FIG. 4 is a diagram showing a specific example of a plurality of periods according to the first embodiment.

[0062] The multiple periods include, for example, an acceleration period in which the walker 1 starts walking and accelerates, a constant velocity period in which the walker 1 walks steadily and is the period following the acceleration period, and a deceleration period in which the walker 1 decelerates and stops and is the period following the constant velocity period. In other words, the classification unit 42c classifies the period in which the walker 1 is walking in the moving image into three periods: an acceleration period, a constant velocity period, and a deceleration period.

[0063] The acceleration period is a period in which the pedestrian 1 starts walking and accelerates. Specifically, the acceleration period is a period in which the pedestrian 1 starts walking from a stopped state, accelerates, and reaches a walking speed that is approximately constant.

[0064] The constant velocity period is a period following the acceleration period during which the walker 1 walks steadily. Specifically, the constant velocity period is a period during which the walking speed of the walker 1 is approximately constant, that is, a period during which the walker 1 walks steadily, until the walker 1 begins to decelerate in order to stop.

[0065] The deceleration period is the period following the constant velocity period during which the pedestrian 1 decelerates and stops. Specifically, the deceleration period is the period from when the pedestrian 1, who is walking at an approximately constant speed, starts to decelerate until he or she stops.

[0066] The classification unit 42c calculates the velocity (movement velocity) of the skeleton point indicating the heel as the velocity (movement velocity) of the walker 1, for example, based on the temporal change in the coordinates of the skeleton point indicating the heel in the skeleton model estimated by the estimation unit 42b. For example, the classification unit 42c calculates the velocity of the skeleton point indicating the heel based on the distance (movement distance) in the Y-axis direction (i.e., the direction of travel of the walker 1) from when the skeleton point of the heel of the right or left foot hits the ground, rises once, and then hits the ground again (one cycle time) and the time. For example, the classification unit 42c calculates the velocity of the walker 1 at each time by calculating (movement time) / (one cycle time). For example, based on the temporal change in the velocity of the walker 1 calculated in this way, the classification unit 42c classifies the period in which the walker 1 is walking in the moving image into three periods: an acceleration period, a constant velocity period, and a deceleration period.

[0067] For example, the classification unit 42c determines the period from when the walker 1 starts walking until the walking speed becomes equal to or greater than a first speed as an acceleration period. Furthermore, for example, the classification unit 42c determines the period from when the walking speed becomes equal to or greater than the first speed until it becomes equal to or less than a second speed as a constant-speed period. Furthermore, for example, the classification unit 42c determines the period from when the walking speed becomes equal to or less than the second speed until the walker 1 stops (i.e., until the walking speed becomes 0) as a deceleration period. The classification unit 42c may also determine the period after the walking speed becomes equal to or less than the second speed as a deceleration period.

[0068] The first speed and the second speed may be arbitrarily determined in advance and are not particularly limited. For example, the first speed is faster than the second speed. The first speed and the second speed may be the same speed.

[0069] Furthermore, the skeleton points used by the classification unit 42c for classifying periods may be arbitrarily determined, such as the skeleton points of the waist, and are not particularly limited.

[0070] Alternatively, the classification unit 42c may calculate the acceleration of the walker 1 (e.g., acceleration in the Y-axis direction) based on a predetermined skeletal point, such as the skeletal point of the heel, and separate the periods based on the calculated acceleration. For example, during the acceleration period, the acceleration of the walker 1 is equal to or greater than a first acceleration. Also, during the constant velocity period, the acceleration of the walker 1 is equal to or greater than a second acceleration but less than the first acceleration. Also, during the deceleration period, the acceleration of the walker 1 is less than the second acceleration. For example, the classification unit 42c may determine that the walker 1 has transitioned from the acceleration period to the constant velocity period when the acceleration of the walker 1 becomes less than the first acceleration after starting to walk, and may further determine that the walker 1 has transitioned from the constant velocity period to the deceleration period when the acceleration of the walker 1 becomes less than the second acceleration. The first acceleration is, for example, a positive number, and the second acceleration is, for example, a negative number. The first acceleration and the second acceleration may be determined arbitrarily and are not particularly limited.

[0071] The threshold information such as the first speed and the first acceleration is stored in advance in the storage unit 43, for example.

[0072] The classification unit 42c may also determine when the walker 1 starts and stops walking based on the posture of the walker 1 (i.e., the posture of the skeletal model) or the like.

[0073] Furthermore, the classification unit 42c may classify a period in which the walker 1 is walking in the moving image into two periods, such as two periods consisting of a first half and a second half of the walking of the walker 1. Alternatively, the classification unit 42c may classify a period in which the walker 1 is walking in the moving image into four or more periods, such as four periods consisting of an acceleration period, a first half of a constant velocity period, a second half of the constant velocity period, and a deceleration period.

[0074] The extraction unit 42d is a processing unit that extracts one or more feature amounts (also referred to as first feature amounts) related to the gait of the walker 1 for each of the multiple periods classified by the classification unit 42c, based on the skeleton of the walker 1 in the moving images estimated by the estimation unit 42b. In the present embodiment, the extraction unit 42d extracts one or more feature amounts during an acceleration period based on a skeletal model of the walker 1 during the acceleration period in the moving images. Furthermore, in the present embodiment, the extraction unit 42d extracts one or more feature amounts during a constant velocity period based on a skeletal model of the walker 1 during a constant velocity period in the moving images. Furthermore, in the present embodiment, the extraction unit 42d extracts one or more feature amounts during a deceleration period based on a skeletal model of the walker 1 during the deceleration period in the moving images.

[0075] The feature amount is a numerical value that quantitatively expresses the walking characteristics of the walker 1. Examples of types of feature amount include trunk movement, upper limb movement, lower limb movement, and footsteps.

[0076] Examples of trunk movement include up and down movement of the chest, amplitude of the chest, angle of the head swinging back and forth, left and right tilt of the trunk, and width of left and right swing of the trunk.

[0077] Examples of upper limb movements include the width of the wrist swinging back and forth, the angle of the shoulder tilting up and down, and the angle of the elbow swinging left and right.

[0078] Examples of lower limb movements include the range of vertical movement of the knee, the width of the knee opening during the swing phase, the width of the ankle opening during the swing phase, the range of motion of the hip joint, the range of motion of the knee joint, and the range of motion of the ankle joint.

[0079] Examples of foot movement include step length, stride width (stride length), step width, walking angle, heel height, and toe height.

[0080] Other examples of features include the number of steps (total number of steps), walking time, walking distance, walking speed, walking rate (number of steps per second), cadence (number of steps per minute), walking ratio (average stride length / cadence), average stride length / average stride width, and average stride length / average stride width.

[0081] The feature amount may be other than the above.

[0082] For example, the one or more first feature amounts extracted by the extraction unit 42d include any of a feature amount related to a joint angle, a feature amount related to a joint movement speed, a feature amount related to a joint movement acceleration, a feature amount related to a walking movement time, and a feature amount related to a distance based on a joint movement distance while the walker 1 is walking. In other words, the extraction unit 42d extracts any of a feature amount related to a joint angle, a feature amount related to a joint movement speed, a feature amount related to a joint movement acceleration, a feature amount related to a walking movement time, and a feature amount related to a distance based on a joint movement distance while the walker 1 is walking as a feature amount for at least any of the multiple periods, in order to evaluate the risk of a fall of the walker 1.

[0083] The feature amount related to the angle of the joint is, for example, the range of motion (movable width) of a specific joint such as a hip joint, ankle joint, or knee joint. Note that the specific joint may be determined arbitrarily and is not particularly limited.

[0084] Furthermore, the joints from which feature amounts related to the movement distance, movement speed, and movement acceleration are extracted may be any joints and are not particularly limited.

[0085] Furthermore, the joints (skeletal points) used when calculating the feature amount related to the travel time during walking may be determined arbitrarily and are not particularly limited.

[0086] Furthermore, for example, the one or more feature amounts extracted by the extraction unit 42d include at least any of the average toe speed, walking speed, double leg support time, forward / backward swing range of the trunk, maximum toe speed, stride length, waist fluctuation height, forward / backward movable range of the hip joint, heel strike angle, up and down movement of the chest, joint movable range of the ankle joint, knee fluctuation height, and joint angle swing range of the knee joint while the walker 1 is walking. In other words, the extraction unit 42d extracts at least any of the average toe speed, walking speed, double leg support time, forward / backward swing range of the trunk, maximum toe speed, stride length, waist fluctuation height, forward / backward movable range of the hip joint, heel strike angle, up and down movement of the chest, joint movable range of the ankle joint, knee fluctuation height, and joint angle swing range of the knee joint while the walker 1 is walking as feature amounts for at least any of the multiple periods.

[0087] The average velocity of the toes is the average velocity of the toes of the walker 1 in the direction of travel.

[0088] The walking speed is the speed at which the walker 1 moves in the direction of travel.

[0089] The double support time is the time during which both feet of the walker 1 are on the ground.

[0090] The forward / backward sway amplitude of the trunk is the sway amplitude in the forward / backward direction of the trunk (e.g., upper body) of the walker 1. When the walker 1 is viewed from the side (in the X-axis direction in this embodiment), the forward / backward sway amplitude of the trunk is, for example, the angle formed between a line segment that passes through the skeletal point of the waist and is parallel to the vertical direction, and a line segment that connects the skeletal point of the waist and the skeletal points of both shoulders.

[0091] The maximum toe speed is the maximum value of the moving speed of the walker 1 on the toes in the direction of travel.

[0092] The stride length is the distance in the direction of travel between the front and rear feet of the walker 1 while walking.

[0093] The fluctuation height of the waist (vertical movement of the waist) is the amplitude of the waist in the vertical direction (in this embodiment, the Z-axis direction) during one walking cycle.

[0094] The longitudinal range of motion of the hip joint is the range of motion of the hip joint in the longitudinal direction during one walking cycle. The longitudinal range of motion of the hip joint is, for example, the angle formed by a perpendicular line (a line normal to the ground on which the walker 1 walks) - the waist - the knee during one walking cycle. Specifically, the longitudinal range of motion of the hip joint is the angle formed by a perpendicular line passing through the waist and a line segment connecting the waist and the knee during one walking cycle when the walker 1 is viewed from the side.

[0095] The heel contact angle is the maximum angle between the sole of the foot and the ground when the heel touches the ground. For example, the heel contact angle (ground contact angle) is the angle between the toe and the ground, with the heel as the base point, when the heel touches the ground. In other words, the heel contact angle is the angle between the ground and the line segment connecting the heel (skeletal point of the heel) and the toe (skeletal point of the toe) when the heel touches the ground, when the pedestrian 1 is viewed from the side.

[0096] The up and down movement of the chest is the amplitude of the up and down movement of the chest during one walking cycle.

[0097] The range of motion of the ankle joint is the range of motion of the ankle joint in one walking cycle. The range of motion of the ankle joint angle is, for example, the angle formed by the knee-ankle-toe in one walking cycle. Specifically, the range of motion of the ankle joint angle is the angle formed by the line segment connecting the knee and ankle and the line segment connecting the toe and ankle in one walking cycle when the walker 1 is viewed from the side.

[0098] The height of knee fluctuation (vertical movement of the knee) is the amplitude of the knee in the vertical direction during one walking cycle.

[0099] The joint angle swing range of the knee joint is the swing range (angle) of the knee joint when the knee joint is bent and straightened in one walking cycle.

[0100] As a result of extensive research, including correlation analysis between a person and their fall history, the inventors of the present application have found that there is a high correlation between a person's fall history (fall risk) and the average toe speed, walking speed, double leg support time, forward and backward swing range of the trunk, maximum toe speed, stride length, waist fluctuation height (up and down movement of the waist), forward and backward movement range of the hip joint, heel contact angle (ground contact angle), chest up and down movement, joint motion range of the ankle joint, knee fluctuation height, and joint angle swing range of the knee joint.

[0101] For example, the slower the walking speed, the higher the risk of falling. In other words, the more a person has a history of falling, that is, the more likely they are to have fallen within a certain period of time since the time the risk of falling was determined, the slower their walking speed. Also, for example, the smaller the up and down movement of the knees, the higher the risk of falling. Also, for example, the smaller the ground contact angle, that is, the more flat-footed a person is, the higher the risk of falling. Also, for example, the longer the time spent supporting both legs, the poorer the balance and the higher the risk of falling. Also, for example, the smaller the forward and backward sway of the trunk, the higher the risk of falling. Also, for example, the smaller the up and down movement of the hips, the higher the risk of falling.

[0102] For these reasons, by using these feature amounts to determine the risk of falling, the risk of falling can be determined more accurately.

[0103] Furthermore, as a result of extensive research, the inventors of the present application have found that a fall risk can be more accurately determined by using walking speed (first two steps), hip joint range of motion (first step), and knee joint range of motion (first step) as feature quantities during the acceleration phase, walking speed (median), double leg support time (median), and trunk forward / backward sway width (median) as feature quantities during the constant velocity phase, and one walking cycle time (last step) and toe-off angle (last two steps) as feature quantities during the deceleration phase. Therefore, in this embodiment, the extraction unit 42d extracts walking speed (first two steps), hip joint range of motion (first step), and knee joint range of motion (first step) as feature quantities during the acceleration phase. Also, in this embodiment, the extraction unit 42d extracts walking speed (median), double leg support time (median), and trunk forward / backward sway width (median) as feature quantities during the constant velocity phase. In the present embodiment, the extraction unit 42d extracts one walking cycle time (the last step) and the toe-off angle (the last two steps) as feature amounts during the deceleration period.

[0104] The time period for one walking cycle (last step) is the time required for one step when the walker 1 stops.

[0105] As described above, for example, the extraction unit 42d extracts one or more feature quantities such that at least one feature quantity differs from each other among the multiple time periods. Of course, the extraction unit 42d may extract the same feature quantity for each of the multiple time periods. For example, when extracting the average toe speed as a feature quantity for each of the multiple time periods, the extraction unit 42d extracts the average toe speed as a feature quantity for each of the multiple time periods.

[0106] Furthermore, the extraction unit 42d may extract a feature amount related to the gait of the walker 1 for a period during which the walker 1 is walking, based on the estimated skeleton of the walker 1.

[0107] The determiner 42e is a processing unit that determines the fall risk of the pedestrian 1 based on one or more feature amounts for each of the multiple periods extracted by the extractor 42d. For example, the determiner 42e determines the fall risk corresponding to the acceleration period based on one or more feature amounts extracted during the acceleration period. The determiner 42e also determines the fall risk corresponding to the constant velocity period based on one or more feature amounts extracted during the constant velocity period. The determiner 42e also determines the fall risk corresponding to the deceleration period based on one or more feature amounts extracted during the deceleration period. The determiner 42e also makes a final determination (evaluation) of the fall risk of the pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period. For example, the determiner 42e finally evaluates the fall risk of the pedestrian 1 by adding up the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period (ensemble inference).

[0108] For example, the judgment unit 42e judges the risk of a pedestrian 1 falling using a machine learning model trained based on features related to a person's gait and a fall history indicating whether the person has fallen within a specified period of time.

[0109] The machine learning model is a classifier constructed in advance by machine learning using one or more feature amounts extracted from a video image of a walking person as training data and the person's fall history (whether or not the person has fallen within a predetermined period of time since the video image was captured) as training data. The machine learning model takes one or more feature amounts as input and outputs the person's risk of falling. The machine learning model is stored in advance in, for example, the storage unit 43.

[0110] The machine learning model is, for example, a machine learning model using a neural network such as deep learning (for example, a convolutional neural network (CNN)), but may be another machine learning model.

[0111] The trained model may be a statistical model or method (multiple regression model or logistic regression model) that uses the fall history as the objective variable and the feature values ​​as the explanatory variables.

[0112] The predetermined period may be set arbitrarily and is not particularly limited, for example, to the past year.

[0113] The output of the machine learning model may be, for example, whether there is a risk of falling (i.e., 0 or 1). Alternatively, the output of the machine learning model may be, for example, the degree of risk of falling (e.g., any numerical value between 0 and 1). In this case, for example, the higher the risk of falling, the more the machine learning model outputs a numerical value closer to 1, such as 0.8, and the lower the risk of falling, the more the machine learning model outputs a numerical value closer to 0, such as 0.2. The determination unit 42e may input, for example, one or more feature quantities into the machine learning model, and the result output from the machine learning model may be the determination result. The determination unit 42e may evaluate the risk of falling based on the numerical value output from the machine learning model. For example, the determination unit 42e may determine the risk of falling to be low if the numerical value output from the machine learning model is equal to or greater than 0 and less than 0.5, to be medium if the numerical value is equal to or greater than 0.5 and less than 0.8, and to be high if the numerical value is 0.8 or greater.

[0114] The thresholds and how to determine the risk of falling may be determined arbitrarily and are not particularly limited. For example, the determination unit 42e may determine that the risk of falling is low if the value output from the machine learning model is less than 0.5, and may determine that the risk of falling is high if the value is 0.5 or greater.

[0115] For example, such machine learning models are provided corresponding to each of a plurality of time periods. For example, the determination unit 42e determines the fall risk corresponding to each time period using a first machine learning model to which one or more feature values ​​extracted during the acceleration period are input, a second machine learning model to which one or more feature values ​​extracted during the constant velocity period are input, and a third machine learning model to which one or more feature values ​​extracted during the deceleration period are input.

[0116] As a result of thorough investigation, the inventors of the present application found that by using walking speed (first two steps), hip joint movable range (first step), and knee joint movable range (first step) as inputs (features), the AUC (Area Under Curve) of the first machine learning model becomes 0.78.

[0117] The AUC is an index indicating how accurately two values ​​can be separated. In the present embodiment, for example, the AUC is an index indicating how accurately the first machine learning model determines a person's risk of falling.

[0118] Furthermore, after careful consideration, the inventors of the present application discovered that the AUC of the second machine learning model becomes 0.80 by using walking speed (median), double leg support time (median), and torso forward / backward sway amplitude (median) as inputs (features).

[0119] Furthermore, after extensive research, the inventors of the present application discovered that the AUC of the third machine learning model becomes 0.58 by using one gait cycle time (the last step) and the toe-off angle (the last two steps) as inputs (features).

[0120] As described above, the inventors of the present application have discovered that the risk of falling can be accurately determined by classifying the walking period into multiple periods and determining the risk of falling using appropriate features for each period.

[0121] For example, when summing the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period, the determination unit 42e may use a majority vote or weight the output of each machine learning model. For example, the fall risk may be determined using only the output of a machine learning model whose AUC is equal to or greater than a predetermined value. In other words, in determining the fall risk, it is not necessary to use all of the multiple periods classified by the classification unit 42c; it is sufficient to use one of the multiple periods or two or more of the multiple periods. The predetermined value may be set arbitrarily and is not particularly limited. Furthermore, for example, weighting may be performed so that the higher the AUC, the larger the coefficient applied to the output of the machine learning model.

[0122] The machine learning model may also be a regression model in which the fall history (a person's fall history in the past year) is used as the dependent variable and the feature amount is included as an explanatory variable.

[0123] The determination unit 42e may also determine the risk of falling using a method such as a decision tree or a random forest.

[0124] The output unit 42f is a processing unit that outputs the determination result of the determination unit 42e. For example, the output unit 42f transmits information indicating the determination result to the information terminal 30 via the communication unit 41. As a result, the determination result is output (for example, displayed) on the information terminal 30.

[0125] The information processing unit 42 may further include a calculation unit 42g.

[0126] The calculation unit 42 g is a processing unit that calculates an auxiliary index for interpreting the fall risk of the pedestrian 1 .

[0127] The auxiliary index is, for example, a quantified factor of a fall risk and / or an index (information) for determining the content of a recommendation. Specifically, the auxiliary index is information indicating at least one of the physical ability and walking style of the walker 1.

[0128] For example, the calculation unit 42g calculates, as a feature amount, an angle (joint angle) between two links connected to a predetermined skeletal point of the walker 1, based on the skeletal model estimated by the estimation unit 42b. Alternatively, for example, the calculation unit 42g calculates, as a feature amount, a distance between the predetermined skeletal point and an extremity, a fluctuation range of the position of the predetermined skeletal point, or the like. For example, the calculation unit 42g calculates, as an auxiliary index, the physical ability of the walker 1, based on whether each calculated value is equal to or greater than a predetermined threshold value or whether it is within a predetermined range.

[0129] This allows, for example, a training plan necessary for maintaining or improving physical abilities based on physical abilities such as muscle strength to be provided to a pedestrian 1 who is at risk of falling (e.g., whose determined fall risk is higher than a predetermined threshold). The predetermined skeletal points, predetermined thresholds, and predetermined ranges may be determined arbitrarily. This information may be stored in advance in the storage unit 43.

[0130] In addition, information regarding the auxiliary indicators of the pedestrian 1 may be stored in advance in the memory unit 43, or the information may be received from the user by the reception unit 34 and acquired by the acquisition unit 42a from the information terminal 30.

[0131] The calculation unit 42g may also generate a training plan based on the calculation result. In this case, for example, the calculation unit 42g may create the training plan based on the determination result of the determination unit 42e in addition to the calculation result.

[0132] For example, the extraction unit 42d extracts one or more feature amounts (second feature amounts) related to the walking of the walker 1 for each of the multiple time periods based on the skeleton of the walker 1 estimated by the estimation unit 42b. In this case, for example, the calculation unit 42g calculates the physical ability of the walker 1 as an auxiliary index based on the one or more feature amounts (second feature amounts) extracted by the extraction unit 42d. For example, the calculation (estimation) of the physical ability is performed for each walking phase (starting walking (acceleration phase), steady walking (constant velocity phase), and ending walking (deceleration phase)), and feature amounts related to the physical ability (for example, five items) are designed and an estimation model is constructed.

[0133] The one or more extracted feature amounts for calculating the physical ability include any of a feature amount related to the angle of a joint, a feature amount related to the speed of movement of a joint, a feature amount related to the acceleration of movement of a joint, a feature amount related to the time spent walking, and a feature amount related to the distance based on the distance moved by a joint while the walker 1 is walking. In other words, in order to calculate the physical ability of the walker 1, the extractor 42d extracts any of a feature amount related to the angle of a joint, a feature amount related to the speed of movement of a joint, a feature amount related to the acceleration of movement of a joint, a feature amount related to the time spent walking, and a feature amount related to the distance based on the distance moved by a joint while the walker 1 is walking, as a feature amount for at least one of the multiple periods.

[0134] The feature amount extracted for assessing the fall risk of the pedestrian 1 and the feature amount extracted for calculating the physical ability of the pedestrian 1 may be the same or different.

[0135] Furthermore, for example, the extraction unit 42d extracts one or more feature amounts (third feature amounts) related to the walking of the walker 1 based on the skeleton of the walker 1 estimated by the estimation unit 42b for a period during which the walker 1 is walking. In this case, for example, the calculation unit 42g calculates the walking style of the walker 1 as an auxiliary index based on the one or more feature amounts (third feature amounts). For example, the calculation (estimation) of the walking style is performed using the feature amounts extracted from the walking of all walking phases.

[0136] The one or more extracted feature amounts for calculating the gait of the walker 1 include at least one of the walking speed, torso sway, hip joint movement range, knee joint movement range, double leg support time, stance time, heel height, heel contact angle, heel-off angle, hip up and down movement, chest up and down movement, step width, head up and down movement, hand up and down movement, elbow up and down movement, stride length, and step length while the walker 1 is walking. In other words, in order to calculate the gait of the walker 1, the extraction unit 42d extracts at least one of the walking speed, torso sway, hip joint movement range, knee joint movement range, double leg support time, stance time, heel height, heel contact angle, heel-off angle, hip up and down movement, chest up and down movement, step width, head up and down movement, hand up and down movement, elbow up and down movement, stride length, and step length while the walker 1 is walking.

[0137] The period during which the walker 1 is walking refers to the period from the start to the end of the walker 1. In this embodiment, the period during which the walker 1 is walking refers to a period including an acceleration period, a constant velocity period, and a deceleration period.

[0138] The forward and backward sway of the trunk is the angle (amplitude) of forward and backward sway of the trunk in one walking cycle.

[0139] The longitudinal range of motion of the knee joint is the range of motion of the knee joint in one walking cycle. The longitudinal range of motion of the knee angle is, for example, the angle formed by the waist, knee, and ankle in one walking cycle. Specifically, the range of motion of the knee joint angle is the angle formed by the line segment connecting the waist and knee and the line segment connecting the ankle and knee in one walking cycle when the walker 1 is viewed from the side.

[0140] The stance time is the time during one gait cycle that the foot is in contact with the ground.

[0141] Heel height is the maximum distance between the heel and the ground during the swing phase.

[0142] The heel-off angle is the angle between the plantar surface of the foot and the ground during the swing phase.

[0143] The step width is the distance between the heels of both feet in the lateral direction (in this embodiment, the X-axis direction) from when the heel of one foot touches the ground to when the heel of the other foot touches the ground.

[0144] The head swing is the swing width (angle) of the head in the forward and backward direction during one walking cycle.

[0145] The forward and backward swing of the hand is the swing width (angle) of the hand in the forward and backward direction during one walking cycle.

[0146] The forward and backward swing of the elbow is the swing width (angle) of the elbow in the forward and backward direction during one walking cycle.

[0147] The stride length is the distance from the contact point of the sole of the foot to the contact point of the sole of the same foot one walking cycle later.

[0148] For example, the calculation unit 42g determines the auxiliary indicators of the pedestrian 1 using a machine learning model trained based on the characteristics of the person's walking and the auxiliary indicators of the person (specifically, physical functions and walking style).

[0149] The machine learning model used to calculate the auxiliary index is a classifier constructed in advance by machine learning using one or more feature amounts extracted from a video image showing a walking person as training data and the auxiliary index of the person as training data. The machine learning model takes one or more feature amounts as input and outputs an auxiliary index. The machine learning model is stored in advance in the storage unit 43, for example.

[0150] The machine learning model is, for example, a machine learning model using a neural network such as deep learning (for example, a convolutional neural network (CNN)), but may be another machine learning model.

[0151] Furthermore, the type of machine learning model used by the determination unit 42e (for example, the method used by the machine learning model) and the type of machine learning model used by the calculation unit 42g may be the same or different.

[0152] For example, such machine learning models are provided corresponding to each of a plurality of time periods in calculating the physical ability of the walker 1. For example, the calculation unit 42g determines the physical ability corresponding to each time period using a first machine learning model to which one or more feature amounts extracted during the acceleration period are input, a second machine learning model to which one or more feature amounts extracted during the constant velocity period are input, and a third machine learning model to which one or more feature amounts extracted during the deceleration period are input. For example, the calculation unit 42g finally evaluates the physical ability of the walker 1 by adding up the physical ability corresponding to the acceleration period, the physical ability corresponding to the constant velocity period, and the physical ability corresponding to the deceleration period.

[0153] Furthermore, for example, when calculating the walking style of the walker 1, such a machine learning model is set up corresponding to the period during which the walker 1 is walking (for example, the entire walking period).

[0154] The output of the machine learning model may be, for example, a high or low evaluation for each item, such as muscle strength and leg movement, included in the auxiliary index described below, in other words, whether the evaluation is sufficient or not compared to the average person (e.g., 1 or 0). Alternatively, the output of the machine learning model may be, for example, a multi-level output (e.g., any numerical value between 0 and 1). In this case, for example, the higher the ability (or the more appropriately the performance), the machine learning model outputs a numerical value closer to 1, such as 0.8, and the lower the ability (or the less appropriately the performance), the more closely the output is a numerical value closer to 0, such as 0.2. The calculation unit 42g, for example, inputs one or more feature quantities into the machine learning model, and the result output from the machine learning model is used as the calculation result. The calculation unit 42g may evaluate the auxiliary index based on the numerical value output from the machine learning model. For example, if the auxiliary indicator is muscle strength among physical functions, the calculation unit 42g may determine that muscle strength is low if the numerical value output from the machine learning model is greater than or equal to 0 and less than 0.5, that muscle strength is medium if the numerical value is greater than or equal to 0.5 and less than 0.8, and that muscle strength is high if the numerical value is 0.8 or greater.

[0155] The method for determining these thresholds and auxiliary indicators may be determined arbitrarily and is not particularly limited. For example, the calculation unit 42g may determine that muscle strength is low if the numerical value output from the machine learning model is less than 0.5, and may determine that muscle strength is high if the numerical value is 0.5 or greater.

[0156] Furthermore, for example, when the auxiliary index includes multiple items, the calculation unit 42g may calculate an overall evaluation of the physical ability and walking style from the sum of the multiple items.

[0157] For example, when adding up the physical ability corresponding to the acceleration period, the physical ability corresponding to the constant velocity period, and the physical ability corresponding to the deceleration period, the calculation unit 42g may use a majority vote or weight the output of each machine learning model. For example, physical ability may be determined using only the output of a machine learning model whose AUC is equal to or greater than a predetermined value. In other words, in determining physical ability, it is not necessary to use all of the multiple periods classified by the classification unit 42c; it is sufficient to use one of the multiple periods or two or more of the multiple periods. The predetermined value may be set arbitrarily and is not particularly limited. Furthermore, for example, weighting may be performed so that the higher the AUC, the larger the coefficient applied to the output of the machine learning model.

[0158] The machine learning model may also be a regression model that uses physical ability or walking style as a dependent variable and includes feature values ​​as explanatory variables.

[0159] Furthermore, the calculation unit 42g may determine physical ability or walking style using a method such as a decision tree or a random forest.

[0160] Furthermore, the feature values ​​used to evaluate the risk of falling and the feature values ​​used to calculate the auxiliary index may be the same or different. Furthermore, the feature values ​​used to evaluate the physical ability and the feature values ​​used to calculate the gait may be the same or different. Furthermore, for example, if the physical ability and the gait each include multiple items, the one or more feature values ​​used to evaluate each item may be the same or different.

[0161] In addition, the output unit 42f may output the calculation results of the calculation unit 42g (specifically, auxiliary information indicating the physical functions and walking style of the walker 1, etc.) and / or a training plan, etc., along with the assessment result of the fall risk of the walker 1.

[0162] Fig. 5 is a diagram showing a specific example of a determination result displayed by the display unit 35 according to embodiment 1. Specifically, Fig. 5 is a diagram showing a specific example of an image showing the determination result of the determination unit 42e, which is output by the output unit 42f and displayed on the display unit 35.

[0163] The control unit 32 causes the display unit 35 to display the acquired determination result (specifically, an image showing the determination result), for example.

[0164] For example, the image shown in FIG. 5 includes information indicating the result of a fall risk assessment for pedestrian 1 and information indicating physical functions related to the fall of pedestrian 1. In the example shown in FIG. 5, the image showing the assessment result includes information such as "The fall risk is medium" as information indicating the result of the fall risk assessment for pedestrian 1. Alternatively, the image showing the assessment result may include information indicating the assessment result in which the degree (level) of the fall risk is expressed in stages by filling in stars or the like, such as "Fall risk: ★★☆." Furthermore, the image includes information indicating the physical functions related to the fall of pedestrian 1, such as evaluation results of physical functions such as "left-right balance" and "muscle strength," and information encouraging pedestrian 1 to train, such as "Your muscle strength has weakened slightly, so we recommend that you train." The image may also include an image showing a training plan.

[0165] FIG. 6 is a diagram showing a second example of an image displayed by the display unit 35 according to the first embodiment.

[0166] The image shown in FIG. 6 includes, for example, display information 100, 101, and 102.

[0167] The display information 100 is an image showing the risk of the pedestrian 1 falling, and shows the determination result of the determination unit 42e.

[0168] The display information 101 is an image showing the calculation results of the walking style of the walker 1, and shows the calculation results of the walking style of the walker 1 calculated by the calculation unit 42g. The display information 101 is an example of auxiliary information. The display information 101 includes, for example, the calculation results of four items (foot movement, trunk, arm swing, and pelvis) that indicate the walking style of the walker 1. The display information 101 includes, for example, the calculation results of these items and an overall result that combines these calculation results. The overall result is displayed from these calculation results, using, for example, an image of an animal's walking style as a metaphor for a characteristic walking style, such as "Your walking style type: Penguin type, level 2."

[0169] The display information 102 is an image showing the calculation results of the physical ability of the walker 1, and shows the calculation results of the physical ability of the walker 1 by the calculation unit 42g. The display information 102 is another example of auxiliary information. The display information 102 includes, for example, calculation results of five items (muscle strength, trunk function, work ability, balance ability, and reflex ability) that indicate the physical ability of the walker 1. The display information 102 includes, for example, the calculation results of these items and an overall result that combines these calculation results. The overall result is displayed, for example, as "physical score 65 points."

[0170] For example, when a user selects "View walking advice" or "View physical ability advice" using an operating device such as a mouse, keyboard, and / or touch panel, advice for these (e.g., recommendations showing recommended exercises) may be displayed.

[0171] 7 is a diagram for explaining specific examples of physical abilities. Specifically, FIG. 7 shows specific examples of physical ability items calculated by the calculation unit 42g, the measurement method (measurement item) of each physical ability item, and the measurement content of each item.

[0172] The calculated physical abilities include, for example, muscle strength, reflexes, balance, trunk function, and work ability. These physical abilities are calculated using, for example, a two-step test, a walking reaction time, standing on one leg with eyes closed, a functional reach test (FRT), and a timed up and go test (TUG test). The physical abilities of the walker 1 may be calculated by performing these measurements.

[0173] In the present embodiment, the calculation unit 42g calculates the physical ability using the feature amount.

[0174] 8 is a diagram showing specific examples of feature values ​​for calculating physical ability. Specifically, FIG. 8 shows specific examples of physical ability items calculated by the calculation unit 42g and specific examples of feature values ​​used to calculate the physical ability for each item for each of a plurality of time periods.

[0175] For example, the calculation unit 42g calculates the physical ability for each item corresponding to each of the multiple periods shown in Figure 8 using each feature shown for each of the multiple periods, and calculates the physical ability by adding up the calculation results for the multiple periods.

[0176] Fig. 9 is a diagram showing specific examples of feature amounts for calculating a walking style. Specifically, Fig. 9 shows specific examples of walking style items calculated by the calculation unit 42g and specific examples of feature amounts used to calculate the walking style for each item.

[0177] The calculated walking style items include, for example, foot movement (rocker function), arm swing, pelvis (tilt of head and trunk), and trunk (sway, balance). For example, the calculation unit 42g calculates each walking style item using the feature amounts shown in FIG. 9. For example, the calculation unit 42g calculates how appropriately the foot movement is performed. Also, for example, the calculation unit 42g calculates how appropriately the arms are swinging while walking.

[0178] The above-described physical ability and gait items and the feature quantities used to calculate them are merely examples, and may be determined arbitrarily. The number of physical ability and gait items to be calculated may be one or more.

[0179] Each of the processing units, namely, the acquisition unit 42 a, the estimation unit 42 b, the classification unit 42 c, the extraction unit 42 d, the determination unit 42 e, the output unit 42 f, and the calculation unit 42 g, is realized by, for example, a memory that stores a control program and a processor that executes the control program. The processor of each processing unit may be realized by a single memory and processor, or may be realized by separate memories and processors.

[0180] The storage unit 43 is a storage device that stores information such as the moving images (moving image data) acquired by the acquisition unit 42a, the trained model, the database, and the various thresholds. The storage unit 43 is realized by, for example, a semiconductor memory or an HDD.

[0181] As described above, the fall risk assessment device 40 assesses (estimates) the fall risk of the pedestrian 1 (e.g., how susceptible the pedestrian 1 is to falling) from the feature quantities (gait feature quantities) of the pedestrian 1. Furthermore, for example, the fall risk assessment device 40 calculates (estimates) the pedestrian 1's walking style (e.g., the state and / or tendency of the pedestrian's walking). Furthermore, for example, the fall risk assessment device 40 calculates the pedestrian 1's physical ability (e.g., the level of ability related to the fall risk that is causing the pedestrian 1 to walk in the current manner). By assessing, or in other words, quantifying, information related to the pedestrian 1's fall risk, it is possible to estimate the likelihood of the pedestrian 1 falling, and also to estimate information necessary for considering the cause and future countermeasures. For example, the gait feature quantities are used to estimate physical abilities (e.g., the above-mentioned five items) that are thought to be related to falls and the pedestrian's walking style, and recommendations are output based on the results.

[0182] For example, in the estimation of physical ability, it is estimated which of the above five items of physical ability of the walker 1 is high and which is low. Here, for example, a recommendation is output for an item where the ability is estimated to be low.

[0183] Furthermore, for example, in estimating the walking style, the posture and movement of the walker 1 are estimated. Based on the estimation result, a recommendation regarding the walking style is output.

[0184] For example, when estimating physical ability based on walking, which is similar to the true value of physical fitness measurements, the feature values ​​for each walking section are important, as differences in characteristics between individuals are more easily detected during the initial walking period. On the other hand, when estimating walking style (gait), evaluation of posture throughout the entire walk is important.

[0185] As described above, optimal recommendations are output based on the results of the walking style assessment and the estimated physical ability. For example, the recommendation may say, "People with ____ walking style have this tendency, so be careful of ____ when walking," or "Your physical ability ____ has declined, so you have a tendency toward ____, so this type of exercise is recommended."

[0186] [Processing Procedure] Next, the processing procedure of the display system 10 will be described.

[0187] FIG. 10 is a sequence diagram showing the processing procedure of the display system 10 according to the first embodiment.

[0188] First, the instructing unit 36 ​​instructs the walker 1 to start walking (S201). For example, when the receiving unit 34 receives an instruction from the user to make the walker 1 start walking, the instructing unit 36 ​​gives an instruction such as "Please start walking."

[0189] When the receiving unit 34 receives an instruction, the control unit 32 may acquire a moving image captured by the camera 20 and identify the pedestrian 1 in the acquired moving image. To identify the pedestrian 1 in the moving image, a known image analysis technique such as pattern matching is used.

[0190] Next, the camera 20 captures an image of the walking pedestrian 1 as a subject, thereby generating a moving image including the walking pedestrian 1 as a subject (S202).

[0191] Next, the control unit 32 outputs the moving image generated by the camera 20 to the fall risk assessment device 40 via the communication unit 31 (S203). At this time, the control unit 32 may conceal the moving image and transmit it to the fall risk assessment device 40. This protects the privacy data of the pedestrian 1.

[0192] Through the processing of the information terminal 30 described above, the fall risk assessment device 40 acquires a moving image and assesses the fall risk of the pedestrian 1 based on the acquired moving image (S100). The fall risk assessment device 40 outputs the assessment result to the information terminal 30.

[0193] Next, the control unit 32 acquires the determination result via the communication unit 31 (S204).

[0194] Next, the display unit 35 displays the determination result acquired by the control unit 32 (S205).

[0195] Fig. 11 is a flowchart showing the processing procedure of the fall risk assessment device 40 according to Embodiment 1. Specifically, Fig. 11 is a flowchart showing the details of the processing of step S100.

[0196] First, the acquisition unit 42a acquires a moving image showing the pedestrian 1 (S110). Specifically, the acquisition unit 42a acquires a moving image showing the pedestrian 1 from when the pedestrian 1 starts walking until when the pedestrian stops walking.

[0197] Next, the estimation unit 42b estimates the skeleton of the pedestrian 1 appearing in the moving image based on the moving image acquired by the acquisition unit 42a (S120). Specifically, the estimation unit 42b calculates a skeleton model indicating the skeleton of the pedestrian 1 appearing in the moving image based on the moving image acquired by the acquisition unit 42a.

[0198] Next, the classification unit 42c classifies the period in which the walker 1 is walking in the moving image acquired by the acquisition unit 42a into multiple periods based on the skeleton estimated by the estimation unit 42b (S130). For example, the classification unit 42c classifies the period in which the walker 1 is walking in the moving image acquired by the acquisition unit 42a into three periods: an acceleration period, a constant velocity period, and a deceleration period based on the skeleton model calculated by the estimation unit 42b. For example, the classification unit 42c calculates the speed (movement speed) of the skeleton point indicating the heel as the speed (movement speed) of the walker 1 based on the time change in the coordinate of the skeleton point indicating the heel in the skeleton model. For example, the classification unit 42c classifies the period in which the walker 1 is walking into an acceleration period, a constant velocity period, and a deceleration period based on the speed of the walker 1.

[0199] Next, the extraction unit 42d extracts one or more feature quantities related to the gait of the walker 1 for each of the multiple periods classified by the classification unit 42c, based on the skeleton of the walker 1 estimated by the estimation unit 42b (S140). For example, the extraction unit 42d extracts one or more feature quantities for each of the acceleration period, constant velocity period, and deceleration period. For example, the extraction unit 42d extracts a first feature quantity, a second feature quantity, and a third feature quantity for the acceleration period, a fourth feature quantity, a fifth feature quantity, and a sixth feature quantity for the constant velocity period, and a seventh feature quantity and an eighth feature quantity for the deceleration period. For example, the feature quantities are different types from each other. Note that some of the feature quantities extracted for each period may be the same type, such as the first feature quantity and the third feature quantity being the same type.

[0200] Next, the determiner 42e determines the fall risk of the pedestrian 1 based on one or more feature amounts for each of the multiple periods extracted by the extractor 42d (S150). For example, the determiner 42e determines the fall risk corresponding to the acceleration period based on one or more feature amounts extracted during the acceleration period. The determiner 42e also determines the fall risk corresponding to the constant velocity period based on one or more feature amounts extracted during the constant velocity period. The determiner 42e also determines the fall risk corresponding to the deceleration period based on one or more feature amounts extracted during the deceleration period. The determiner 42e also finally evaluates (determines) the fall risk of the pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period.

[0201] Next, the output unit 42f outputs the determination result of the determination unit 42e to the information terminal 30 (S160). As a result, the determination result of the determination unit 42e is displayed on the display unit 35 of the information terminal 30.

[0202] (Embodiment 2) Next, a description will be given of a fall risk assessment device according to embodiment 2. Note that in the description of embodiment 2, some or all of the description that overlaps with the description of embodiment 1 may be omitted or simplified.

[0203] FIG. 12 is a block diagram showing the functional configuration of a fall risk determination device 400 according to the second embodiment.

[0204] The fall risk assessment device 400 is, for example, a computer that the display system 10 shown in Fig. 1 includes in place of the fall risk assessment device 40. The fall risk assessment device 400 assesses (evaluates) the fall risk of the pedestrian 1 using the information terminal 30, for example.

[0205] In this embodiment, the fall risk assessment device 400 assesses the fall risk without using moving images that include the pedestrian 1 as a subject. In other words, the display system equipped with the fall risk assessment device 400 is a system that assesses the fall risk without using moving images that include the pedestrian 1 as a subject.

[0206] For example, in the first embodiment, the acquisition unit 42 a acquires a moving image of the pedestrian 1 captured by the camera 20 , and the estimation unit 42 b estimates the skeleton of the pedestrian 1 .

[0207] Here, in addition to the method using the camera 20, there is also a method using radio waves to estimate the skeleton of the pedestrian 1. For example, the skeleton (posture) of the pedestrian 1 can be estimated based on the propagation and reflection states of radio waves. For example, the propagation and reflection states of radio waves can be acquired by transmitting radio waves to the pedestrian 1 and receiving reflected waves of the transmitted radio waves.

[0208] For example, the skeleton of the pedestrian 1 can be estimated using multipath caused by radio wave reflection from Wi-Fi (registered trademark). For example, a trained machine learning model that has previously learned the correlation between multipath and walking behavior is generated, and the machine learning model is pre-stored in the fall risk assessment device 400. Then, the pedestrian 1 is made to walk in a Wi-Fi environment, multipath is detected, and information indicating the detected multipath is acquired by the fall risk assessment device 400. Based on the information indicating the changes in the detected multipath, the fall risk assessment device 400 estimates the skeleton (posture) of the pedestrian 1 using the machine learning model. For example, a Wi-Fi access point is installed in the location where the pedestrian 1 walks, and multipath is detected using multiple receivers. Based on the information detected in this way, the fall risk assessment device 400 estimates the skeleton of the pedestrian 1.

[0209] In this way, the skeleton of the pedestrian 1 may be estimated by detecting the skeleton of the pedestrian 1 using other means, without using the image captured by the camera 20 .

[0210] The strength of the radio waves used to estimate the skeleton of the pedestrian 1, the wavelength of the radio waves, the communication standard used to detect multipath, etc. may be determined arbitrarily and are not particularly limited.

[0211] The user walks in such a place, for example. The information (motion information) generated thereby is transmitted to the fall risk assessment device 400. The fall risk assessment device 400 assesses the fall risk of the pedestrian 1 based on the received motion information, and transmits the assessment result to the information terminal 30. The information terminal 30 displays the received assessment result.

[0212] In this embodiment, the display system may include a Wi-Fi access point and multiple receivers. In addition, in this embodiment, the information terminal 30 used by the pedestrian 1 may not include the camera 20. For example, in this embodiment, the information terminal 30 may not capture an image of the pedestrian 1 using the camera 20. The number of multiple receivers may be determined arbitrarily and is not particularly limited. In addition, the layout of the access point and multiple receivers may be set arbitrarily.

[0213] The fall risk assessment device 400 includes a communication unit 41 , an information processing unit 420 , and a memory unit 43 .

[0214] The communication unit 41 is a communication interface that communicates with the information terminal 30. Specifically, the communication unit 41 allows the fall risk assessment device 400 to communicate with the information terminal 30 via a network 5 such as the Internet. The communication unit 41 is realized, for example, by a wireless communication circuit for wirelessly communicating with the information terminal 30.

[0215] The information processing unit 420 is a processing unit that performs various types of information processing in the fall risk assessment device 400. The information processing unit 420 is realized, for example, by a microcomputer. Alternatively, the information processing unit 420 may be realized by a processor such as a CPU. The functions of the information processing unit 420 are realized, for example, by the microcomputer or processor that constitutes the information processing unit 420 executing a computer program stored in the storage unit 43.

[0216] The storage unit 43 is a storage device that stores various types of information. In this embodiment, for example, the storage unit 43 stores a trained machine learning model that has previously learned the correlation between multipaths and walking behavior. Of course, the storage unit 43 may also store information such as the trained model, database, and various thresholds described in the first embodiment.

[0217] The information processing unit 420 includes a walking movement acquisition unit 420a, a classification unit 420c, an extraction unit 420d, a determination unit 420e, and an output unit 420f.

[0218] The walking motion acquisition unit 420a is a processing unit that acquires various types of information, similar to the acquisition unit 42a. The walking motion acquisition unit 420a acquires the walking motion of the walker 1. Specifically, the walking motion acquisition unit 420a acquires motion information indicating the walking motion of the walker 1. The motion information is, for example, information indicating the above-mentioned multipath. For example, the walking motion acquisition unit 420a acquires, as motion information, information indicating the radio waves received by the above-mentioned multiple receivers (not shown) via the communication unit 41.

[0219] Similar to the classifying unit 42c, the classifying unit 420c is a processing unit that classifies into a plurality of periods the period during which the walker 1 is walking. The classifying unit 420c classifies into a plurality of periods the period from when the walker 1 starts walking until when the walker 1 stops walking, based on the movement information acquired by the walking movement acquiring unit 420a.

[0220] The classification unit 420c classifies the period in which the walker 1 is walking into three periods: an acceleration period, a constant velocity period, and a deceleration period, for example, based on the movement of the skeleton (skeleton model) of the walking motion of the walker 1 estimated using a machine learning model stored in the memory unit 43, and calculates the speed (movement speed) of the skeleton point indicating the heel as the speed (movement speed) of the walker 1 from the change over time in the coordinates of the skeleton point indicating the heel in the skeletal model, similar to the processing performed by the classification unit 42c.

[0221] Similar to the extraction unit 42d, the extraction unit 420d is a processing unit that extracts, for each of a plurality of time periods, one or more feature amounts (first feature amounts) related to the walking of the walker 1. The extraction unit 420d extracts, for each of the plurality of time periods classified by the classification unit 420c, one or more first feature amounts related to the walking of the walker 1 based on the skeletal movement of the walking motion of the walker 1 estimated by the machine learning model using the motion information.

[0222] Similar to the determination unit 42e, the determination unit 420e is a processing unit that determines the fall risk of the pedestrian 1 based on one or more feature amounts (for the first feature). The determination unit 420e determines the fall risk of the pedestrian 1 based on one or more feature amounts for each of the multiple periods extracted by the extraction unit 420d. For example, the determination unit 420e determines the fall risk corresponding to the acceleration period based on one or more feature amounts extracted during the acceleration period. The determination unit 420e also determines the fall risk corresponding to the constant velocity period based on one or more feature amounts extracted during the constant velocity period. The determination unit 420e also determines the fall risk corresponding to the deceleration period based on one or more feature amounts extracted during the deceleration period. The determination unit 420e also finally determines (evaluates) the fall risk of the pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period. For example, the determination unit 420e finally evaluates the fall risk of the pedestrian 1 by adding up the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period.

[0223] The output unit 420f is a processing unit that outputs the determination result, similar to the output unit 42f. The output unit 420f outputs the determination result of the determination unit 420e.

[0224] [Processing Procedure] Next, the processing procedure of the fall risk determination device 400 will be described.

[0225] FIG. 13 is a flowchart showing the processing procedure of the fall risk determination device 400 according to the second embodiment.

[0226] First, the walking motion acquisition unit 420a acquires the walking motion of the walker 1. Specifically, the walking motion acquisition unit 420a acquires motion information indicating the walking motion of the walker 1 from when the walker 1 starts walking until when the walker 1 stops walking (S310).

[0227] Next, the classification unit 420c classifies the period during which the walker 1 is walking into a plurality of periods (S320). For example, the classification unit 420c classifies the period during which the walker 1 is walking into three periods: an acceleration period, a constant velocity period, and a deceleration period, based on the movement information acquired by the walking movement acquisition unit 420a. For example, the classification unit 420c calculates the speed (movement speed) of the skeleton point indicating the heel as the speed (movement speed) of the walker 1 based on the change over time in the coordinates of the skeleton point indicating the heel in the skeleton model. For example, the classification unit 420c classifies the period during which the walker 1 is walking into an acceleration period, a constant velocity period, and a deceleration period, based on the speed of the walker 1.

[0228] Next, the extraction unit 420d extracts one or more feature amounts related to the walking of the walker 1 based on the skeletal movement of the walking motion of the walker 1 for each of the multiple periods classified by the classification unit 420c (S330). For example, the extraction unit 420d extracts one or more feature amounts for each of the acceleration period, constant velocity period, and deceleration period. For example, the extraction unit 420d extracts a first feature amount, a second feature amount, and a third feature amount for the acceleration period, a fourth feature amount, a fifth feature amount, and a sixth feature amount for the constant velocity period, and a seventh feature amount and an eighth feature amount for the deceleration period. For example, the feature amounts are different types from each other. Note that some of the feature amounts extracted for each period may be the same type, such as the first feature amount and the third feature amount being the same type.

[0229] Next, the determination unit 420e determines the fall risk of the pedestrian 1 based on one or more feature amounts for each of the multiple periods extracted by the extraction unit 420d (S340). For example, the determination unit 420e determines the fall risk corresponding to the acceleration period based on one or more feature amounts extracted during the acceleration period. The determination unit 420e also determines the fall risk corresponding to the constant velocity period based on one or more feature amounts extracted during the constant velocity period. The determination unit 420e also determines the fall risk corresponding to the deceleration period based on one or more feature amounts extracted during the deceleration period. The determination unit 420e also finally evaluates (determines) the fall risk of the pedestrian 1 based on the fall risk corresponding to the acceleration period, the fall risk corresponding to the constant velocity period, and the fall risk corresponding to the deceleration period.

[0230] Next, the output unit 420f outputs the determination result of the determination unit 420e (S350). For example, the output unit 420f outputs the determination result of the determination unit 420e to the information terminal 30. As a result, the determination result of the determination unit 420e is displayed on the display unit 35 of the information terminal 30.

[0231] [Effects, etc.] Hereinafter, examples of techniques that can be obtained from the disclosure of this specification will be given, and effects, etc. that can be obtained from the exemplified techniques will be described.

[0232] Technique 1 is a fall risk assessment device 40 including an acquisition unit 42a that acquires a moving image showing a pedestrian 1, an estimation unit 42b that estimates the skeleton of the pedestrian 1 that appears in the moving image based on the moving image, a classification unit 42c that classifies periods in which the pedestrian 1 is walking in the moving image into multiple periods based on the skeleton, an extraction unit 42d that extracts one or more first feature amounts related to the walking of the pedestrian 1 for each of the multiple periods based on the skeleton, a assessment unit 42e that assesses the risk of the pedestrian 1 falling based on the one or more first feature amounts, and an output unit 42f that outputs the assessment result of the assessment unit 42e.

[0233] According to this, the fall risk assessment device 40 extracts one or more feature amounts for each of a plurality of periods (each walking phase) based on the estimated skeleton (skeleton model) of the pedestrian 1. The inventors of the present application have discovered that the fall risk of the pedestrian 1 can be accurately, i.e., accurately assessed by extracting one or more feature amounts for each period, such as an acceleration period, a constant velocity period, and a deceleration period, assessing the fall risk, and, for example, summing the fall risks for each period to assess (calculate) the fall risk of the pedestrian 1. Therefore, the fall risk assessment device 40 can accurately assess the fall risk of the pedestrian 1. Furthermore, the fall risk assessment device 40 can calculate the fall risk of the pedestrian 1 from the walking state of the pedestrian 1, so the fall risk of the pedestrian 1 can be calculated through simple processing.

[0234] Technology 2 is the fall risk determination device 40 described in Technology 1, in which the determination unit 42e determines the fall risk of the pedestrian 1 using a machine learning model trained based on features related to a person's gait and a fall history indicating whether the person has fallen within a specified period of time.

[0235] This allows the risk of the pedestrian 1 falling to be determined easily and accurately.

[0236] Technology 3 is the fall risk assessment device 40 described in Technology 1 or 2, in which the one or more first feature amounts include any of a feature amount related to the angle of the joint, a feature amount related to the movement speed of the joint, a feature amount related to the movement acceleration of the joint, a feature amount related to the movement time during walking, and a feature amount related to the movement distance of the joint while the walker 1 is walking.

[0237] The inventors of the present application have found that using these feature amounts to determine the fall risk of the pedestrian 1 makes it possible to accurately determine the fall risk of the pedestrian 1. Therefore, this makes it possible to accurately determine the fall risk of the pedestrian 1. Furthermore, for example, because the fall risk of the pedestrian 1 is determined without extracting feature amounts that do not need to be used to simply and accurately determine the fall risk, it is possible to accurately determine the fall risk while reducing the amount of processing required to determine the fall risk.

[0238] Technique 4 is a fall risk assessment device 40 according to any one of techniques 1 to 3, wherein the one or more first feature amounts include at least one of the average toe speed, walking speed, both leg support time, forward and backward swing range of the trunk, maximum toe speed, stride length, waist fluctuation height, forward and backward swing range of the hip joint, heel contact angle, chest up and down movement, joint motion range of the ankle joint, knee fluctuation height, and joint angle swing range of the knee joint while the walker 1 is walking.

[0239] The inventors of the present application have found that by using these feature amounts in particular to determine the fall risk of the pedestrian 1, it is possible to more accurately determine the fall risk of the pedestrian 1. Therefore, this allows for more accurately determining the fall risk of the pedestrian 1. Furthermore, for example, because the fall risk of the pedestrian 1 is determined without extracting feature amounts that do not need to be used to simply and accurately determine the fall risk, it is possible to reduce the amount of processing required to determine the fall risk and to more accurately determine the fall risk.

[0240] Technique 5 is a fall risk assessment device 40 according to any one of techniques 1 to 4, in which the multiple periods include an acceleration period in which the walker 1 starts walking and accelerates, a constant velocity period in which the walker 1 walks steadily and is the period following the acceleration period, and a deceleration period in which the walker 1 decelerates and stops and is the period following the constant velocity period.

[0241] The acceleration period, constant velocity period, and deceleration period can be easily divided by the speed and / or acceleration of the pedestrian 1. Therefore, by dividing the multiple periods into acceleration periods, constant velocity periods, and deceleration periods, each period can be easily divided.

[0242] Technology 6 is a fall risk assessment device 40 described in any one of Technologies 1 to 5, further comprising a calculation unit 42g that calculates an auxiliary index for interpreting the fall risk of pedestrian 1, and an output unit 42f that further outputs auxiliary information indicating the auxiliary index.

[0243] In this way, auxiliary information is output together with the determination result, so that, for example, a pedestrian 1 who has been determined to have a high risk of falling can easily understand what measures to take.

[0244] Technology 7 is the fall risk assessment device 40 described in Technology 6, in which the extraction unit 42d extracts one or more second feature amounts related to the walking of the walker 1 based on the skeleton for each of a plurality of time periods, and the calculation unit 42g calculates the physical ability of the walker 1 as an auxiliary index based on the one or more second feature amounts, and the one or more second feature amounts include any of a feature amount related to the angle of the joints, a feature amount related to the movement speed of the joints, a feature amount related to the acceleration of the movement of the joints, a feature amount related to the movement time during walking, and a feature amount related to the movement distance of the joints while the walker 1 is walking.

[0245] The inventors of the present application have found that by using these feature quantities to calculate the physical ability of the walker 1, the physical ability of the walker 1 can be accurately determined. Therefore, this allows the physical ability of the walker 1 to be accurately determined. Furthermore, for example, the physical ability of the walker 1 is determined without extracting feature quantities that are not necessary for easily and accurately determining the physical ability, so that the physical ability can be accurately determined while reducing the amount of processing required for determining the physical ability. Furthermore, for example, by outputting the physical ability of the walker 1 that is lower than that of a typical subject and / or the physical ability of the walker 1 that is declining as auxiliary information, it can be easily understood what measures should be taken by the pedestrian 1 who has been determined to have a high risk of falling.

[0246] Technology 8 is the fall risk assessment device 40 described in Technology 6 or 7, in which the extraction unit 42d extracts one or more third feature amounts related to the walking of the walker 1 based on the skeleton for a period during which the walker 1 is walking, and the calculation unit 42g calculates the walking style of the walker 1 as an auxiliary indicator based on the one or more third feature amounts, and the one or more third feature amounts include at least any of walking speed, forward and backward sway of the trunk, forward and backward movement range of the hip joints, forward and backward movement range of the knee joints, double leg support time, stance time, heel height, heel contact angle, heel off angle, up and down movement of the waist, up and down movement of the chest, step width, forward and backward swing of the head, forward and backward swing of the hands, forward and backward swing of the elbows, stride width, and step length while the walker 1 is walking.

[0247] The inventors of the present application have found that by using these feature amounts to calculate the walking style of the walker 1, the walking style of the walker 1 can be determined with high accuracy. Therefore, this allows the walking style of the walker 1 to be determined with high accuracy. Furthermore, for example, the walking style of the walker 1 is determined without extracting feature amounts that are not necessary for easily and accurately determining the walking style, so the walking style can be determined with high accuracy while reducing the amount of processing required for determining the walking style. Furthermore, for example, by outputting the walking style of the walker 1 as auxiliary information, it can be easily understood what measures should be taken by the walker 1 who has been determined to have a high risk of falling.

[0248] Technique 9 is a fall risk assessment method executed by a computer, the fall risk assessment method including: an acquisition step (S110) of acquiring a video image showing pedestrian 1; an estimation step (S120) of estimating the skeleton of pedestrian 1 shown in the video image based on the video image; a classification step (S130) of classifying periods in which pedestrian 1 is walking in the video image into a plurality of periods based on the skeleton; an extraction step (S140) of extracting, for each of the plurality of periods, one or more first feature amounts related to the walking of pedestrian 1 based on the skeleton; a determination step (S150) of assessing the fall risk of pedestrian 1 based on the one or more first feature amounts; and an output step (S160) of outputting the assessment result from the determination step.

[0249] This provides the same effect as the fall risk assessment device 40.

[0250] Technique 10 is a program for causing a computer to execute the fall risk determination method described in Technique 9.

[0251] This provides the same effect as the fall risk assessment device 40.

[0252] Technique 11 is a fall risk assessment device 400 including a walking motion acquisition unit 420a that acquires the walking motion of the walker 1, a classification unit 420c that classifies the period during which the walker 1 is walking into a plurality of periods, an extraction unit 420d that extracts one or more first feature amounts related to the walking of the walker 1 for each of the plurality of periods based on skeletal movement of the walking motion of the walker 1, a determination unit 420e that determines the fall risk of the walker 1 based on the one or more first feature amounts, and an output unit 420f that outputs the determination result of the determination unit 420e.

[0253] According to this, the fall risk assessment device 400 extracts one or more feature amounts for each of a plurality of periods (each walking phase) based on the movement of the skeleton (skeleton model) of the walking motion of the walker 1. The inventors of the present application have discovered that the fall risk of the pedestrian 1 can be accurately, i.e., accurately assessed by extracting one or more feature amounts for each period, such as an acceleration period, a constant velocity period, and a deceleration period, assessing the fall risk, and, for example, summing the fall risks for each period to assess (calculate) the fall risk of the pedestrian 1. Therefore, the fall risk assessment device 400 can accurately assess the fall risk of the pedestrian 1. Furthermore, the fall risk assessment device 400 can calculate the fall risk of the pedestrian 1 from the walking state of the pedestrian 1, so the fall risk of the pedestrian 1 can be calculated through simple processing.

[0254] (Other Embodiments) Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0255] For example, the instruction to start walking may be given by the user. In this case, the information terminal 30 does not need to have an instruction unit.

[0256] In addition, the information terminal 30 may transmit a moving image of the pedestrian 1 when walking to the fall risk determination device 40.

[0257] Furthermore, for example, the information terminal 30 may transmit to the fall risk assessment device 40 an instruction to start assessing the risk of a fall of the pedestrian 1, based on an instruction from the user received by the reception unit 34. In this case, for example, the fall risk assessment device 40 may transmit to the information terminal 30 information indicating an instruction to the pedestrian 1 to start walking. Furthermore, in this case, the information terminal 30 may cause the pedestrian 1 to start walking by the instruction unit 36, and may photograph the pedestrian 1 by the camera 20, based on the received information.

[0258] Furthermore, the first feature amount, the second feature amount, and the third feature amount may be the same feature amount or may be different feature amounts.

[0259] Furthermore, for example, in each of the above embodiments, a process executed by a specific processing unit may be executed by another processing unit, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0260] Furthermore, for example, in each of the above embodiments, each component of a processing unit such as information processing unit 42, 420 may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0261] Furthermore, each component may be realized by hardware. Each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0262] Furthermore, the method of communication between the devices in each of the above embodiments is not particularly limited. In addition, a relay device (such as a broadband router) (not shown) may be involved in the communication between the devices.

[0263] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0264] Furthermore, for example, the present disclosure may be realized as a fall risk assessment method, as a program for causing a computer to execute the fall risk assessment method, or as a non-transitory recording medium readable by a computer on which such a program is recorded.

[0265] Furthermore, in the above-described first embodiment, the display system 10 includes an information terminal 30 and a fall risk assessment device 40. However, the display system according to the present disclosure may be implemented as a single device, such as an information processing device, or may be implemented by multiple devices. For example, the display system may be implemented as a client-server system. When the display system is implemented by multiple devices, the components of the display system described in the above-described first embodiment may be distributed among the multiple devices in any manner. For example, in the above-described first embodiment, the fall risk assessment device 40 extracts features. The feature extraction may be performed by the information terminal 30. Furthermore, for example, the information terminal 30 may include an estimation unit. In this case, for example, the fall risk assessment device may acquire information indicating a skeleton (specifically, a skeleton model) estimated by the information terminal, and perform the processing of the classification unit, extraction unit, determination unit, and output unit based on the acquired information. When the display system according to the present disclosure includes a fall risk assessment device 400, the components of the display system may be distributed among the multiple devices in any manner, similar to the fall risk assessment device 40.

[0266] Furthermore, the display system according to the present disclosure may include one or both of the fall risk assessment device 40 and the fall risk assessment device 400. Furthermore, the fall risk assessment device according to the present disclosure may be realized by any combination of the functional configuration of the fall risk assessment device 40 and the functional configuration of the fall risk assessment device 400. For example, the fall risk assessment device according to the present disclosure may include both the acquisition unit 42a and the walking movement acquisition unit 420a. For example, the estimation unit, classification unit, extraction unit, determination unit, and output unit included in the fall risk assessment device according to the present disclosure may perform predetermined processing as described in each of the above embodiments, depending on the information acquired by the acquisition unit 42a or the walking movement acquisition unit 420a.

[0267] Furthermore, the fall risk assessment device 40 may or may not include the calculation unit 42g. Furthermore, the fall risk assessment device 400 may or may not include the calculation unit 42g.

[0268] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope of this disclosure.

[0269] 1 Pedestrian 40, 400 Fall risk determination device 42a Acquisition unit 42b Estimation unit 42c, 420c Classification unit 42d, 420d Extraction unit 42e, 420e Determination unit 42f, 420f Output unit 42g Calculation unit 420a Walking motion acquisition unit

Claims

1. A fall risk assessment device comprising: an acquisition unit that acquires a video in which a pedestrian is shown; an estimation unit that estimates the skeleton of the pedestrian shown in the video based on the video; a classification unit that classifies periods in which the pedestrian is walking in the video into a plurality of periods based on the skeleton; an extraction unit that extracts, for each of the plurality of periods, one or more first feature amounts related to the pedestrian's gait based on the skeleton; a determination unit that determines a fall risk of the pedestrian based on the one or more first feature amounts; and an output unit that outputs the determination result of the determination unit.

2. The fall risk determination device of claim 1, wherein the determination unit determines the fall risk of the pedestrian using a machine learning model trained on the basis of features related to a person's gait and a fall history indicating whether the person has fallen within a specified period of time.

3. The fall risk assessment device of claim 1, wherein the one or more first features include any one of a feature related to a joint angle, a feature related to a joint movement speed, a feature related to a joint movement acceleration, a feature related to a walking movement time, and a feature related to a joint movement distance while the pedestrian is walking.

4. A fall risk assessment device as described in claim 1, wherein the one or more first feature amounts include at least any of the average toe speed, walking speed, double leg support time, forward and backward sway range of the trunk, maximum toe speed, stride length, waist fluctuation height, forward and backward movement range of the hip joint, heel contact angle, chest up and down movement, joint movement range of the ankle joint, knee fluctuation height, and joint angle swing range of the knee joint while the pedestrian is walking.

5. A fall risk assessment device as described in claim 1, wherein the multiple periods include an acceleration period during which the pedestrian begins walking and accelerates, a constant velocity period during which the pedestrian walks steadily and which is the period following the acceleration period, and a deceleration period during which the pedestrian decelerates until he or she stops and which is the period following the constant velocity period.

6. A fall risk assessment device as claimed in any one of claims 1 to 5, further comprising a calculation unit that calculates an auxiliary index for interpreting the pedestrian's fall risk, and the output unit further outputs auxiliary information indicating the auxiliary index.

7. The fall risk assessment device of claim 6, wherein the extraction unit extracts one or more second feature amounts related to the gait of the walker based on the skeleton for each of the multiple time periods, and the calculation unit calculates the physical ability of the walker as the auxiliary index based on the one or more second feature amounts, and the one or more second feature amounts include any of a feature amount related to a joint angle, a feature amount related to a joint movement speed, a feature amount related to a joint movement acceleration, a feature amount related to a movement time during walking, and a feature amount related to a joint movement distance while the walker is walking.

8. The fall risk assessment device as described in claim 6, wherein the extraction unit extracts one or more third feature amounts related to the pedestrian's walking based on the skeleton for a period during which the pedestrian is walking, and the calculation unit calculates the pedestrian's walking style as the auxiliary indicator based on the one or more third feature amounts, and the one or more third feature amounts include at least any of the walking speed, forward and backward sway of the torso, forward and backward movement range of the hip joint, forward and backward movement range of the knee joint, both leg support time, stance time, heel height, heel contact angle, heel off angle, up and down movement of the waist, up and down movement of the chest, step width, forward and backward swing of the head, forward and backward swing of the hands, forward and backward swing of the elbows, stride width, and step length while the pedestrian is walking.

9. A fall risk assessment method executed by a computer, comprising: an acquisition step of acquiring a video showing a pedestrian; an estimation step of estimating the skeleton of the pedestrian shown in the video based on the video; a classification step of classifying periods in which the pedestrian is walking in the video into a plurality of periods based on the skeleton; an extraction step of extracting, for each of the plurality of periods, one or more first feature amounts related to the pedestrian's gait based on the skeleton; a determination step of determining the pedestrian's fall risk based on the one or more first feature amounts; and an output step of outputting a determination result from the determination step.

10. A program for causing a computer to execute the fall risk assessment method according to claim 9.

11. A fall risk assessment device comprising: a walking motion acquisition unit that acquires the walking motion of a pedestrian; a classification unit that classifies a period during which the pedestrian is walking into a plurality of periods; an extraction unit that extracts, for each of the plurality of periods, one or more first feature amounts related to the pedestrian's walking based on skeletal movement of the pedestrian's walking motion; a judgment unit that judges the pedestrian's risk of falling based on the one or more first feature amounts; and an output unit that outputs the judgment result of the judgment unit.

Images