Walking phase determination support system

Abstract

To improve determination accuracy by supporting a determination of a walking phase by a determiner.SOLUTION: An imaging section 20 images a walking motion of a walking person, and a detection section 25 detects a specular change of a force, which is applied for each of a plurality of portions in a sole, synchronously with the imaging. An input section 34 is input-operated when designating a determination target walking phase subjected to a determination in a walking period consisting of a plurality of kinds of walking phases. A control section 35 displays a moving image, which is captured by the imaging section 20, on a display section 33 and displays a plurality of still images captured in a plurality of mutually different pieces of timing from among a plurality of still images constituting the moving image on the display section 33 as display still images. Based on a specular change of a detection value of the detection section 25, the control section 35 estimates, as specific timing, timing in which a walking phase of the walking person becomes the determination target walking phase designated by an operation of the input section 34. The control section 35 displays the display still image corresponding to the specific timing in the plurality of display still images on the display section 33 in a mode related to the determination target walking phase.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a walking phase determination support system that supports determination of a walking phase by observing the walking motion of a pedestrian. [Background technology]

[0002] Physical therapists, orthopedic surgeons, and the like provide exercise therapy, physical therapy, and the like to individuals with physical disabilities due to injury, illness, or the like, or those who are predicted to develop disabilities (hereinafter, these individuals are referred to as patients) to support them in leading independent daily lives. To provide this support, physical therapists and the like typically perform gait analysis of the patient. Gait analysis requires observing the patient's walking motion and accurately determining which phase of the gait cycle the patient is in. A gait cycle is, for example, the period from when the heel of one foot touches the ground until it touches the ground again. The gait cycle is composed of a stance phase, which consists of multiple gait phases, and a swing phase, which consists of multiple gait phases. Each gait phase has its own role to play. If any gait phase deviates from the normal state, there is room for improvement in the walking posture of that gait phase and the gait phases before and after. It is necessary to take measures to restore the patient to a normal state. Therefore, accurate determination of gait phases is important.

[0003] Conventionally, assessors, such as physical therapists and orthopedic surgeons, make patients walk as if they were pedestrians. By visually observing the walking motion of the pedestrian, the gait phase is judged based on subjective judgment such as experience.

[0004] As a related technique, Patent Document 1 describes a technique for acquiring walking cycle data from walking-related data related to walking using a walker and calculating the stride length of a walker based on both data. The walking-related data includes walking distance data, walking speed data, heart rate data, and arm swing time data. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-185064 Summary of the Invention [Problem to be solved by the invention]

[0006] However, it is difficult for inexperienced physical therapists to accurately determine the walking phase using a method of determining the walking phase simply by observing the walking motion. Therefore, there is a demand for a technology to assist in the determination so that the walking phase can be accurately determined regardless of experience.

[0007] Although the technology described in Patent Document 1 uses walking cycle data, it is not a technology that supports the determination of walking phases by observing walking movements, and therefore it is difficult to meet the above demands. [Means for solving the problem]

[0008] Various aspects of a walking phase determination support system for solving the above problems will be described. a control unit that causes a video image captured by the capturing unit to be displayed on the display unit, and also causes a plurality of still images that constitute the video image, the still images being captured at a plurality of different timings, to be displayed on the display unit as display still images; wherein the control unit estimates, based on the change over time in the detected value of the detection unit, a timing at which the gait phase of the walker becomes the target gait phase that has been specified by operation of the input unit, as a specific timing; and causes the display unit to display, from among the plurality of display still images, the display still image that corresponds to the specific timing, in a manner associated with the target gait phase.

[0009] According to the above configuration, the walking motion of the walker is captured by the capturing unit. The detecting unit detects changes over time in the forces acting on each of the plurality of parts of the sole of the walker in synchronization with the capturing of the images. The assessor operates the input unit to specify a gait phase to be assessed.

[0010] The video captured by the imaging unit is displayed on the display unit by the control unit. Furthermore, from among the still images that make up the video, a plurality of still images captured at a plurality of different timings are displayed on the display unit as display still images by the control unit. By looking at the display unit, the assessor can determine the walking phase by selecting a display still image that is thought to correspond to the gait phase to be determined from the plurality of displayed display still images.

[0011] Here, the change over time in the detected value of the detection unit is closely related to the walking phase. The change over time in the detected value is expressed as, for example, a pressure waveform. The pressure waveform is a waveform that periodically repeats a section where the value is approximately constant and a section where the value changes. The former section is a section where the foot is off the ground, and the latter section is a section where the foot is in contact with the ground. For example, the force acting on the heel increases as the heel touches the ground. This force changes as the part of the sole of the foot that is in contact with the ground moves from the heel toward the toes. The force is minimum when the heel leaves the ground or is off the ground. The force acting on the toes also changes in a similar manner.

[0012] Therefore, the control unit estimates the timing when the walker's walking phase is the walking phase to be determined as the specific timing based on the change over time in the detection value of the detection unit. Then, of the multiple display still images, the display still image corresponding to the specific timing is displayed on the display unit in a manner associated with the walking phase to be determined. Of the multiple display still images displayed on the display unit, the display still image corresponding to the specific timing is displayed in a manner different from the other display still images.

[0013] Therefore, by looking at the display unit, the assessor can determine which of the multiple displayed still images corresponds to the specified gait phase to be determined. Also, by comparing the displayed still image corresponding to the gait phase to be determined with the display still image that the assessor has determined (selected), the assessor can determine whether or not his / her own determination of the gait phase was correct.

[0014] [Aspect 2] The walking phase determination support system described in [Aspect 1], wherein the control unit removes a background image from the still image and causes the display unit to display the still image after the removal as the display still image.

[0015] According to the above configuration, the control unit removes the background image that is not involved in determining the walking phase from the still image. The remaining image of the pedestrian is displayed on the display unit as a display still image. Therefore, the pedestrian is easier to see than when a still image including the background is displayed on the display unit, making it easier to determine the walking phase.

[0016] [Aspect 3] The walking phase determination support system described in [Aspect 1] or [Aspect 2], wherein the control unit estimates characteristic quantities related to the walking behavior of the walker based on the detection value of the detection unit, determines whether the walking behavior at the specific timing is abnormal walking based on the characteristic quantities, and displays the display still image corresponding to the specific timing on the display unit in a manner associated with the result of the determination regarding the abnormal walking.

[0017] According to the above configuration, the control unit estimates a feature quantity related to the walking behavior of the walker based on the detection value of the detection unit. Whether the walking behavior at a specific timing is abnormal is determined based on the feature quantity. A display still image corresponding to the specific timing is displayed on the display unit in association with the result of the determination of whether the walking behavior is abnormal.

[0018] Therefore, by looking at the display unit, the assessor can determine whether the walking at a particular timing is abnormal or not. [Aspect 4] A walking phase determination support system described in any one of [Aspect 1] to [Aspect 3], wherein the control unit estimates skeletal data of the pedestrian based on at least the displayed still image corresponding to the specific timing, and causes the display unit to display an image corresponding to the skeletal data and simulating the skeleton, superimposed on the image of the pedestrian in the displayed still image corresponding to the specific timing.

[0019] According to the above configuration, the control unit estimates skeletal data of the pedestrian based on at least the displayed still image corresponding to the specific timing. That is, multiple body parts of the pedestrian are recognized, and data related to the pedestrian's skeleton is estimated from the positional relationships of these body parts. An image corresponding to the estimated skeletal data and simulating the skeleton is displayed on the display unit, superimposed on the image of the pedestrian in the displayed still image corresponding to the specific timing.

[0020] Therefore, by looking at the display unit, the assessor can determine the state of the walker's skeleton when the walking phase is at least the walking phase to be assessed, such as the degree of joint angle. [Effects of the Invention]

[0021] According to the present invention, the accuracy of the determination can be improved by supporting the determiner in determining the walking phase. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is an explanatory diagram of a display unit in one embodiment, in which a display still image in which an image resembling a skeleton is superimposed on an image of a pedestrian is displayed. [Figure 2] FIG. 2 is a block diagram showing the configuration of the walking phase determination support system in the above embodiment. [Figure 3] FIG. 3 is an explanatory diagram illustrating a part of the configuration of the walking phase determination support system in the above embodiment. [Figure 4] FIG. 4 is an explanatory diagram illustrating the arrangement of the pressure sensors in the insole of the above embodiment and their positional relationship with each part of the sole of the foot. [Figure 5] FIG. 5 is an explanatory diagram illustrating the relationship between the walking posture that changes as the walker walks and each walking phase in the walking cycle in the above embodiment. [Figure 6] FIG. 6 is a flowchart showing the processing performed by the control unit when assisting in determining the walking phase in the embodiment. [Figure 7] FIG. 7 is a characteristic diagram showing a pressure waveform created based on the detection value of the pressure sensor of the above embodiment. [Figure 8] FIG. 8 is a characteristic diagram showing an actual pressure waveform of the pressure applied to the sole of the left foot in the above embodiment. [Figure 9] FIG. 9 is a characteristic diagram showing an actual pressure waveform of the pressure applied to the sole of the right foot in the above embodiment. [Figure 10] FIG. 10 is an explanatory diagram illustrating lines of force estimated based on the detected values ​​of the pressure sensors in the above embodiment. [Figure 11] FIG. 11 is a characteristic diagram of lines of force estimated for the left foot based on the detected values ​​of the pressure sensor in the above embodiment. [Figure 12] FIG. 12 is a characteristic diagram of lines of force estimated for the right foot based on the detected values ​​of the pressure sensor in the above embodiment. [Figure 13] FIG. 13 is an explanatory diagram of a display unit in a modified example in which a still image of a pedestrian is displayed without an image resembling a skeleton being superimposed thereon. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, a specific embodiment of the walking phase determination support system will be described with reference to FIGS. The walking phase determination support system is a system for supporting the determination of walking phases by observing the walking motion of a walker P1. The walker P1 may be, for example, a patient undergoing treatment at an orthopedic clinic or undergoing rehabilitation for motor function.

[0024] Figure 5 shows the relationship between the walking motion of a walker P1 and a walking cycle consisting of multiple walking phases. A walking cycle is the period from when one foot 10 touches down on a walkway 15, swings out, and then touches down on the walkway 15 again. The walking cycle consists of a stance phase, in which the sole of the foot steps on the walkway 15 and supports the body, and a swing phase, in which one foot 10 is lifted and swung out forward.

[0025] The stance phase consists of multiple gait phases, such as initial contact IC, load response phase LR, mid-stance phase MSt, end-stance phase TSt, and pre-swing phase PSw. The swing phase consists of multiple gait phases, such as initial swing ISw, mid-swing MSw, and end-swing TSw. Each gait phase is defined as follows: In FIG. 5, one foot 10 (right foot) of the walker P1 is represented by a thick solid line, and the contralateral lower limb (left lower limb, no symbol) is represented by a thick dashed line.

[0026] Initial contact IC: The moment when one foot 10 touches the walkway 15. Load-bearing response period LR: The period from the moment one foot 10 touches the ground to the moment the contralateral lower limb leaves the walking path 15.

[0027] Mid-stance MSt: The period from when the contralateral lower limb leaves the walking path 15 until the heel of the contralateral lower limb leaves the walking path 15. In the mid-stance MSt, only one foot 10 is in contact with the ground. End-stance phase TSt: The period from when the heel of the contralateral lower limb leaves the walking path 15 until the contralateral lower limb touches the ground. The contralateral lower limb makes initial contact with the ground at the end of the end-stance phase TSt.

[0028] Pre-swing phase (PSw): The period from when the contralateral lower limb touches the ground until the toe of one foot (10) leaves the walking path (15). Initial swing phase ISw: The period from when the toe of one foot 10 leaves the walking path 15 until both lower legs cross.

[0029] Mid-swing phase (MSw): The period from when both legs cross until they become vertical. End-swing phase TSw: The period from when the lower leg becomes vertical to when the foot on one side touches the ground. 2 and 3, the walking phase determination support system includes an imaging unit 20, a detection unit 25, and a determination support device 30. Next, each unit constituting the walking phase determination support system will be described.

[0030] <Photography Team 20> The imaging unit 20 is for imaging the walking movement of the walker P1. The imaging unit 20 is installed on a horizontal or nearly horizontal flat walkway 15. The walkway 15 may be installed indoors or outdoors.

[0031] The photographing unit 20 includes at least one photographing device 21. Examples of the photographing device 21 include a digital camera and a tablet equipped with a digital camera. In this embodiment, the photographing device 21 is disposed near one end 15a of the walkway 15, at a location offset outward in the width direction of the walkway 15, and facing the other end 15b of the walkway 15.

[0032] If the direction from end 15b toward end 15a is defined as the forward direction, then photographing device 21 generates an image by photographing pedestrian P1 walking on walkway 15 from diagonally forward. Then, photographing device 21 generates photographed data of a moving image in which the above-mentioned multiple images are successively displayed over time.

[0033] The photographing unit 20 includes a transmission unit 22 in addition to the photographing device 21. The transmission unit 22 transmits photographed data of the moving image generated by the photographing device 21 to a reception unit 31 of the judgment support device 30.

[0034] <Detection unit 25> As shown in FIGS. 2 to 4, the detection unit 25 detects changes over time in the force acting on each of multiple parts of the sole of the foot of the walker P1 in synchronization with the imaging by the imaging unit 20. The detection unit 25 is provided in each of the left and right shoes 11. Each detection unit 25 includes an insole 26 placed on the inside bottom of the shoe 11, and a pressure sensor 27 and a transmission unit 28 incorporated into the insole 26. The pressure sensor 27 detects pressure as a force acting on each of multiple parts of the sole of the foot 10 while walking. The pressure sensor 27 includes a heel sensor 27a, a toe sensor 27b, an inner sensor 27c, and an outer sensor 27d.

[0035] Heel sensor 27a is placed in the insole 26 at a portion of the heel 10a where weight is applied, and detects the pressure applied to the heel 10a of the sole. Toe sensor 27b is placed in the insole 26 at a portion of one of the five toes that make up the toes 10b where weight is applied, and detects the pressure applied to the toes 10b of the sole.

[0036] Here, the inside of the foot 10 is the side closer to the opposite foot 10 in the left-right direction, and the outside is the side farther from the opposite foot 10 in the same direction. The inner sensor 27c is located on the insole 26 inside the imaginary line L1 connecting the heel sensor 27a and the toe sensor 27b, in a portion where the ball of the foot bears weight, and detects pressure on the inner part of the sole. The outer sensor 27d is located on the insole 26 outside the imaginary line L1, in a portion where the ball of the little toe bears weight, and detects pressure on the outer part of the sole.

[0037] When it is not necessary to distinguish the above-mentioned heel sensor 27a, toe sensor 27b, inner sensor 27c, and outer sensor 27d from one another, they may be simply referred to as "pressure sensors 27."

[0038] Each pressure sensor 27 independently detects the pressure applied to each part of the sole at predetermined time intervals. The reason why the pressure sensors 27 are arranged at the four locations on the insole 26 is as follows.

[0039] When walking, the heel 10a of the sole of the foot lands first, and the landing position changes from the heel 10a to the toes 10b. After the heel 10a lands and before the toes 10b land, at least one of the inner and outer parts of the virtual line L1 lands. The toe gripping force generated varies depending on the landing position.

[0040] Therefore, in this embodiment, pressure sensors 27 (heel sensor 27a, toe sensor 27b) are incorporated into the insole 26 in the portion where weight is applied to the heel 10a and the portion where weight is applied to the toe 10b. In addition, pressure sensors 27 (inner sensor 27c, outer sensor 27d) are incorporated into the insole 26 in the portion that is on the inner side and the portion that is on the outer side of the virtual line L1. Therefore, the pressure sensors 27 incorporated into the insole 26 can appropriately detect changes over time in the pressure applied to each of the multiple parts of the sole of the foot.

[0041] A known pressure-sensitive sensor using a piezoelectric element or the like can be used as the pressure sensor 27. In particular, considering that the pressure sensor is placed on the sole of the foot, it is preferable to use a capacitance-type sensor made of an elastomer using a dielectric elastomer from the viewpoints of stretchability and durability. Examples of the dielectric elastomer include cross-linked polyrotaxane, silicone elastomer, acrylic elastomer, and urethane elastomer.

[0042] The elastomer capacitance sensor can be made thin overall, so even when placed inside the insole 26, it has the advantage of not significantly increasing the thickness of the insole 26. This sensor has a structure in which a dielectric elastomer is placed between a pair of electrodes. When the dielectric elastomer deforms due to tension or stress, the amount of electricity (capacitance) stored in the electrodes changes. This change is detected as pressure on the sole of the foot.

[0043] The detection value detected by each pressure sensor 27 can be converted into a pressure value such as a capacitance value or an electrical resistance value according to the detection method of the pressure sensor 27. Therefore, the "detection value" described below can be interpreted as the detection value of the pressure at the position where the pressure sensor 27 is attached on the sole of the foot.

[0044] The transmitting unit 28 transmits the detection values ​​of the pressure sensors 27 to the receiving unit 31 of the judgment support device 30 . <Judgment support device 30> As shown in FIG. 2, the judgment support device 30 is configured by a computer such as a mobile terminal or a tablet terminal.

[0045] The judgment support device 30 includes a receiving unit 31 that receives the imaging data transmitted from the transmitting unit 22 of the imaging unit 20 and the detection values ​​transmitted from the transmitting unit 28 of the detecting unit 25, and a storage unit 32 that stores the received imaging data and detection values. The receiving unit 31 has a wired or wireless communication means and communicates with the transmitting units 22 and 28 by a known communication method.

[0046] The storage unit 32 may be, for example, an HDD, an SSD, a semiconductor memory element, etc. The storage unit 32 may also be a storage device connected to the judgment support device 30 via a network.

[0047] Furthermore, the judgment support device 30 includes a display unit 33, an input unit 34, and a control unit 35. Various images are displayed on the display unit 33. The images displayed on the display unit 33 include footage of the walking movement of the walker P1. As the display unit 33, for example, a known display device such as a liquid crystal display can be used.

[0048] The input unit 34 is operated when inputting various types of information. For example, the input unit 34 is operated when specifying a gait phase to be determined among a gait cycle consisting of multiple types of gait phases. As the input unit 34, for example, a known input device such as a keyboard, a mouse, or a touch panel can be used.

[0049] The control unit 35 performs a process of assisting in determining the walking phase based on the photographed data, detected values, etc. stored in the storage unit 32. <Operation of this embodiment> Next, a procedure for assisting in determining a walking phase in the walking phase determination support system of this embodiment configured as described above will be described.

[0050] As shown in Figures 2 to 4, a walker P1 wears shoes 11 with insoles 26. The walker P1 walks on the walkway 15, starting from the other end 15b and heading toward one end 15a.

[0051] The photographing device 21 generates images by photographing the walking movement of the pedestrian P1 while the pedestrian P1 is walking along the walkway 15. The photographing device 21 then generates photographed data of a video in which the plurality of images are successively displayed over time. The photographing device 21 transmits the generated photographed data from the transmitting unit 22 of the photographing unit 20 to the receiving unit 31 of the judgment support device 30.

[0052] Additionally, while the walker P1 is walking on the walkway 15, the pressure sensors 27 detect changes over time in pressure applied to each of multiple parts of the sole of the foot in synchronization with the image capture. That is, the heel sensor 27a detects pressure applied to the heel 10a of the sole, and the toe sensor 27b detects pressure applied to the toes 10b of the sole. The inner sensor 27c detects pressure applied to the inner part of the sole, and the outer sensor 27d detects pressure applied to the outer part of the sole. The transmitter 28 of the detector 25 transmits the detected values ​​to the receiver 31 of the judgment support device 30.

[0053] The receiving unit 31 of the judgment support device 30 receives the imaging data transmitted from the transmitting unit 22 of the imaging unit 20 and stores it in the memory unit 32. The receiving unit 31 also receives the detection value transmitted from the transmitting unit 28 of the detection unit 25 and stores it in the memory unit 32.

[0054] An assessor who views the walking motion of the walker P1 and attempts to determine a predetermined gait phase operates the input unit 34 to specify the gait phase to be determined. The assessor is typically a physical therapist or an orthopedic surgeon, but may also be the walker P1. The assessor may perform an input operation on the input unit 34, for example, after the assessor views a plurality of display still images 41 displayed on the display unit 33 and selects one that corresponds to the gait phase to be determined (gait phase to be determined).

[0055] The flowchart in FIG. 6 shows the process performed by the control unit 35 when assisting in determining the walking phase. In step S11, the shooting data of the moving image that has been shot by the shooting unit 20 and stored in the storage unit 32 is read.

[0056] In step S12, background images that are not involved in determining the walking phase are removed from the still images that make up the moving image. In step S13, skeletal data of the pedestrian P1 is estimated from the still image using a technique called skeletal estimation. Skeletal estimation is a technique that recognizes the positions of multiple parts such as the joints of the human body and facial features (eyes, nose, ears, etc.), and estimates the human skeleton from the relative positions of these parts.

[0057] In step S14, an image 42 that corresponds to the estimated skeletal data and resembles a skeleton, for example, an image 42 consisting of multiple dots (·) and multiple lines connecting adjacent dots, is superimposed on the image of the pedestrian P1. The dots represent joints, and the lines are also called skeletal lines and represent the skeleton.

[0058] In step S15, as described above, the display unit 33 displays a moving image made up of a plurality of still images in which the background image has been removed and the skeleton-like image 42 has been superimposed on the image of the pedestrian P1.

[0059] In step S16, as shown in Fig. 1, from among the still images constituting the moving image, a plurality of still images captured at a plurality of different timings are displayed on the display unit 33 as display still images 41. The plurality of timings may be, for example, timings at regular time intervals, but are not limited to this. This technique of freezing one of the still images constituting the moving image as if time had stopped and making it into a still image state, that is, leaving it as an afterimage, is called a freeze frame.

[0060] In the displayed display still image 41, the background image has been removed by the processing of step S12. In addition, in the displayed display still image 41, the image 42 resembling a skeleton has been superimposed on the image of the pedestrian P1 by the processing of step S14. The display unit 33 displays the display still image 41 showing the pedestrian P1 remaining after the removal and the image 42 resembling a skeleton. By looking at the display unit 33, the assessor can determine the walking phase by selecting the display still image 41 that is thought to correspond to the walking phase to be determined from the multiple displayed display still images 41.

[0061] In step S17, the control unit 35 reads the gait phase to be determined, which is input through an input operation of the input unit 34 shown in FIG. In step S18, the detection values ​​detected by the four types of pressure sensors 27 for each of the left and right feet 10 and stored in the memory unit 32 are read. Based on the read detection values, a pressure waveform is created that represents the change in the detection values ​​over time.

[0062] An example of a pressure waveform created by the control unit 35 is shown in Figure 7. In Figure 7, the solid line indicates a pressure waveform created based on the detection value of the heel sensor 27a, and the dashed line indicates a pressure waveform created based on the detection value of the toe sensor 27b. The dashed line indicates a pressure waveform created based on the detection value of the inner sensor 27c, and the broken line indicates a pressure waveform created based on the detection value of the outer sensor 27d. Figures 8 and 9 show examples of actual pressure waveforms.

[0063] Fig. 8 shows a pressure waveform created for the left foot of a particular walker P1, and Fig. 9 shows a pressure waveform created for the right foot of the same walker P1 as in Fig. 8. Here, the pressure waveform is closely related to the gait phase. As shown in FIG. 7, the pressure waveform is a waveform that periodically repeats a section where the value is approximately constant and a section where the value changes. The former section is a section where the foot 10 is off the walkway 15 (swing phase), and the latter section is a section where the foot 10 is in contact with the walkway 15 (stance phase). For example, the force applied to the heel 10a increases as the heel 10a touches the ground. This force changes as the contact portion of the sole of the foot 10 moves from the heel 10a toward the toe 10b. The force is minimum when the heel 10a is leaving or is off the walkway 15. The force applied to the toe 10b also changes in a similar manner. Therefore, it is possible to estimate the timing and the gait phase from the pressure waveform.

[0064] Therefore, in step S19, the timing when the gait phase of the walker P1 becomes the gait phase to be determined designated by operating the input unit 34 is estimated as the specific timing from the pressure waveform.

[0065] In step S20, the characteristic quantities relating to the walking movement of the walker P1 are estimated based on the detected values ​​stored in the storage unit 32. Figures 11 and 12 show lines of force, which are one of the feature quantities related to walking behavior. Figure 11 shows lines of force estimated for the left foot of the same walker P1 for which the pressure waveforms in Figures 8 and 9 were created. Figure 12 shows lines of force estimated for the right foot of the same walker P1 as in Figure 11.

[0066] As shown in FIG. 10 , a circle with a radius of 1 is assumed to represent the lines of force. This circle corresponds to the sole of the foot. The center of the circle indicates the origin. The direction in which the vertical axis extends indicates the front-to-back direction in which the force of the foot 10 is applied to the insole 26. The direction in which the horizontal axis extends indicates the left-to-right direction in which the force of the foot 10 is applied to the insole 26.

[0067] Heel sensor 27a is located within the circle and behind the origin. Toe sensor 27b is located within the circle and in front of the origin. Inner sensor 27c and outer sensor 27d are located within the circle at positions sandwiching the origin on both the left and right sides.

[0068] When a force is applied to the insole 26, the force is applied to each pressure sensor 27. The center where the force is applied to the four pressure sensors 27 is referred to as the "point of force." When the forces applied to the four pressure sensors 27 are balanced, the point of force is located at the origin.

[0069] If the forces acting on the four pressure sensors 27 are not balanced, the point of force will be located at a point away from the origin. The balance of forces is expressed by the ratio of the detected values ​​of the four pressure sensors 27. For example, if the ratio of the force acting on the toe sensor 27b increases, the point of force will move forward from the origin to a point away by an amount corresponding to that ratio.

[0070] Therefore, the ratio of the detected values ​​of the four pressure sensors 27 is calculated. The point of force is moved by the calculated ratio toward the side where the pressure sensor 27 with the higher ratio is located. Specifically, the point of force is moved forward by the proportion of the force applied to the toe sensor 27b. The point of force is moved backward by the proportion of the force applied to the heel sensor 27a. If the point of force is on the left foot, the point of force is moved right by the proportion of the pressure applied to the inner sensor 27c. The point of force is moved left by the proportion of the pressure applied to the outer sensor 27d.

[0071] The relationship between the inside and outside of the right foot is reversed in the left-right direction. Then, the lines of force that are estimated from the locus of the force point that moves over time become the lines of force shown in Figures 11 and 12. In step S20, the lines of force are estimated as described above.

[0072] In step S21, based on the force lines estimated in step S20, it is determined whether the walking behavior at the specific timing is abnormal, i.e., whether there is an abnormality in the walking behavior. For example, a region of force lines that a normal walker P1 can take is determined in advance, and it is determined whether the force lines estimated in step S20 are within this region. If the force lines are within this region, it is determined that there is no abnormality in the walking behavior, i.e., normal walking, and if they are outside this region, it is determined that there is an abnormality in the walking behavior, i.e., abnormal walking. For example, as shown in Figure 12, most of the force lines estimated for the right foot change in the front-back direction within a predetermined region in the left-right direction. However, the portion of the force lines indicated by the symbol X changes so that it jumps out significantly diagonally forward from the region. In this case, it can be determined that there is an abnormality in the walking behavior.

[0073] In step S22, the display unit 33 displays, among the multiple display still images 41, a display still image 41 corresponding to the specific timing estimated in step S19 in a manner associated with the gait phase to be determined. This display still image 41 shows an image of the pedestrian P1 remaining after removing the background image and an image 42 simulating a skeleton. For example, as shown in FIG. 1 , in the display still image 41 corresponding to the specific timing, characters indicating the gait phase to be determined are displayed on or near the image of the pedestrian P1. If the gait phase to be determined read in step S17 is the stance end phase TSt, the characters "TSt" are displayed above the image of the pedestrian P1 in the corresponding display still image 41. At this time, the image of the pedestrian P1 may be highlighted in a manner different from that of the other display still images 41. For example, the image of the pedestrian P1 may be displayed in a color different from that of the images of the pedestrian P1 in the other display still images 41. Furthermore, the image of the pedestrian P1 may be displayed in a flashing manner.

[0074] Furthermore, in step S22, the display unit 33 displays a display still image 41 corresponding to the specific timing in a manner associated with the determination result in step S21, i.e., the presence or absence of an abnormality. For example, as shown in FIG. 1, in the display still image 41 corresponding to the specific timing, text indicating the determination result is displayed on or near the image of the walker P1. For example, the text "abnormal walking" or "normal walking" is displayed.

[0075] In this way, of the plurality of display still images 41 displayed on the display unit 33, the display still image 41 corresponding to a specific timing is displayed in a manner different from the other display still images 41. Therefore, by looking at the display unit 33, the assessor can determine which of the multiple displayed still images 41 corresponds to the specified gait phase to be determined. Also, by comparing the still image 41 corresponding to the gait phase to be determined with the still image 41 that the assessor determined (selected) himself, the assessor can determine whether his own determination of the gait phase was correct. In addition, the presence or absence of abnormalities in the walking movement can be determined. Furthermore, by looking at the image 42 that is superimposed on the image of the walker P1 and that resembles a skeleton, the condition of the walker's skeleton can also be determined.

[0076] <Effects of this embodiment> (1) In this embodiment, a moving image captured by the image capturing unit 20 is displayed on the display unit 33 (step S15). In addition, from among the still images that make up the moving image, a plurality of still images captured at different times are displayed on the display unit 33 as display still images 41 (step S16).

[0077] Therefore, the assessor can subjectively determine the walking phase by selecting a display still image 41 that is thought to correspond to the walking phase to be determined from among the multiple display still images 41 displayed on the display unit 33.

[0078] In this embodiment, a specific timing at which the walking phase of the walker P1 becomes the walking phase to be determined, which is designated by operating the input unit 34, is estimated based on the change over time in the detection value of the detection unit 25 (step S19). Of the multiple display still images 41, the display still image 41 corresponding to the specific timing is displayed on the display unit 33 in a manner associated with the walking phase to be determined (step S22).

[0079] Therefore, the assessor can know that the display still image 41 displayed on the display unit 33 corresponds to the gait phase to be determined that he / she has designated. In addition, the assessor can know whether the result of his / her determination (selection) was correct. If it was incorrect, the assessor can consider the reason and use this information to improve the accuracy of the determination when determining the next gait phase.

[0080] In this way, according to this embodiment, it is possible to provide support for improving the accuracy of the assessor's determination of the walking phase. Furthermore, as long as the environment allows for transmission and reception of photographed data and pressure detection values, the above-mentioned effects can be achieved even in a remote location, i.e., a location away from the location where the walking movement of walker P1 is photographed and the pressure on the soles of the feet of walker P1 is detected.

[0081] (2) In this embodiment, background images that are not involved in determining the walking phase are removed from the still images that make up the video (step S12), and an image showing only the pedestrian P1 is displayed on the display unit 33 as the display still image 41 (step S16).

[0082] Therefore, compared to when the display still image 41 showing the background is displayed on the display unit 33, the image of the walker P1 is easier to see, and the walking phase can be more easily determined. (3) In this embodiment, a line of force is estimated as a feature quantity related to the walking behavior of the walker P1 based on the detection value of the detection unit 25 (step S20). Based on the line of force, it is determined whether the walking behavior at a specific timing is abnormal walking (step S21). A display still image 41 corresponding to the specific timing is displayed on the display unit 33 in a manner associated with the result of the determination (step S22).

[0083] Therefore, by looking at the display unit 33, the assessor can know whether the walking at a particular timing is abnormal or not. (4) In this embodiment, skeletal data of the pedestrian P1 is estimated based on each still image (step S13). Each still image also includes a display still image 41 corresponding to a specific timing. An image 42 corresponding to the skeletal data and simulating the skeleton is displayed on the display unit 33 in a state where it is superimposed on the image of the pedestrian P1 in each still image (step S16).

[0084] Therefore, the assessor can see the skeletal state of the walker P1 when the walking phase is at least the walking phase to be assessed, such as the degree of joint angle, by looking at the display unit 33. In this case, for example, by measuring the joint angle, it becomes possible to evaluate the joint angle.

[0085] <Example of change> This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0086] At least one of the number and the positions of the photographing devices 21 may be changed. For example, if the photographing unit 20 is equipped with two photographing devices 21, the photographing devices 21 may be arranged facing each other at two locations on either side of the ends 15a and 15b of the walkway 15, near the ends 15a and 15b. When a pedestrian P1 walks along the walkway 15 from the end 15b toward the end 15a, the photographing device 21 arranged near the end 15a photographs the pedestrian P1 from the front as he approaches. The photographing device 21 arranged near the end 15b photographs the pedestrian P1 from the back as he moves away as he walks.

[0087] It is also possible to use only one of the photographing device 21 that photographs the pedestrian P1 from the front side and the photographing device 21 that photographs the pedestrian P1 from the back side. In addition to or instead of at least one of the two photographing devices 21, another photographing device 21 may be placed between the ends 15a and 15b, at a location away from the walkway 15 in the width direction of the walkway 15. This photographing device 21 photographs the pedestrian P1 walking on the walkway 15 from the side.

[0088] The number and arrangement of the pressure sensors 27 incorporated in the insole 26 may be changed as appropriate. In the above embodiment and modified example, pressure has been described as one type of force, but the force may also be acceleration or torque. The force sensor may be an acceleration sensor, torque sensor, or the like that detects such forces. These sensors may be incorporated into the insole 26, for example.

[0089] The photographing unit 20 may include a storage unit that stores photographed data photographed by the photographing device 21. The detecting unit 25 may include a storage unit that stores the detected value of the pressure sensor 27.

[0090] Note that, when a removable recording medium such as a memory card is used as the storage unit in the imaging unit 20, the transmission unit 22 may be omitted. Also, when a recording medium similar to the above is used as the storage unit in the detection unit 25, the transmission unit 28 may be omitted. When both the transmission units 22 and 28 are omitted, the reception unit 31 of the judgment support device 30 may be omitted.

[0091] 6, i.e., the process of superimposing the skeleton image 42 on the image of the pedestrian P1, may be omitted. In this case, by performing the processes of steps S16 and S22, the display unit 33 displays only the display still image 41 of the pedestrian P1 without the skeleton image 42 being superimposed, as shown in FIG.

[0092] 6, the processes of steps S17 to S22 may be omitted. In this case, a moving image captured by the image capturing unit 20 is displayed on the display unit 33. Also, a display still image 41, which is a plurality of still images captured at different times and in which an image 42 resembling a skeleton is superimposed on an image of the pedestrian P1, is displayed on the display unit 33.

[0093] Therefore, the assessor can accurately determine the walking phase by looking at the image of the walker P1 in the displayed still image 41 and the skeleton-like image 42. Therefore, this modified example also has the effect of supporting the determination of the walking phase by looking at the walking movement of the walker P1.

[0094] The process of displaying on the display unit 33 an image 42 that corresponds to the skeletal data and that resembles a skeleton, superimposed on the image of the pedestrian P1 in each still image, may be performed only on the display still image 41 that corresponds to a specific timing.

[0095] The number of gait phases to be determined input through operation of the input unit 34 may be one or more. In this case, a number of specific timings corresponding to the input gait phases to be determined are estimated. Then, of the multiple display still images 41, the display still images 41 corresponding to each specific timing are displayed on the display unit 33 in a manner associated with the gait phase to be determined.

[0096] <Additional Notes> The technical ideas that can be understood from the above-described embodiment and modified examples will be described. (A) A walking phase determination support system that supports determining a walking phase by observing a walking motion of a pedestrian, an imaging unit that images the walking motion of the pedestrian; a detection unit that detects a change over time in force applied to each of a plurality of parts of the sole of the walker in synchronization with the photographing by the photographing unit; an input unit that is operated to specify a gait phase to be determined among a plurality of types of gait phases in a gait cycle; a display unit that displays an image relating to the walking motion of the pedestrian; a control unit that causes the display unit to display the moving image captured by the imaging unit, and also causes the display unit to display, from among still images that constitute the moving image, a plurality of still images that have been captured at a plurality of different timings; and wherein the control unit estimates skeletal data of the pedestrian based on at least the displayed still image corresponding to the specific timing, and causes the display unit to display an image that corresponds to the skeletal data and imitates the skeleton, superimposed on an image of the pedestrian in the displayed still image.

[0097] According to the above configuration, the assessor can accurately determine the walking phase by viewing the image of the pedestrian in the displayed still image and the image that resembles the skeleton. In this case, too, the accuracy of the determination can be improved by supporting the assessor in determining the walking phase. [Explanation of symbols]

[0098] 20...Photography Department 25...Detection unit 33…Display section 34...Input section 35...Control unit 41…Display still image 42...Image P1...Pedestrian

Claims

1. A walking phase determination support system that supports the determination of a walking phase by observing the walking motion of a pedestrian, an imaging unit that images the walking motion of the pedestrian; a detection unit that detects a change over time in force applied to each of a plurality of parts of the sole of the walker in synchronization with the photographing by the photographing unit; an input unit that is operated to specify a gait phase to be determined among a plurality of types of gait phases in a gait cycle; a display unit that displays an image relating to the walking motion of the pedestrian; a control unit that causes the display unit to display the moving image captured by the imaging unit, and also causes the display unit to display, from among still images that constitute the moving image, a plurality of still images that have been captured at a plurality of different timings; and the control unit estimates, based on a change over time in the detection value of the detection unit, a timing at which the walking phase of the walker becomes the walking phase to be determined that is specified by operating the input unit, as a specific timing, and causes the display unit to display, from among the plurality of display still images, the display still image that corresponds to the specific timing in a manner associated with the walking phase to be determined.

2. The walking phase determination support system according to claim 1 , wherein the control unit removes a background image from the still image, and causes the display unit to display the still image after the removal of a background image as the display still image.

3. 2. The walking phase determination support system according to claim 1, wherein the control unit estimates a feature amount related to the walking behavior of the walker based on the detection value of the detection unit, determines whether the walking behavior at the specific timing is abnormal walking based on the feature amount, and causes the display unit to display the display still image corresponding to the specific timing in a manner associated with the result of the determination regarding the abnormal walking.

4. The walking phase determination support system according to any one of claims 1 to 3, wherein the control unit estimates skeletal data of the pedestrian based on at least the display still image corresponding to the specific timing, and causes the display unit to display an image corresponding to the skeletal data and simulating the skeleton, superimposed on the image of the pedestrian in the display still image corresponding to the specific timing.

Images