Gait information generating device, gait information generating method, and program

Abstract

To provide a gait information generation device, etc., for generating gait information by which the gait of a subject can be intuitively acquired.SOLUTION: Provided is a gait information generation device, including: an acquisition unit that acquires waist position information including time-series data of a waist position of a subject in a predetermined gait cycle; a waist oscillation calculation unit that calculates, for a plurality of gait phases included in the predetermined gait cycle, a waist oscillation corresponding to a distance between a reference line set for the time-series data of the waist position of the subject and the waist position in each of the plurality of gait phases; a gait information generation unit that generates gait information according to the waist oscillation calculated with respect to the predetermined gait cycle; and an output unit that outputs the generated gait information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a gait information generation device and the like that generates information according to features included in a walking pattern. [Background technology]

[0002] With the growing interest in healthcare, attention is being focused on services that provide information (also called gait information) according to the characteristics contained in walking patterns (also called gait).If gait information that allows users to intuitively understand gait can be presented, healthcare services that meet the needs of users can be provided.

[0003] Patent Document 1 discloses a motion analysis method for analyzing a user's motion information. The method of Patent Document 1 analyzes the user's motion using the detection results of an inertial sensor. The method of Patent Document 1 generates multiple pieces of motion information about the user who is exercising. The method of Patent Document 1 presents the user with a comparison result between at least one piece of motion information among the multiple pieces of motion information and a preset reference value.

[0004] Patent Document 2 discloses a gait measurement system that calculates a gait index based on acceleration data measured by an inertial measurement unit. The system of Patent Document 2 detects at least one walking phase from the acceleration data measured by the inertial measurement unit. The system of Patent Document 2 also calculates velocity data by time-integrating the acceleration data. The system of Patent Document 2 calculates a correction amount corresponding to the walking phase using the walking phase and the velocity data. The system of Patent Document 2 calculates corrected velocity data by subtracting the correction amount from the velocity data corresponding to the walking phase, and then calculates trajectory data by time-integrating the calculated corrected velocity data. The system of Patent Document 2 uses the calculated trajectory data to calculate a gait index, which is a numerical value for quantitatively evaluating gait. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-034478 [Patent Document 2] International Publication No. 2020 / 105115 Summary of the Invention [Problem to be solved by the invention]

[0006] The technique of Patent Document 1 makes it possible to present the results of comparing the user's exercise information with a reference value. However, the technique of Patent Document 1 does not provide information that allows the user to intuitively grasp the user's exercise.

[0007] According to the method of Patent Document 2, gait indices are calculated using trajectory data obtained by time-integrating corrected velocity data that has been corrected in association with walking phases, and therefore gaits can be measured with high accuracy. However, the method of Patent Document 2 fails to present information that allows for intuitive understanding of gait.

[0008] An object of the present disclosure is to provide a gait information generation device and the like that generates gait information that enables intuitive understanding of the gait of a subject. [Means for solving the problem]

[0009] A gait information generating device according to one aspect of the present disclosure includes an acquisition unit that acquires waist position information including time series data of the waist position of a subject in a predetermined walking cycle; a waist sway calculation unit that calculates waist sway corresponding to the distance between a reference line set for the time series data of the waist position of the subject and the waist position in each of the plurality of walking phases included in the predetermined walking cycle; a gait information generation unit that generates gait information corresponding to the waist sway calculated for the predetermined walking cycle; and an output unit that outputs the generated gait information.

[0010] In one aspect of the gait information generation method of the present disclosure, waist position information including time series data of the waist position of a subject in a predetermined walking cycle is acquired, and for multiple walking phases included in the predetermined walking cycle, waist sway corresponding to the distance between a reference line set for the time series data of the waist position of the subject and the waist position in each of the multiple walking phases is calculated, gait information corresponding to the waist sway calculated for the predetermined walking cycle is generated, and the generated gait information is output.

[0011] A program according to one aspect of the present disclosure causes a computer to perform the following processes: acquiring waist position information including time series data of the waist position of a subject in a predetermined walking cycle; calculating waist sway corresponding to the distance between a reference line set for the time series data of the waist position of the subject and the waist position in each of the plurality of walking phases included in the predetermined walking cycle; generating gait information corresponding to the waist sway calculated for the predetermined walking cycle; and outputting the generated gait information. [Effects of the Invention]

[0012] According to the present disclosure, it is possible to provide a gait information generation device and the like that generates gait information that allows a subject's gait to be intuitively grasped. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of a gait information generating device according to a first embodiment. [Figure 2] FIG. 2 is a conceptual diagram for explaining an example of a walking event related to gait information generated by the gait information generating device according to the first embodiment. [Figure 3] FIG. 2 is a conceptual diagram showing an example of the relationship between the tracking center of a virtual camera and the waist position in display information included in gait information generated by the gait information generation device according to the first embodiment. [Figure 4] 2 is a conceptual diagram showing a display example of display information included in gait information generated by the gait information generating device according to the first embodiment. FIG. [Figure 5]3 is a graph showing an example of time-series data of waist position used by the gait information generating device according to the first embodiment to calculate the waist position. [Figure 6] FIG. 2 is a conceptual diagram for explaining calculation of a waist position by the gait information generating device according to the first embodiment. [Figure 7] 4 is a graph showing an example of time-series data of waist position calculated by the gait information generating device according to the first embodiment. [Figure 8] 2 is a conceptual diagram showing a display example of display information generated by the gait information generating device according to the first embodiment. FIG. [Figure 9] 4 is a graph showing an example of time-series data of waist position used by the gait information generating device according to the first embodiment to calculate waist position over a plurality of walking cycles. [Figure 10] 4 is a graph showing an example of time-series data of waist position over a plurality of walking cycles calculated by the gait information generating device according to the first embodiment. [Figure 11] 4 is a flowchart for explaining an example of the operation of the gait information generating device according to the first embodiment. [Figure 12] FIG. 10 is a conceptual diagram for explaining an application example 1-1 according to the first embodiment. [Figure 13] FIG. 10 is a conceptual diagram for explaining application example 1-2 related to the first embodiment. [Figure 14] FIG. 10 is a conceptual diagram for explaining application example 1-3 according to the first embodiment. [Figure 15] FIG. 10 is a conceptual diagram for explaining application example 1-3 according to the first embodiment. [Figure 16] FIG. 10 is a block diagram showing an example of the configuration of a gait information generating device according to a second embodiment. [Figure 17] 10 is a graph for explaining calculation of the velocity of waist sway by the gait information generating device according to the second embodiment. [Figure 18] FIG. 10 is a conceptual diagram showing a display example of display information generated by a gait information generating device according to a second embodiment. [Figure 19] 10 is a flowchart for explaining an example of the operation of the gait information generating device according to the second embodiment. [Figure 20] FIG. 10 is a conceptual diagram for explaining an application example 2-1 according to the second embodiment. [Figure 21] FIG. 10 is a block diagram showing an example of the configuration of a gait information generating device according to a third embodiment. [Figure 22] FIG. 11 is a conceptual diagram showing an example of the relationship between the center of tracking and the waist position in display information included in gait information generated by a gait information generating device according to a third embodiment. [Figure 23] FIG. 10 is a conceptual diagram showing a display example of display information included in gait information generated by a gait information generating device according to a third embodiment. [Figure 24] 10 is a flowchart for explaining an example of the operation of the gait information generating device according to the third embodiment. [Figure 25] FIG. 10 is a conceptual diagram for explaining an application example 3-1 according to the third embodiment. [Figure 26] FIG. 11 is a conceptual diagram for explaining an application example 3-2 according to the third embodiment. [Figure 27] FIG. 10 is a block diagram showing an example of the configuration of a gait information generating device according to a fourth embodiment. [Figure 28] FIG. 2 is a block diagram showing an example of a hardware configuration for executing processing according to each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, the embodiments described below are limited in a manner that is technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following. In all drawings used to describe the following embodiments, the same reference numerals are used for similar parts unless otherwise specified. Furthermore, in the following embodiments, repeated explanations of similar configurations and operations may be omitted.

[0015] (First embodiment) First, the configuration of a gait information generation device according to a first embodiment will be described with reference to the drawings. The gait information generation device according to this embodiment calculates fluctuations in the waist (also referred to as waist sway) using waist position information in the direction of travel measured according to the walking of a person to be verified (also referred to as a user). The gait information generation device according to this embodiment generates information (also referred to as gait information) according to features (also referred to as gait) included in a walking pattern, according to the calculated waist sway.

[0016] (composition) 1 is a block diagram showing the configuration of a gait information generation device 10 according to this embodiment. The gait information generation device 10 includes an acquisition unit 11, a waist sway calculation unit 12, a gait information generation unit 15, and an output unit 17.

[0017] The acquisition unit 11 acquires waist position information in the direction of travel of the subject. The acquisition unit 11 acquires waist position information in a predetermined walking section. For example, the predetermined walking section is a one-step gait cycle. The predetermined walking section may be a plurality of walking cycles. Hereinafter, a one-step gait cycle is defined as the period from when the heel of the right foot lands on the ground until the heel of the right foot lands on the ground again.

[0018] FIG. 2 is a conceptual diagram illustrating gait events detected in a stride cycle based on the right foot. The horizontal axis in FIG. 2 represents a normalized gait cycle, with one gait cycle of the right foot, starting from the point when the heel of the right foot hits the ground and ending at the point when the heel of the right foot hits the ground, being 100 percent (%). Each of the multiple timings included in one gait cycle is called a gait phase. One gait cycle of one foot is broadly divided into a stance phase, in which at least a portion of the sole of the foot is in contact with the ground, and a swing phase, in which the sole of the foot is off the ground. In the example of FIG. 2, the gait cycle is normalized so that the stance phase accounts for 60% and the swing phase accounts for 40%. The stance phase is further divided into an early stance phase T1, a mid-stance phase T2, a late stance phase T3, and an early swing phase T4. The swing phase is further divided into an early swing phase T5, a mid-swing phase T6, and a late swing phase T7. The starting point of the walking waveform in a step cycle does not have to be the point at which the heel hits the ground. For example, the starting point of the walking waveform in a step cycle may be set to the center of the stance phase.

[0019] Gait event E1 represents heel strike (HS) at the beginning of a stride cycle. Heel strike is an event in which the heel of the right foot, which was off the ground during the swing phase, lands on the ground. Gait event E2 represents opposite toe off (OTO). Opposite toe off is an event in which the toe of the left foot leaves the ground while the sole of the right foot is in contact with the ground. Gait event E3 represents heel rise (HR). Heel rise is an event in which the right heel rises while the sole of the right foot is in contact with the ground. Gait event E4 represents opposite heel strike (OHS). Opposite heel strike is an event in which the heel of the left foot, which was off the ground during the swing phase of the left foot, lands on the ground. Gait event E5 represents toe off (TO). Toe off is an event in which the toe of the right foot leaves the ground while the sole of the left foot is in contact with the ground. Gait event E6 represents foot adjacent (FA). Foot crossing is an event in which the left and right feet cross while the sole of the left foot is in contact with the ground. Gait event E7 represents tibia vertical (TV). Tibia vertical is an event in which the tibia of the right foot becomes nearly perpendicular to the ground while the sole of the left foot is in contact with the ground. Gait event E8 represents heel strike (HS) at the end of a stride cycle. Gait event E8 corresponds to the end of the gait cycle that began with gait event E1 and also corresponds to the starting point of the next gait cycle.

[0020] The waist position information is a time change in the position information of the waist in the direction of travel. In this embodiment, the forward direction in the direction of travel is defined as positive, and the backward direction in the direction of travel is defined as negative. There are no particular limitations on the method for measuring the waist position information.

[0021] For example, waist position information is measured by motion capture. In motion capture, markers are attached to various parts of the subject's body. For example, the markers are attached to parts including the waist. The subject is photographed with a camera as they walk, and the positions of the markers in the captured image (video) are measured. Motion capture allows the waist position to be measured directly, so highly accurate waist position information can be obtained.

[0022] For example, waist position information can be measured by analyzing images (video) captured by a camera. Using software such as OpenPose, waist position information can be measured by calculating the waist position based on the positions of the bones and joints detected from the person in the image.

[0023] For example, the waist position information is measured using acceleration and angular velocity measured by an inertial sensor attached to the waist. When an inertial sensor is used, the waist position can be calculated by integrating the acceleration and angular velocity. For example, the waist position information may be measured using smart apparel in which inertial sensors are attached to each part of the body.

[0024] FIG. 3 is a conceptual diagram showing an example in which a walking subject is displayed on screen 100. The subject is displayed in the center of screen 100. The viewpoint (tracking center FC) of virtual camera 150 that tracks the subject is set to a position that captures the center of screen 100. The subject's waist position LP is fixed to the center of screen 100. In the example of FIG. 3, the tracking center FC and waist position LP are positioned to coincide with each other.

[0025] Fig. 4 is a conceptual diagram showing an example of changes in an image in response to the walking of a subject tracked by virtual camera 150 installed as in Fig. 3. The viewpoint (tracking center FC) of virtual camera 150 is fixed toward the center of screen 100. Therefore, the waist position LP of the subject hardly changes, and only the movements of the feet and hands relative to the waist position LP are displayed.

[0026] The lumbar sway calculation unit 12 acquires lumbar position information of the subject from the acquisition unit 11. The lumbar sway calculation unit 12 uses the acquired lumbar position information to calculate the position difference (also referred to as lumbar sway) between the subject's position corresponding to the average speed and the lumbar position. The lumbar sway is the position difference corresponding to the movement of the lumbar in a predetermined walking section. The lumbar sway calculation unit 12 calculates the lumbar sway using the walking speed in the predetermined walking section as a reference. For example, the lumbar sway calculation unit 12 assumes that the walking speed in the predetermined walking section is uniform linear movement, and approximates the walking speed in the predetermined walking section with a straight line (also referred to as a reference line). For each of a plurality of walking phases included in the predetermined walking section, the lumbar sway calculation unit 12 calculates the distance between the lumbar position and the reference line as the lumbar sway for that walking phase.

[0027] Lumbar sway calculation unit 12 may calculate lumbar sway by approximating the walking speed in a predetermined walking section with a curve. For example, lumbar sway calculation unit 12 approximates the walking speed in a predetermined walking section with a curve (also called a reference curve). For example, lumbar sway calculation unit 12 sets a reference curve that smoothly connects the start point and end point of time-series data of lumbar position in a predetermined walking section. For example, lumbar sway calculation unit 12 sets a Bezier curve, a spline curve, or the like that smoothly connects the start point and end point as the reference curve. For each of multiple walking phases included in the predetermined walking section, lumbar sway calculation unit 12 calculates the distance between the lumbar position and the reference curve as the lumbar sway for that walking phase. The reference straight line and the reference curve are collectively called the reference line.

[0028] Lumbar sway calculation unit 12 may calculate lumbar sway by dividing the time-series data of lumbar position in a walking cycle into multiple sections extracted from the data. For example, lumbar sway calculation unit 12 sets a reference line or a reference curve for each of the multiple sections. Lumbar sway calculation unit 12 calculates the distance between the reference line or the reference curve set for each of the multiple sections and the lumbar position for each walking phase as the lumbar sway for that walking phase.

[0029] FIG. 5 is a graph illustrating an example of time-series data of the lumbar position in the direction of travel. The graph in FIG. 5 shows time-series data C of the lumbar position in the direction of travel during a walking cycle. The lumbar position changes at different speeds depending on the walking cycle. FIG. 5 also shows a reference line S obtained by approximating the time-series data C of the lumbar position during a walking cycle with a linear function. For example, the reference line S is a regression line of the time-series data C of the lumbar position during a walking cycle. The lumbar sway calculation unit 12 calculates the distance between each point of the time-series data C of the lumbar position during a walking cycle and the reference line S as the lumbar sway. The lumbar sway is an index of the lumbar position based on the position of the subject corresponding to the average speed. When the lumbar sway is positive, the lumbar position is located forward of the position of the subject corresponding to the average speed. When the lumbar sway is negative, the lumbar position is located behind the position of the subject corresponding to the average speed. Using the lumbar sway makes it possible to understand the subject's movements, which cannot be understood from foot movements alone.

[0030] FIG. 6 is a conceptual diagram emphasizing lumbar sway in the graph of FIG. 5. Point P(x, y) in the lumbar position time-series data C indicates the lumbar position in walking phase x. The length of the perpendicular line drawn from point P(x, y) in the lumbar position time-series data C to the reference line S is the lumbar sway Dx in walking phase x. When the lumbar position of the subject is forward of the position of the subject corresponding to the average speed in a predetermined walking section, the lumbar sway Dx is positive. When the lumbar position of the subject is behind the position of the subject corresponding to the average speed in a predetermined walking section, the lumbar sway Dx is negative. The farther the lumbar position is from the position of the subject corresponding to the average speed, the larger the absolute value of the lumbar sway Dx. The closer the lumbar position is from the position of the subject corresponding to the average speed, the smaller the absolute value of the lumbar sway Dx.

[0031] Gait information generation unit 15 acquires the lumbar sway calculated by lumbar sway calculation unit 12. Gait information generation unit 15 generates gait information according to the lumbar sway calculated by lumbar sway calculation unit 12. For example, gait information generation unit 15 generates time-series data of lumbar sway as the gait information. For example, gait information generation unit 15 generates a graph showing the time-series data of lumbar sway as the gait information. The gait information generated by gait information generation unit 15 is not limited to time-series data or graphs of lumbar sway.

[0032] FIG. 7 is a graph showing an example of gait information generated by the gait information generating unit 15. FIG. 7 is an example in which time-series data of lumbar sway is displayed in association with a walking cycle. The graph in FIG. 7 allows intuitive understanding of fluctuations in lumbar sway in association with the walking cycle. In other words, the graph in FIG. 7 allows understanding of the movement of the subject while walking, which cannot be understood from the movement of the feet alone.

[0033] The output unit 17 outputs the gait information generated by the gait information generation unit 15. For example, the output unit 17 outputs the gait information to a terminal device having a screen. The gait information output to the terminal device is displayed on the screen of the terminal device. For example, the output unit 17 displays the gait information on the screen of a mobile terminal of a subject (user). For example, the output unit 17 displays the gait information on the screen of a terminal device used by a specialist, such as a doctor, physical therapist, or care worker, who examines the physical condition of the subject. The specialist can provide the subject with a diagnosis or advice based on the gait information displayed on the screen of the terminal device. For example, the output unit 17 may output the gait information to an external system that uses the gait information. There are no particular limitations on the use of the gait information output from the output unit 17.

[0034] For example, the gait information generating device 10 is connected to an external system built on a cloud or a server via a mobile terminal (not shown) carried by a subject (user). The mobile terminal is a portable communication device. For example, the mobile terminal is a portable communication device having a communication function, such as a smartphone, a smartwatch, or a mobile phone.

[0035] For example, the gait information generating device 10 is connected to a terminal device (not shown) used by a person who verifies the physical condition of a subject (user). Software for processing gait information and displaying images corresponding to the gait information is installed in the terminal device. For example, the terminal device is an information processing device such as a desktop personal computer, a notebook personal computer, a tablet, or a mobile terminal. The terminal device may be a dedicated terminal for processing gait information.

[0036] For example, the gait information generation device 10 is connected to a mobile terminal or a terminal device via a wire such as a cable. For example, the gait information generation device 10 is connected to a mobile terminal or a terminal device via wireless communication. For example, the gait information generation device 10 is connected to a mobile terminal or a terminal device via a wireless communication function (not shown) conforming to standards such as Bluetooth (registered trademark) or WiFi (registered trademark). Note that the communication function of the gait information generation device 10 may conform to standards other than Bluetooth (registered trademark) or WiFi (registered trademark). The gait information may be used by an application installed on the mobile terminal or a terminal device. In that case, the mobile terminal or the terminal device executes processing using the gait information by application software or the like installed on the device. Furthermore, the gait information generation device 10 may be implemented in the mobile terminal or the terminal device.

[0037] FIG. 8 is a conceptual diagram showing an example in which gait information output from gait information generating device 10 is superimposed on a video showing a walking subject. In the example of FIG. 8, a graph showing time-series data of lumbar sway is superimposed on a video showing a walking subject. In the example of FIG. 8, the subject's lumbar region is displayed in the center of screen 100. A graph showing time-series data of lumbar sway is displayed in the upper right corner of screen 100. The display position of the graph showing time-series data of lumbar sway may be in an area other than the upper right corner of screen 100. The graph showing time-series data of lumbar sway shows how indicator I (vertical line) is displayed at a position that matches the walking phase of the subject.

[0038] As shown in FIG. 8, if a graph of lumbar movement is displayed on the screen 100, it is easy to intuitively grasp the lumbar movement of the subject in accordance with the walking of the subject displayed in the center of the screen 100. For example, an actually captured image may be used for the image. For example, the image may include a virtual person (character) that moves in accordance with the walking of the subject. Furthermore, the image may not show the entire body of the subject, but only the part of the subject from the waist down (lower body).

[0039] Fig. 9 is a graph illustrating an example of time-series data of waist position in the direction of travel for multiple walking cycles. Fig. 9 shows time-series data C1 to C5 of waist position in the direction of travel for multiple consecutive walking cycles. The order of waist position over time is time-series data C1, time-series data C2, time-series data C3, time-series data C4, and time-series data C5. For consecutive waist position time-series data, the waist position at the end point of the preceding time-series data matches the waist position at the start point of the following time-series data. The shape of the waist position time-series data C1 to C5 changes depending on the walking cycle.

[0040] FIG. 10 is a graph showing an example of gait information generated based on the time-series data of waist position in FIG. 9. FIG. 10 is an example in which time-series data of waist sway over multiple gait cycles is displayed side by side in association with the gait cycles. The graph in FIG. 10 allows intuitive understanding of changes in waist sway over multiple gait cycles in association with the gait cycles. The graph in FIG. 10 allows verification of the subject's movement over multiple gait cycles. For example, statistical values ​​such as the arithmetic mean, geometric mean, variance, and standard deviation of waist sway over multiple gait cycles may be derived. Using average values ​​such as the arithmetic mean and geometric mean of waist sway over multiple gait cycles allows understanding of waist movement over multiple gait cycles on an average basis. Using the variance and standard deviation of waist sway over multiple gait cycles allows understanding of fluctuations in waist movement over multiple gait cycles.

[0041] (operation) Next, an example of the operation of the gait information generating device 10 will be described with reference to the drawings. Fig. 11 is a flowchart for explaining an example of the operation of the gait information generating device 10. In the description following the flowchart of Fig. 11, the gait information generating device 10 will be the subject of the operation.

[0042] 11, first, the gait information generation device 10 acquires waist position information in a predetermined walking cycle (step S11). For example, the gait information generation device 10 acquires waist position information in a one-step walking cycle. The gait information generation device 10 may acquire waist position information in a plurality of walking cycles.

[0043] Next, the gait information generation device 10 derives a reference line related to the waist position in a predetermined walking cycle (step S12). For example, the gait information generation device 10 derives a reference line related to the waist position in one step walking cycle. The gait information generation device 10 may derive reference lines related to the waist position in multiple walking cycles.

[0044] Next, the gait information generating device 10 calculates the distance between the waist position and the reference line (waist sway) for each walking cycle (step S13).

[0045] Next, the gait information generating device 10 generates gait information related to the calculated waist sway (step S14). For example, the gait information generating device 10 generates gait information including time-series data of waist sway or a graph of the time-series data of waist sway.

[0046] Next, the gait information generating device 10 outputs the generated gait information (step S15). For example, the gait information generating device 10 outputs gait information including time-series data of waist sway or a graph of the time-series data of waist sway.

[0047] (Application example) Next, application examples of the gait information generating device 10 will be described with reference to the drawings. Here, an example will be given in which gait information output from the gait information generating device 10 is displayed on the screen of a terminal device. In the following application examples, an example will be given in which gait information is displayed in a video of a walking subject. The video of the subject may be an actual video, or may be a virtual person (character) that imitates the subject's movements. In the following, an example will be given in which a character is displayed in a video. The display information shown in the following application examples may be generated by the gait information generating device 10, or may be generated by another device or system that acquires the gait information.

[0048] [Application Example 1-1] FIG. 12 is a conceptual diagram of Application Example 1-1 related to the gait information generating device 10. In Application Example 1-1, a graph of time-series data of lumbar sway is displayed in a frame constituting a video of a subject's walking as gait information output from the gait information generating device 10. In the example of FIG. 12, the graph of time-series data of lumbar sway is displayed in the upper right region of the screen. The graph of time-series data of lumbar sway may be displayed in a region other than the upper right region of the screen.

[0049] Figure 12 shows three frames extracted from multiple frames included in a video of a subject walking. An actual video is made up of many more frames. Time (walking cycle) progresses from the upper left to the lower right in the three frames in Figure 12. In the graph of the time-series data of lumbar sway displayed in each frame, indicator I (vertical line) is displayed at a position that matches the walking phase of the subject. Indicator I moves in the direction shown by the arrow in accordance with the walking phase of the subject.

[0050] In this application example, a graph of time-series data of lumbar sway is displayed in association with the walking phase of the subject. According to this application example, it is possible to intuitively grasp the gait based on the graph of time-series data of lumbar sway associated with the walking of the subject. In particular, according to this application example, it is possible to intuitively grasp the gait according to the movement of indicator I displayed on the graph of time-series data of lumbar sway.

[0051] [Application Example 1-2] FIG. 13 is a conceptual diagram of Application Example 1-2 related to the gait information generating device 10. In Application Example 1-2, a graph of time-series data of lumbar sway in a plurality of walking cycles is displayed in frames constituting a video of a subject's walking as gait information output from the gait information generating device 10. In the example of FIG. 13, the graph of time-series data of lumbar sway in a plurality of walking cycles is displayed in the upper right region of the screen. The graph of time-series data of lumbar sway in a plurality of walking cycles may be displayed in a region other than the upper right region of the screen.

[0052] FIG. 13 shows one frame extracted from multiple frames included in a video of a subject walking. An actual video is made up of multiple frames. In the graph of the time-series data of lumbar sway displayed in the frame, indicator I (vertical line) is displayed at a position that matches the subject's walking phase. Indicator I moves in the direction shown by the arrow in accordance with the subject's walking phase.

[0053] FIG. 13 displays information according to fluctuations in time-series data of lumbar sway over multiple walking cycles. FIG. 13 shows an example in which the variance and standard deviation of lumbar sway over multiple walking cycles increase with walking. On screen 100, information such as "Your walking is becoming unstable" is displayed in accordance with the increase in the variance and standard deviation of lumbar sway. For example, when the subject's walking is displayed on screen 100 in real time, recommendation information for the subject may be displayed in accordance with the increase in the variance and standard deviation of lumbar sway. For example, recommendation information recommending a break or correcting one's walking may be displayed in accordance with the increase in the variance and standard deviation of lumbar sway.

[0054] In this application example, a graph of time-series data of lumbar sway in multiple walking cycles is displayed in association with the walking phases of the subject. According to this application example, it is possible to intuitively grasp the gait over multiple walking cycles based on the graph of time-series data of lumbar sway in multiple walking cycles associated with the walking of the subject. Furthermore, according to this application example, by displaying information according to fluctuations in the variance and standard deviation of lumbar sway, it is possible to intuitively grasp changes in gait as walking continues.

[0055] [Application Example 1-3] 14 to 15 are conceptual diagrams relating to Application Example 1-3 related to the gait information generating device 10. In Application Example 1-3, the viewpoint of a virtual camera set for a character in a frame is switched in response to an operation by the subject. Also, in Application Example 1-3, a graph of time-series data of lumbar sway is displayed in a frame constituting a video relating to the walking of the subject as gait information output from the gait information generating device 10. In FIGS. 14 to 15, the graph of time-series data of lumbar sway is displayed in the upper right region of the screen. Graphs of time-series data of lumbar sway for multiple walking cycles may be displayed in a region other than the upper right region of the screen. In FIGS. 14 to 15, the orientation of the graph is changed to match the viewpoint of the virtual camera.

[0056] 14 and 15, a button for changing the viewpoint of the virtual camera is displayed in the upper left of the screen. In the examples of FIGS. 14 and 15, the viewpoint of the virtual camera can be set to backward RE, left L, right R, upward U, and forward FR. The position where the viewpoint of the virtual camera is set and the arrangement of the buttons are not limited to the examples of FIGS. 14 and 15.

[0057] FIG. 14 shows an example in which a button for changing the virtual camera's viewpoint to the left L is selected when the viewpoint is set to the right R (upper left). In response to the selection of the button for changing the virtual camera's viewpoint to the left L, the viewpoint of the virtual camera changes to a state in which it is set to the left L (lower right). In the example of FIG. 14, the direction of the horizontal axis of the graph is reversed to match the subject's direction of travel. In the example of FIG. 14, the direction in which the indicator I (vertical line) corresponding to the walking phase moves is also reversed to match the subject's direction of travel. In response to the selection of the button for changing the virtual camera's viewpoint, the display positions of the button and graph may be changed.

[0058] FIG. 15 shows an example in which a button for changing the virtual camera's viewpoint to an upward U is selected when the viewpoint is set to the right R (upper left). In response to the selection of the button for changing the virtual camera's viewpoint to an upward U, the viewpoint of the virtual camera changes to a state set to an upward U (lower right). In the example of FIG. 15, the direction of travel of the subject on the screen 100 does not change, so the orientation of the horizontal axis of the graph does not change. In the example of FIG. 15, the direction in which the indicator I (vertical line) corresponding to the walking phase moves does not change. In response to the selection of the button for changing the viewpoint of the virtual camera, the display positions of the buttons and graphs may be changed.

[0059] In this application example, the viewpoint of the virtual camera is changed in response to an operation by the subject. According to this application example, by changing the viewpoint of the virtual camera, walking can be verified from multiple viewpoints, and gait can be grasped more intuitively.

[0060] As described above, the gait information generation device of this embodiment includes an acquisition unit, a lumbar sway calculation unit, a gait information generation unit, and an output unit. The acquisition unit acquires lumbar position information including time-series data of the subject's lumbar position in a predetermined walking cycle. The lumbar sway calculation unit calculates lumbar sway for multiple walking phases included in the predetermined walking cycle. The lumbar support corresponds to the distance between a reference line set for the time-series data of the subject's lumbar position and the lumbar position in each of the multiple walking phases. The gait information generation unit generates gait information corresponding to the lumbar sway calculated for the predetermined walking cycle. The output unit outputs the generated gait information.

[0061] If gait information (display information) is displayed as a video in conjunction with the subject's walking, it becomes easier to intuitively grasp the gait. For example, if the viewpoint (tracking center) of the virtual camera is made to track the subject's waist, the gait can be intuitively grasped from the subject's foot movements. However, when the tracking center is made to track the subject's waist, although the foot movements can be intuitively grasped, it is not possible to intuitively grasp the acceleration and deceleration of the walking.

[0062] In this embodiment, gait information is generated according to the lumbar sway of the subject. Therefore, according to this embodiment, the gait of the subject can be intuitively grasped from the gait information including information about the lumbar sway that varies according to the walking of the subject.

[0063] In one aspect of this embodiment, the acquisition unit acquires lumbar position information including time-series data of the subject's lumbar position during a gait cycle. The lumbar sway calculation unit calculates lumbar sway corresponding to the distance between a reference line set for the time-series data of the subject's lumbar position and the lumbar position during each of the multiple walking phases included in the gait cycle. The gait information generation unit generates gait information including information corresponding to the variation in lumbar sway calculated for the lumbar sway cycle. According to this aspect, the gait of the subject during the gait cycle can be intuitively grasped using the gait information corresponding to the lumbar sway during the gait cycle.

[0064] In one aspect of this embodiment, the acquisition unit acquires lumbar position information including time-series data of the subject's lumbar position in multiple gait cycles. The lumbar sway calculation unit calculates, for multiple walking phases included in the multiple walking cycles, lumbar sway corresponding to the distance between a reference line set for the time-series data of the subject's lumbar position in each of the multiple walking phases and the lumbar position in each of the multiple walking phases. The gait information generation unit generates gait information including information corresponding to the fluctuations in lumbar sway calculated for each of the multiple walking cycles. According to this aspect, the gait information corresponding to the lumbar sway in the multiple walking cycles allows intuitive understanding of the fluctuations in the subject's gait in those walking cycles.

[0065] In one aspect of this embodiment, the gait information generator generates, as gait information, display information in which time-series data of lumbar sway in a predetermined walking cycle is superimposed on frames constituting a video showing the walking state of the subject. According to this aspect, the time-series data of lumbar sway displayed in the frames constituting the video allows the subject's gait to be more intuitively grasped in accordance with the video of the subject walking.

[0066] In one aspect of the present embodiment, the gait information generator generates display information in which indicators associated with the walking phases of the subject are superimposed on time-series data of waist sway in a predetermined walking cycle. According to this aspect, the indicators superimposed on the time-series data of waist sway allow the subject's gait to be more intuitively understood in accordance with the video of the subject walking.

[0067] In this embodiment, an example is given in which gait information regarding waist sway corresponding to the movement of a person (subject) in the real world is generated. The method of this embodiment may also be applied to generating gait information regarding waist sway corresponding to the movement of a virtual person, such as an avatar, in a virtual world. If the avatar is made to move in accordance with waist sway, the movement of the avatar can be expressed more realistically. Furthermore, the method of this embodiment may also be applied to generating gait information regarding the movement of a body part other than the waist. Furthermore, the method of this embodiment may also be used to verify waist sway other than walking.

[0068] (Second embodiment) Next, a gait information generating device according to a second embodiment will be described with reference to the drawings. This embodiment differs from the first embodiment in that it calculates velocity information according to lumbar sway. In the following, descriptions of configurations and functions similar to those of the first embodiment may be omitted.

[0069] (composition) 16 is a block diagram showing the configuration of a gait information generation device 20 according to this embodiment. The gait information generation device 20 includes an acquisition unit 21, a waist sway calculation unit 22, a speed information calculation unit 23, a gait information generation unit 25, and an output unit 27.

[0070] The acquisition unit 21 has the same configuration as the acquisition unit 11 of the first embodiment. The acquisition unit 21 acquires waist position information in the travel direction of the subject. The acquisition unit 21 acquires waist position information in a predetermined walking section.

[0071] Lumbar sway calculation unit 22 has the same configuration as lumbar sway calculation unit 12 of the first embodiment. Lumbar sway calculation unit 22 acquires lumbar position information of the subject from acquisition unit 21. Lumbar sway calculation unit 22 uses the acquired lumbar position information to calculate the position difference (lumbar sway) between the subject's position corresponding to the average speed and the lumbar position. Lumbar sway calculation unit 22 calculates lumbar sway using the subject's position corresponding to the average speed in a predetermined walking section as a reference.

[0072] The speed information calculation unit 23 acquires the lumbar sway calculated by the lumbar sway calculation unit 22. The speed information calculation unit 23 calculates speed information corresponding to the acquired lumbar sway. For example, the speed information calculation unit 23 calculates speed information corresponding to the slope (speed) of the tangent to the curve representing the time-series data of lumbar sway in each walking phase. For example, the speed information calculation unit 23 calculates speed information corresponding to the rate of change (speed) of lumbar sway in a small section of the curve representing the time-series data of lumbar sway.

[0073] FIG. 17 is a graph for explaining the velocity information calculated by the velocity information calculation unit 23. For example, the velocity information calculation unit 23 calculates the gradient of the tangent Tx1 at point P1 (x1, y1) in walking phase x1 for the curve showing the time series data of lumbar sway as the velocity of lumbar sway. For example, the velocity information calculation unit 23 calculates the rate of change of lumbar sway in the small section between point P2 (x2, y2) in walking phase x2 and point P3 (x3, y3) in walking phase x3 for the curve showing the time series data of lumbar sway as the velocity of lumbar sway. 23 The rate of change of the lumbar sway in the small section between point P2 (x2, y2) and point P 23 This corresponds to the slope of the line Tx2 that passes through (x3, y3).

[0074] Gait information generation unit 25 acquires speed information related to lumbar sway calculated by speed information calculation unit 23. Gait information generation unit 25 generates gait information according to the speed information related to lumbar sway. For example, gait information generation unit 25 generates time-series data in which the direction and magnitude of speed related to lumbar sway are associated with walking phases as gait information. The gait information may include information related to lumbar sway calculated by lumbar sway calculation unit 22. In that case, gait information generation unit 25 acquires the lumbar sway calculated by lumbar sway calculation unit 22. For example, gait information generation unit 25 generates gait information including time-series data of lumbar sway and a graph showing the time-series data of lumbar sway. The gait information generated by gait information generation unit 25 is not limited to the direction and magnitude of speed related to lumbar sway, or the time-series data and graph of lumbar sway.

[0075] The output unit 27 has the same configuration as the output unit 17 of the first embodiment. The output unit 27 outputs the gait information generated by the gait information generation unit 25. There are no particular limitations on how the gait information output from the output unit 27 is used.

[0076] FIG. 18 is a conceptual diagram showing an example in which gait information output from gait information generating device 20 is superimposed on a video image showing a walking subject. FIG. 18 shows an example in which arrows (also referred to as indicators) indicating the direction and magnitude of the speed of lumbar sway are displayed in association with the subject on screen 200. In the example of FIG. 18, the subject's lumbar region is displayed in the center of screen 200. As in the first embodiment, a graph showing time-series data of lumbar sway is displayed in the upper right corner of screen 200. The display position of the graph showing time-series data of lumbar sway may be in an area other than the upper right corner of screen 200. The graph showing time-series data of lumbar sway shows how indicator I (vertical line) is displayed at a position that matches the walking phase of the subject. The graph showing time-series data of lumbar sway may be omitted. Furthermore, a graph showing time-series data of lumbar sway speed may be displayed on screen 200.

[0077] 18, by displaying arrows on the screen 200 indicating the direction and magnitude of the velocity related to lumbar movement, it becomes easier to intuitively grasp the velocity of the lumbar movement of the subject in accordance with the walking of the subject displayed in the center of the screen 200. The direction and magnitude of the velocity of lumbar movement may be expressed by the thickness, color, shading, pattern, etc. of the arrow. By expressing the direction and magnitude of the velocity of lumbar movement by the thickness, color, shading, pattern, etc. of the arrow, it becomes easier to visually grasp the fluctuations in lumbar movement.

[0078] (operation) Next, an example of the operation of the gait information generator 20 will be described with reference to the drawings. Fig. 19 is a flowchart for explaining an example of the operation of the gait information generator 20. In the description following the flowchart of Fig. 19, the gait information generator 20 will be the subject of the operation.

[0079] 19, first, the gait information generation device 20 acquires waist position information in a predetermined walking cycle (step S21). For example, the gait information generation device 20 acquires waist position information in a one-step walking cycle. The gait information generation device 20 may acquire waist position information in a plurality of walking cycles.

[0080] Next, the gait information generation device 20 derives a reference line related to the waist position in a predetermined walking cycle (step S22). For example, the gait information generation device 20 derives a reference line related to the waist position in one step walking cycle. The gait information generation device 20 may derive reference lines related to the waist position in multiple walking cycles.

[0081] Next, the gait information generating device 20 calculates the distance between the waist position and the reference line (waist sway) for each walking cycle (step S23).

[0082] Next, the gait information generating device 20 generates velocity information corresponding to the calculated lumbar sway (step S24). For example, the gait information generating device 20 generates velocity information indicating the magnitude and direction of the velocity related to the lumbar sway.

[0083] Next, the gait information generating device 20 generates gait information related to the calculated waist sway and velocity information (step S25). For example, the gait information generating device 20 generates gait information including arrows (indicators) indicating the magnitude and direction of velocity related to waist sway. For example, the gait information generating device 20 generates gait information including time-series data of waist sway and a graph of the time-series data of waist sway.

[0084] Next, the gait information generating device 20 outputs the generated gait information (step S26). For example, the gait information generating device 20 outputs gait information including an arrow (indicator) indicating the magnitude and direction of the velocity of waist sway, or a graph of time-series data of waist sway.

[0085] (Application example) Next, application examples of the gait information generating device 20 will be described with reference to the drawings. Here, an example will be given in which gait information output from the gait information generating device 20 is displayed on the screen of a terminal device. In the following application examples, an example will be given in which gait information is displayed in a video of a walking subject. The video of the subject may be an actual video, or may be a virtual person (character). In the following, an example will be given in which a character is displayed in a video. The display information shown in the following application examples may be generated by the gait information generating device 20, or may be generated by another device or system that acquires the gait information.

[0086] [Application Example 2-1] FIG. 20 is a conceptual diagram of Application Example 2-1 related to the gait information generating device 20. In Application Example 2-1, an arrow corresponding to lumbar sway velocity information is displayed in a frame constituting a video of a subject walking as gait information output from the gait information generating device 20. The arrow corresponding to the lumbar sway velocity information is displayed in accordance with the lumbar position of the character in the frame. The arrow corresponding to the lumbar sway velocity information may be displayed in a position different from the lumbar position. For example, the arrow corresponding to the lumbar sway velocity information may be displayed in front of or behind the character in the frame depending on the direction of the lumbar sway velocity. Furthermore, in the example of FIG. 20 , as in the first embodiment, a graph of time-series data of lumbar sway is displayed in the upper right region of the frame. The graph of time-series data of lumbar sway may be omitted. A graph of time-series data related to the lumbar sway velocity may also be displayed in the frame.

[0087] FIG. 20 shows three frames extracted from multiple frames included in a video of a subject walking. An actual video is composed of many more frames. In the three frames of FIG. 20, time (walking cycle) progresses from the upper left to the lower right. An arrow corresponding to the lumbar sway velocity information is displayed at the position of the character's waist in the frame. The arrow has a shape that reflects the direction and magnitude of the lumbar sway velocity. The direction of the arrow is oriented in the direction of the lumbar sway velocity. The size of the arrow is set to a length corresponding to the lumbar sway velocity. The greater the lumbar sway velocity, the longer the arrow, and the slower the lumbar sway velocity, the shorter the arrow. In addition, an indicator I (vertical line) is displayed in the graph of the lumbar sway time series data displayed in the frame at a position that corresponds to the subject's walking phase. Indicator I moves in the direction indicated by the arrow in accordance with the subject's walking phase. The arrow displayed at the position of the character's waist in the frame indicates the lumbar sway velocity information indicated by indicator I in the graph.

[0088] In the upper left frame, the velocity of lumbar movement is positive, so the arrow corresponding to the velocity information of lumbar movement is pointing forward. In the middle frame, the velocity of lumbar movement is negative, so the arrow corresponding to the velocity information of lumbar movement is pointing backward. The directions of lumbar movement are opposite in the upper left and middle frames, so the arrows are pointing in opposite directions. Furthermore, the absolute value of the velocity of lumbar movement is larger in the middle frame than in the upper left frame. Therefore, the length of the arrow is longer in the middle frame than in the upper left frame. In the lower right frame, the velocity of lumbar movement is positive, so the arrow corresponding to the velocity information of lumbar movement is pointing forward. The velocity of lumbar movement is smaller in the lower right frame than in the upper left frame. Therefore, the length of the arrow is shorter in the lower right frame than in the upper left frame. If the velocity of lumbar movement is zero, the arrow corresponding to the velocity information of lumbar movement is not displayed. If the velocity of lumbar movement is zero, a symbol or shape other than an arrow may be displayed.

[0089] In this application example, an arrow corresponding to the velocity information of lumbar sway is displayed in association with the walking phase of the subject. According to this application example, the gait of the subject can be intuitively grasped based on the arrow corresponding to the velocity information of lumbar sway associated with the walking of the subject. In particular, according to this application example, the gait can be intuitively grasped based on changes in the direction and length of the arrow corresponding to the velocity information of lumbar sway.

[0090] As described above, the gait information generating device of this embodiment includes an acquisition unit, a lumbar sway calculation unit, a speed information generation unit, a gait information generation unit, and an output unit. The acquisition unit acquires lumbar position information including time-series data of the lumbar position of the subject in a predetermined walking cycle. The lumbar sway calculation unit calculates lumbar sway corresponding to the distance between a reference line set for the time-series data of the lumbar position of the subject and the lumbar position in each of the plurality of walking phases included in the predetermined walking cycle. The speed information generation unit generates speed information corresponding to the direction and magnitude of the speed of lumbar sway. The gait information generation unit generates gait information corresponding to the lumbar sway calculated for the predetermined walking cycle. The gait information generation unit generates, as gait information, display information in which arrows representing the direction and magnitude of the speed of lumbar sway are superimposed on frames constituting a video showing the walking state of the subject. The output unit outputs the gait information including the generated display information.

[0091] In this embodiment, gait information is generated for a subject, including an arrow corresponding to the direction and magnitude of the velocity of waist sway. Therefore, according to this embodiment, the gait of the subject can be more intuitively understood by the arrow corresponding to the velocity of waist sway caused by the subject's walking.

[0092] (Third embodiment) Next, a gait information generating device according to a third embodiment will be described with reference to the drawings. This embodiment differs from the first and second embodiments in that the viewpoint (center of tracking) of the virtual camera is changed in accordance with waist sway. In the following, an example will be described in which a configuration for changing the center of tracking of the virtual camera is added to the configuration of the first embodiment. The configuration for changing the center of tracking of the virtual camera may be added to the configuration of the second embodiment. In the following, descriptions of configurations and functions similar to those of the first and second embodiments may be omitted.

[0093] (composition) 21 is a block diagram showing the configuration of a gait information generation device 30 according to this embodiment. Gait information generation device 30 includes an acquisition unit 31, a waist sway calculation unit 32, a following center calculation unit 34, a gait information generation unit 35, and an output unit 37.

[0094] The acquisition unit 31 has the same configuration as the acquisition unit 11 of the first embodiment. The acquisition unit 31 acquires waist position information in the travel direction of the subject. The acquisition unit 31 acquires waist position information in a predetermined walking section.

[0095] Lumbar sway calculation unit 32 has the same configuration as lumbar sway calculation unit 12 of the first embodiment. Lumbar sway calculation unit 32 acquires lumbar position information of the subject from acquisition unit 31. Lumbar sway calculation unit 32 uses the acquired lumbar position information to calculate the position difference (lumbar sway) between the subject's position corresponding to the average speed and the lumbar position. Lumbar sway calculation unit 32 calculates lumbar sway using the subject's position corresponding to the average speed in a predetermined walking section as a reference.

[0096] The tracking center calculation unit 34 acquires the lumbar sway calculated by the lumbar sway calculation unit 32. The tracking center calculation unit 34 calculates the position of the viewpoint (tracking center) of the virtual camera according to the acquired lumbar sway. The tracking center corresponds to the center of multiple frames that make up the video. The tracking center is set at the center of the screen that displays the video related to the subject's gait. The difference between the lumbar position and the tracking center corresponds to the lumbar sway.

[0097] Fig. 22 is a conceptual diagram showing an example in which a walking subject is displayed on screen 300. The viewpoint (tracking center FC) of virtual camera 350 that tracks the subject is set to a position capturing the center of screen 300. On screen 300, the subject is displayed at a position according to lumbar fluctuation Dx, which is the difference between lumbar position LP and tracking center FC. In the example of Fig. 22, the tracking center FC and lumbar position LP are at different positions.

[0098] The gait information generator 35 generates gait information according to the difference between the position of the following center calculated by the following center calculator 34 and the lumbar sway calculated by the lumbar sway calculator 32. For example, the gait information generator 35 generates display information in which the lumbar position is set at a position shifted from the following center by the amount of lumbar sway. The gait information generator 35 may generate gait information including time-series data of lumbar sway or a graph showing the time-series data of lumbar sway. The gait information generator 35 may also generate gait information including time-series data in which the direction and magnitude of velocity related to lumbar sway are associated with walking phases. The gait information generated by the gait information generator 35 is not limited to display information reflecting the difference between the lumbar position and the following center (lumbar sway), the direction and magnitude of velocity related to lumbar sway, or the time-series data or graph of lumbar sway.

[0099] The output unit 37 has the same configuration as the output unit 17 of the first embodiment. The output unit 37 outputs the gait information generated by the gait information generation unit 35. There are no particular limitations on how the gait information output from the output unit 37 is used.

[0100] FIG. 23 is a conceptual diagram for explaining the display position of the subject according to lumbar movement. When lumbar movement Dx is positive (left), the tracking center FC lags behind the lumbar position LP. Therefore, when lumbar movement Dx is positive (left), the subject is displayed at a position shifted to the right from the center of the screen 300 (tracking center FC) by the absolute value |Dx| of lumbar movement Dx. When lumbar movement Dx is zero (center), the lumbar position LP and the tracking center FC coincide. Therefore, when lumbar movement Dx is zero (center), the subject is displayed at a position in the center of the screen 300 where the lumbar position LP and the tracking center FC overlap. When lumbar movement Dx is negative (right), the tracking center FC advances relative to the lumbar position LP. Therefore, when lumbar movement Dx is negative (right), the subject is displayed at a position shifted to the left from the center of the screen 300 (tracking center FC) by the absolute value |Dx| of lumbar movement Dx.

[0101] As shown in FIG. 23, if the subject is displayed at a position corresponding to the difference between the waist position and the center of tracking (waist sway), it becomes easier to intuitively grasp the acceleration and deceleration of the subject's walking.

[0102] (operation) Next, an example of the operation of the gait information generator 30 will be described with reference to the drawings. Fig. 24 is a flowchart for explaining an example of the operation of the gait information generator 30. In the description following the flowchart of Fig. 24, the gait information generator 30 will be the subject of the operation.

[0103] 24, first, the gait information generator 30 acquires waist position information in a predetermined walking cycle (step S31). For example, the gait information generator 30 acquires waist position information in one step walking cycle. The gait information generator 30 may acquire waist position information in multiple walking cycles.

[0104] Next, the gait information generation device 30 derives a reference line related to the waist position in a predetermined walking cycle (step S32). For example, the gait information generation device 30 derives a reference line related to the waist position in one step walking cycle. The gait information generation device 30 may derive a reference line related to the waist position in multiple walking cycles.

[0105] Next, the gait information generating device 30 calculates the distance between the waist position and the reference line (waist sway) for each walking cycle (step S33).

[0106] Next, the gait information generation device 30 calculates the tracking center of the virtual camera 350 in accordance with the calculated waist sway (step S34). For example, the gait information generation device 30 calculates the tracking center based on the waist position and waist sway.

[0107] Next, gait information generating device 30 generates gait information including display information according to the calculated waist sway and tracking center (step S35). For example, gait information generating device 30 generates gait information including display information reflecting the difference between the waist position and the tracking center (waist sway). For example, gait information generating device 30 generates gait information including time-series data of waist sway and a graph of the time-series data of waist sway.

[0108] Next, the gait information generating device 30 outputs the generated gait information (step S36). For example, the gait information generating device 30 outputs display information reflecting the difference between the waist position and the tracking center (waist sway) and gait information including a graph of time-series data of waist sway.

[0109] (Application example) Next, application examples of the gait information generating device 30 will be described with reference to the drawings. Here, an example will be given in which gait information output from the gait information generating device 30 is displayed on the screen of a terminal device. In the following application examples, an example will be given in which gait information is displayed in a video of a walking subject. The video of the subject may be an actual video, or may be a virtual person (character). In the following, an example will be given in which a character is displayed in a video. The display information shown in the following application examples may be generated by the gait information generating device 30, or may be generated by another device or system that acquires the gait information.

[0110] [Application Example 3-1] FIG. 25 is a conceptual diagram of Application Example 3-1 related to the gait information generating device 30. In Application Example 3-1, display information in which the display position of a subject changes according to the subject's lumbar sway is displayed as gait information output from the gait information generating device 30. In the example of FIG. 25, a graph of time-series data of lumbar sway is displayed in the upper right region of the frame, as in the first embodiment. The graph of time-series data of lumbar sway may be omitted. A graph of time-series data related to the speed of lumbar sway may also be displayed in the frame.

[0111] FIG. 25 shows three frames extracted from multiple frames included in a video of a subject walking. An actual video is composed of many more frames. Time (walking cycle) progresses from the upper left to the lower right in the three frames in FIG. 25. The character in the frame is displayed at a position corresponding to the lumbar sway. The character is displayed at a position corresponding to the direction of the lumbar sway relative to the tracking center FC. The character is displayed at a position corresponding to the magnitude of the lumbar sway. The greater the lumbar sway, the greater the deviation of the character from the tracking center FC. The smaller the lumbar sway, the smaller the deviation of the character from the tracking center FC. In addition, in the graph of the time-series data of lumbar sway displayed in the frame, indicator I (vertical line) is displayed at a position corresponding to the walking phase of the subject. Indicator I moves in the direction shown by the arrow in accordance with the walking phase of the subject. The position of the character in the frame corresponds to the lumbar sway indicated by indicator I in the graph.

[0112] In the upper left frame, the waist sway is positive, so the character is displayed to the right of the tracking center FC. In the middle frame, the waist sway is zero, so the character is displayed at the tracking center FC. In the lower right frame, the waist sway is negative, so the character is displayed to the left of the tracking center FC.

[0113] In this application example, a character resembling the subject is displayed at a position corresponding to the direction and magnitude of the waist movement in association with the walking phase of the subject. According to this application example, the gait of the subject can be intuitively grasped based on the movement of the character displayed at a position corresponding to the direction and magnitude of the waist movement.

[0114] [Application Example 3-2] Fig. 26 is a conceptual diagram of Application Example 3-2 related to the gait information generating device 30. In Application Example 3-2, the display position of the character according to the waist sway is switched in response to a user operation. In Fig. 26, in order to make the character's movement clearer, the character's movement is expressed with a displacement amount greater than the actual waist sway.

[0115] In the example of Fig. 26, a button for changing the displacement amount of the subject on screen 300 is displayed at the top left of the screen. In the example of Fig. 26, a button for changing the displacement amount to five times (x5) the actual lumbar movement is displayed. The displacement amount (magnification) that can be set and the arrangement of the buttons are not limited to the example of Fig. 26.

[0116] In the state of the upper left screen 300, the button is not pressed. In the state of the upper left screen 300, the subject is displayed in a position to the right of the tracking center FC by the absolute value |Dx| of the waist fluctuation Dx. In the state of the lower right screen 300, the button is pressed and the change in the displacement amount is active. In the state of the lower right screen 300, in response to pressing of the button, the subject is displayed in a position that is 5 times the absolute value |Dx| of the waist fluctuation Dx (5 × |Dx|). In the state of the lower right screen 300, waist fluctuation is emphasized, making it easy to recognize fluctuations in waist position as the subject walks.

[0117] In this application example, in response to a user operation, the displacement amount of the subject on the screen 300 is changed. According to this application example, the waist fluctuation according to the subject's walking can be highlighted, so that the gait can be grasped more intuitively.

[0118] As described above, the gait information generating device of this embodiment includes an acquisition unit, a lumbar sway calculation unit, a following center calculation unit, a gait information generation unit, and an output unit. The acquisition unit acquires lumbar position information including time-series data of the lumbar position of the subject in a predetermined walking cycle. The lumbar sway calculation unit calculates lumbar sway for multiple walking phases included in the predetermined walking cycle. The lumbar sway corresponds to the distance between a reference line set for the time-series data of the lumbar position of the subject and the lumbar position in each of the multiple walking phases. The following center calculation unit calculates a following center that follows the subject according to the direction and magnitude of the lumbar sway. The gait information generation unit generates gait information according to the lumbar sway calculated for the predetermined walking cycle. The gait information generation unit generates, as gait information, display information in which the subject is displayed at a position according to the difference between the following center and the lumbar sway in frames constituting a video showing the walking state of the subject. The output unit outputs the gait information including the generated display information.

[0119] In this embodiment, display information is generated in which the subject is displayed at a position corresponding to the difference between the tracking center and the waist position (waist sway). Therefore, according to this embodiment, the subject's gait can be more intuitively grasped according to the fluctuation of the subject's position in the frames constituting the video. In this embodiment, an example is given in which display information is generated in which the subject is displayed at a position corresponding to waist sway. The technique of this embodiment may also be applied to generating display information in which the subject is displayed at a position corresponding to waist sway speed information.

[0120] (Fourth embodiment) Next, a gait information generating device according to a fourth embodiment will be described with reference to the drawings. The gait information generating device of this embodiment has a simplified configuration of the first to third gait information generating devices.

[0121] 27 is a block diagram showing an example of the configuration of a gait information generation device 40 according to this embodiment. The gait information generation device 40 includes an acquisition unit 41, a waist sway calculation unit 42, a gait information generation unit 45, and an output unit 47.

[0122] The acquisition unit 41 acquires lumbar position information including time-series data of the subject's lumbar position in a predetermined walking cycle. The lumbar sway calculation unit 42 calculates lumbar sway corresponding to the distance between a reference line set for the time-series data of the subject's lumbar position and the lumbar position in each of the plurality of walking phases included in the predetermined walking cycle. The gait information generation unit 45 generates gait information corresponding to the lumbar sway calculated for the predetermined walking cycle. The output unit 47 outputs the generated gait information.

[0123] According to this embodiment, gait information that allows intuitive understanding of the gait of the subject is generated by generating gait information that corresponds to the lumbar sway of the subject.

[0124] (Hardware) Here, a hardware configuration for executing the processes according to each embodiment of the present disclosure will be described using an information processing device 90 (computer) in Fig. 28 as an example. Note that the information processing device 90 in Fig. 28 is an example configuration for executing the processes according to each embodiment, and does not limit the scope of the present disclosure.

[0125] As shown in Fig. 28, an information processing device 90 includes a processor 91, a main storage device 92, an auxiliary storage device 93, an input / output interface 95, and a communication interface 96. In Fig. 28, interface is abbreviated as I / F (Interface). The processor 91, the main storage device 92, the auxiliary storage device 93, the input / output interface 95, and the communication interface 96 are connected to each other via a bus 98 so as to be able to communicate data with each other. The processor 91, the main storage device 92, the auxiliary storage device 93, and the input / output interface 95 are also connected to a network such as the Internet or an intranet via the communication interface 96.

[0126] The processor 91 loads a program (instructions) stored in the auxiliary storage device 93 or the like into the main storage device 92. For example, the program is a software program for executing the processing of each embodiment. The processor 91 executes the program loaded into the main storage device 92. The processor 91 executes the program to perform the processing of each embodiment.

[0127] The main memory device 92 has an area in which a program is loaded. The processor 91 loads a program stored in the auxiliary memory device 93 or the like into the main memory device 92. The main memory device 92 is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory). Alternatively, a non-volatile memory such as an MRAM (Magneto-resistive Random Access Memory) may be configured / added to the main memory device 92.

[0128] The auxiliary storage device 93 stores various data such as programs. The auxiliary storage device 93 is realized by a local disk such as a hard disk or flash memory. Note that it is also possible to configure the main storage device 92 to store various data, thereby omitting the auxiliary storage device 93.

[0129] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to external systems and devices via a network such as the Internet or an intranet based on standards and specifications. The input / output interface 95 and the communication interface 96 may be a common interface for connecting to external devices.

[0130] Input devices such as a keyboard, mouse, and touch panel may be connected to the information processing device 90 as needed. These input devices are used to input information and settings. When a touch panel is used as the input device, a screen having the function of the touch panel serves as the interface. The processor 91 and the input devices are connected via an input / output interface 95.

[0131] The information processing device 90 may be equipped with a display device for displaying information. When a display device is equipped, the information processing device 90 is equipped with a display control device (not shown) for controlling the display of the display device. The information processing device 90 and the display device are connected via an input / output interface 95.

[0132] The information processing device 90 may be equipped with a drive device. The drive device acts as an intermediary between the processor 91 and a recording medium (program recording medium) to read data and programs stored on the recording medium and to write processing results of the information processing device 90 to the recording medium. The information processing device 90 and the drive device are connected via an input / output interface 95.

[0133] The above is an example of a hardware configuration for enabling the processing according to each embodiment of the present invention. The hardware configuration of Fig. 28 is an example of a hardware configuration for executing the processing according to each embodiment, and does not limit the scope of the present invention. A program for causing a computer to execute the processing according to each embodiment is also included in the scope of the present invention.

[0134] The scope of the present invention also includes a program recording medium on which the program according to each embodiment is recorded. The recording medium can be realized, for example, as an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The recording medium may also be realized as a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. The recording medium may also be realized as a magnetic recording medium such as a flexible disk or other recording medium. When a program executed by a processor is recorded on a recording medium, the recording medium corresponds to a program recording medium.

[0135] The components of each embodiment may be combined in any manner, may be realized by software, or may be realized by a circuit.

[0136] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0137] 10, 20, 30, 40 Gait information generation device 11, 21, 31, 41 Acquisition section 12, 22, 32, 42 Lumbar movement calculation section 15, 25, 35, 45 Gait information generation section 17, 27, 37, 47 Output section 23 Speed ​​information calculation section 34 Tracking center calculation unit

Claims

1. an acquisition means for acquiring waist position information including time series data of the waist position of the subject in a predetermined walking cycle; a lumbar sway calculation means for calculating, as lumbar sway, a distance between a reference line set for time-series data of the lumbar position of the subject and the lumbar position in each of the plurality of walking phases included in the predetermined walking cycle; a tracking center calculation means for calculating a tracking center that follows the subject according to the direction and magnitude of the waist movement; a gait information generating means for generating gait information according to the waist sway calculated for the predetermined walking cycle; an output means for outputting the generated gait information, The gait information generating means a gait information generating device that generates, as the gait information, display information in which the subject is displayed at a position corresponding to a difference between the center of tracking and the waist sway in frames constituting a video showing a walking state of the subject.

2. The acquisition means acquiring the waist position information including time series data of the waist position of the subject in one gait cycle; The lumbar sway calculation means With respect to the plurality of walking phases included in the step cycle, calculating the lumbar sway corresponding to the distance between the reference line set for the time-series data of the lumbar position of the subject and the lumbar position in each of the plurality of walking phases; The gait information generating means The gait information generating device according to claim 1 , wherein the gait information generating device generates the gait information including information according to the fluctuation of the waist sway calculated with respect to the step cycle.

3. The acquisition means acquiring the waist position information including time series data of the waist position of the subject in a plurality of gait cycles; The lumbar sway calculation means With respect to a plurality of walking phases included in a plurality of walking cycles, a distance between the reference line set for time-series data of the waist position of the subject in each of the plurality of walking cycles and the waist position in each of the plurality of walking phases is calculated as the waist sway; The gait information generating means The gait information generating device according to claim 1 , wherein the gait information generating device generates the gait information including information according to fluctuations in the waist sway calculated for each of a plurality of the walking cycles.

4. The gait information generating means 2. The gait information generating device according to claim 1, wherein display information is generated as the gait information by superimposing time-series data of the waist sway in the predetermined walking cycle on frames constituting a video showing the walking state of the subject.

5. The gait information generating means The gait information generating device according to claim 4 , wherein the display information is generated by superimposing an indicator associated with the walking phase of the subject on the time-series data of the waist sway in the predetermined walking cycle.

6. a speed information calculation means for generating speed information according to the direction and magnitude of the speed of the lumbar movement; The gait information generating means The gait information generating device according to claim 1, wherein the gait information is generated by superimposing an arrow representing the direction and magnitude of the velocity of the waist sway on a frame constituting a video showing the walking state of the subject.

7. The output means outputting the display information regarding the subject to a terminal device; The gait information generating device according to claim 4 , wherein the display information is displayed on a screen of the terminal device.

8. The computer acquire waist position information including time series data of the waist position of the subject in a predetermined walking cycle; With respect to a plurality of walking phases included in the predetermined walking cycle, calculating a distance between a reference line set for time-series data of the waist position of the subject and the waist position in each of the plurality of walking phases as waist sway; Calculating a tracking center that follows the subject according to the direction and magnitude of the waist movement; generating gait information according to the waist sway calculated for the predetermined walking cycle; outputting the generated gait information; In the generation, a gait information generation method for generating, as the gait information, display information in which the subject is displayed at a position corresponding to a difference between the center of tracking and the waist sway in frames constituting a video showing the walking state of the subject;

9. A process of acquiring waist position information including time series data of the waist position of the subject in a predetermined walking cycle; a process of calculating, as lumbar sway, a distance between a reference line set for time-series data of the lumbar position of the subject and the lumbar position in each of the plurality of walking phases included in the predetermined walking cycle; A process of calculating a tracking center that tracks the subject according to the direction and magnitude of the waist movement; generating gait information according to the waist sway calculated for the predetermined walking cycle; a process of outputting the generated gait information; In the generating process, and generating, as the gait information, display information in which the subject is displayed at a position corresponding to the difference between the center of tracking and the waist sway in frames constituting an image showing the walking state of the subject.

Images