Posture analysis system and posture analysis method

Abstract

This invention provides a posture analysis system and method for analyzing the posture of a subject using images from a camera. [Solution] The posture analysis system comprises a camera positioned on the subject to capture images of the subject's surroundings, and an analysis unit that estimates the movement of the part of the subject on which the camera is positioned from images taken by the camera with a time delay, and estimates the change in the subject's posture from the estimated movement of the part.

Description

Technical Field

[0001] The present invention relates to a posture analysis system and a posture analysis method.

Background Art

[0002] Since walking is a daily activity, even if one's own walking state or posture changes gradually, there are times when it is difficult to become aware of such changes. Meanwhile, techniques for analyzing walking states have been reported. When using such techniques, detailed movements of the leg can be analyzed by installing sensors near the joints of the subject's leg to measure the movement of each part. Furthermore, a technique for analyzing posture during walking using a fixed camera arranged at a fixed point has been reported.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] However, the method described in Patent Document 1 requires specialized knowledge for measurement by providing the subject with sensors for detecting the state, a dedicated measurement environment, and cooperation from related parties. Therefore, if there is no dedicated measurement environment, or if there is no specialized knowledge even if such an environment exists, it may be difficult to apply the method to the purpose of identifying one's own walking state. Furthermore, the movements of each part of the subject that can be identified from images captured by the fixed camera are limited to the range that can be captured by the fixed camera. In addition, even within the range captured by the fixed camera, it has been difficult to identify the position of the subject in the optical axis direction of the fixed camera. For this reason, there have been cases where it is difficult to identify the posture state of a subject from an image of the subject captured using a fixed camera.

[0005] The problem that this invention aims to solve is to provide a posture analysis system and a posture analysis method that analyze the posture of a subject using images from a camera. [Means for solving the problem]

[0006] (1) The present invention has been made to solve the above problems, and a posture analysis system in a first aspect of the present invention is a posture analysis system comprising a camera placed on a subject to photograph the surroundings of the subject, and an analysis unit that estimates the movement of the part of the subject on which the camera is placed from images of the camera taken with a time difference, and estimates the change in the subject's posture from the estimation result of the movement of the part. (2) The posture analysis system described above is configured such that the subject's body part is at least one of the subject's upper arm, shoulder, chest, or head, and the analysis unit obtains images taken from the subject's body part with the camera at intervals of time, and estimates the movement of the body part where the camera is located from the position of the image common to the images taken at intervals of time. (3) When the subject's body part is the upper arm of the subject, the posture analysis system described above estimates the movement of the upper arm where the camera is located from the position of the image common to the images taken with the time difference. (4) The posture analysis system described above includes an analysis unit which identifies common images in the images taken with a time difference while the subject is walking, and estimates the movement of the upper arm where the camera is located from the position of the common image within the image. (5) The posture analysis system described above includes an image extraction unit which, in the initial stage before the subject starts walking, has the optical axis of the camera pointed approximately to the side of the subject and is positioned on the subject's upper arm, and acquires a reference image from the camera in the initial stage and acquires a first image while the subject is walking, and the analysis unit estimates the movement of the subject's upper arm where the camera is positioned while the subject is walking from the position of images common to the reference image and the first image. (6) The posture analysis system described above includes an image extraction unit which acquires a first image from the camera while the subject is walking, and then acquires a second image while the subject is walking. The analysis unit which detects changes in the movement of the upper arm where the camera is located from the position of an image common to the reference image, the first image, and the second image, and estimates the posture of the subject while walking using the detection result of the change in the movement of the upper arm. (7) One aspect of the present invention is a computer posture analysis method for analyzing images from a camera placed on a subject to capture the surrounding environment of the subject, the method comprising estimating the movement of the part of the subject on which the camera is placed from the images of the camera taken with a time difference, and estimating the change in the subject's posture from the estimation result of the movement of the part. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a posture analysis system and a posture analysis method that analyze the posture of a subject using images from a camera. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram of the posture analysis system according to the embodiment. [Figure 2] This is a diagram illustrating the configuration of the posture analysis system according to the embodiment. [Figure 3] This is a diagram illustrating the orientation of subject U in the embodiment. [Figure 4] This is a diagram illustrating the walking posture of subject U in the embodiment. [Figure 5] This is a flowchart of the analysis process for the embodiment. [Figure 6] This diagram illustrates the relationship between the movement of the left arm of subject U in the embodiment and the range that can be captured by the camera. [Modes for carrying out the invention]

[0009] A posture analysis system and a posture analysis method according to an embodiment will be described with reference to the drawings.

[0010] First, an overview of the posture analysis system of the embodiment will be described. Figure 1 is a schematic configuration diagram of a posture analysis system 1 according to the embodiment.

[0011] The posture analysis system 1 analyzes the posture of a subject U during walking.

[0012] This posture analysis system 1 includes, for example, a movable floor device 2, an analysis device 3, a camera 5, and a terminal device 6.

[0013] The camera 5 is arranged on the subject U, for example, so as to capture an image of the situation around the subject U. The camera 5 is preferably arranged on the upper body, arm, or the like of the subject U. For example, the camera 5 may be capable of switching between and using a first mode for generating a so-called panoramic image and a second mode for capturing images at a standard or wide angle of view. The consistency between an image of a specific range in the panoramic image obtained in the first mode and an image obtained in the second mode is used to estimate the posture of the subject U.

[0014] The movable floor device 2 is an example of an exercise apparatus used when a user walks indoors or the like without moving around the surroundings. The movable floor device 2 may be referred to as a treadmill, a walking machine, or the like. The movable floor device 2 is configured to be capable of adjusting exercise load with respect to walking or the like of the operator.

[0015] For example, the movable floor device 2 includes a frame 21 arranged substantially along the floor surface, a movable belt 22 (movable floor), a pair of rollers (not shown), an electric motor (not shown), and a controller unit 24.

[0016] The frame 21 of the movable floor device 2 also serves as the housing of the movable floor device 2, and supports each part of the movable floor device 2. For example, the frame 21 (movable floor device 2) is arranged on the floor surface.

[0017] The movable belt 22 of the movable floor device 2 is supported by a pair of rollers and the like provided at the front and rear of the frame 21 of the movable floor device 2, and rolls as the pair of rollers rotate. For example, the shaft of a motor (not shown) is connected to one shaft of the pair of rollers, and by driving the motor, the movable belt 22 of the movable floor device 2 can be caused to roll at a desired speed. By adjusting the speed of this motor, the rolling speed of the movable belt 22 of the movable floor device 2 can be adjusted. The configuration for rolling the movable belt 22 of the movable floor device 2 is not limited to the above, and other known configurations may be adopted.

[0018] The controller unit 24 performs processing for speed control of the motor and collection of various data. The controller unit 24 executes the above control in accordance with instructions from the analysis device 3. This controller unit 24 is configured as a so-called embedded computer.

[0019] The analysis device 3 of the embodiment may be incorporated in the movable floor device 2 described above, or may be configured as a separate body from the movable floor device 2.

[0020] The analysis device 3 includes, for example, a storage unit 31, an image extraction unit 32, an analysis unit 33, and a communication processing unit 34. The storage unit 31 is configured including a semiconductor memory and the like, and stores data and programs used for various processes in the analysis device 3. The communication processing unit 34 communicates with, for example, a camera 5 provided outside the analysis device 3 and a terminal device 6 (not shown). The communication processing unit 34 receives, from the terminal device 6 and the like, an instruction related to control of the movable floor device 2, an instruction to execute analysis processing by the analysis device 3, and the like. The communication processing unit 34 outputs the display of the control state of the movable floor device 2 and the result of the analysis processing. The terminal device 6 is, for example, a portable computer such as a smartphone.

[0021] The image extraction unit 32 acquires image data from the camera 5, for example, via the communication processing unit 34. The image extraction unit 32 detects images of feature points around the subject U from the image based on the image data from the camera 5. The image based on the image data from the camera 5 is an example of an image obtained by the camera 5.

[0022] The analysis unit 33 estimates the movement of the part of subject U where camera 5 is placed from images taken by camera 5 with a time delay, and estimates the change in subject U's posture from the estimated movement of that part. The part of subject U where camera 5 is placed is at least one of the upper arm, shoulder, chest, or head of subject U. The same analysis method can be applied even if the part is changed.

[0023] The analysis unit 33 acquires images taken by the camera 5 from parts of the subject U at different time intervals via the image extraction unit 32. The analysis unit 33 estimates the movement of the area where the camera 5 is located from the position of images common to the images taken at different time intervals.

[0024] The following embodiment will be described using the case where the part of subject U is the upper arm of subject U as an example. Camera 5 is positioned on the upper arm of subject U, with its optical axis pointed approximately to the side of subject U, during the initial stage before subject U begins walking.

[0025] The analysis unit 33 estimates the movement of the upper arm where the camera 5 is located from the position of images common to the images taken with a time difference.

[0026] For example, the analysis unit 33 identifies common images in images taken at different time intervals while subject U is walking. The analysis unit 33 estimates the movement of the upper arm where camera 5 is located from the position of the common images within the images.

[0027] Let me give a more specific example. The image extraction unit 32 acquires an initial reference image from the camera 5 and then acquires the first image while subject U is walking. First, the analysis unit 33, Based on the position of images common to both the reference image and the first image, the movement of subject U during walking on the upper arm where camera 5 is located is estimated.

[0028] Furthermore, the image extraction unit 32 acquires a first image from the camera 5 while subject U is walking, and then acquires a second image while subject U is walking. In response, the analysis unit 33 detects changes in the movement of the upper arm where the camera 5 is located from the position of images common to the above reference image, the first image, and the second image, and estimates the posture of subject U while walking using the detection results of changes in the movement of the upper arm.

[0029] Figure 2 is a diagram showing the configuration of the posture analysis system 1 according to the embodiment. As shown in Figure 2, the movable floor device 2 is positioned on the floor (floor panel).

[0030] The frame 21 of the movable floor device 2 is formed at a predetermined height. The movable belt 22, a pair of rollers, and an electric motor (not shown) are arranged within the cross-section of the frame 21.

[0031] Referring to Figures 3 to 5, the walking posture of subject U in the embodiment will be described. Figure 3 is a diagram illustrating the orientation of subject U in the embodiment. As shown in Figure 3, subject U walks on the movable floor device 2. The direction of movement of the movable belt 22 of the movable floor device 2 is set to the +Y axis direction, and this direction is called 0 degrees. Subject U performs a walking motion in the direction of 0 degrees. In this case, the direction behind subject U (-Y axis) becomes 180 degrees, and the direction to subject U's left hand (-X axis) becomes 270 degrees. Here, we assume that in a space where the position of subject U's head is the origin O, there is a side surface of a cylinder whose axis passes through the origin O and extends in the Z-axis direction. We also assume that the surrounding scenery, as seen from the origin O, is projected onto this side surface of the cylinder. Let's organize the above from a different perspective. A panoramic image is obtained with the origin O as the reference point, and this panoramic image is projected onto the side surface of the cylinder. Using the image projected onto the side surface of the cylinder in this way, the direction relative to the origin O can be detected.

[0032] Figure 4 is a diagram illustrating the walking posture of subject U in the embodiment. The model shown in Figure 4(a) is an example of a view of subject U, who has relatively good posture, from above. The model shown in Figure 4(b) is a model of subject U, who has a relatively poor posture, viewed from above. The subject shown in Figure 4(b) exhibits a posture commonly referred to as rounded shoulders, straight neck, or hunchback. As a result, the back is rounded, and both shoulders and the head are jutting forward.

[0033] These differences in posture can lead to differences in arm movement. In the model shown in Figure 4(a), the line segment connecting the base of the neck and the shoulder joint is a straight line passing through the base of the neck along the X-axis. For example, a person with good posture can generally maintain this posture. This is a posture that a person with poor posture can reproduce if they consciously correct their posture. When walking in this posture, the movement of the arms follows a plane perpendicular to the X-axis (the YZ plane).

[0034] In contrast, in the model shown in Figure 4(b), the line segment connecting the base of the neck and the shoulder joint is not a straight line passing through the base of the neck along the X-axis, but is bent at the base of the neck. The line segment passing through the left shoulder in this plan view is tilted from the -X-axis direction to the +Y-axis direction. This posture is one that may occur, for example, when a person with generally poor posture does not consciously try to correct it. When walking in this posture, the movement of the arms follows a plane perpendicular to this line segment.

[0035] For example, camera 5 is fixed to the outside of subject U's left arm (opposite side of the upper body). The optical axis of camera 5 is fixed so that it points approximately to the left (-X direction) of subject U. As described above, in the case of a model with a relatively good posture, the direction of the optical axis will be along the X-axis. In contrast, for models with relatively poor posture, the direction of the optical axis will be tilted more towards the +Y axis than towards the X axis.

[0036] Thus, it is clear that the shooting range of camera 5 differs depending on the orientation of the optical axis of camera 5. Therefore, by identifying the image within the range captured by camera 5, the movement of subject U's left arm can be indirectly identified.

[0037] Referring to Figure 5, the processing of the embodiment will be described in order. (Analysis process) First, please refer to Figure 5 to explain the overall flow of the analysis process. Figure 5 is a flowchart of the analysis process in the embodiment.

[0038] The walking speed during the gait evaluation is pre-set using information from the terminal device 6 (S11). For example, the analysis unit 33 sets the speed at which the movable belt 22 rotates (command speed) based on the walking speed during the gait evaluation identified by the subject U's actions. Subject U stands on the movable belt 22 and takes a panoramic image using the camera 5. The analysis device 3 acquires the panoramic image data from the camera 5 via the communication processing unit 34 and adds it to the storage unit 31. Subject U stands on the movable belt 22 facing the direction of movement, adjusts their posture, and relaxes their left arm while the first image is taken using camera 5. This first image is a view of the left side of subject U before the start of the exercise. Subject U operates the controller of the movable floor device 2 to start its operation when ready to walk. Subject U starts walking on the movable belt 22 at an appropriate speed corresponding to the movement speed of the movable belt 22, and begins taking images to evaluate their walking state. Accordingly, the analysis unit 33 starts capturing a second image with the camera 5 (S12) and continues recording for a predetermined period of time. The analysis unit 33 acquires image data of the second image for a predetermined period of time that has been recorded (S13).

[0039] The analysis unit 33 extracts the movement of the camera 5 attached to subject U in the optical axis direction from the relationship between the image data of the first and second images and the aforementioned panoramic image (S14). This movement of the camera 5 in the optical axis direction is related to the movement of the arm (left arm) during walking. The analysis unit 33 identifies the movement of the subject U's arm (left arm) during walking based on the movement of the camera 5 in the optical axis direction, and uses the identification result to extract characteristic features of the subject U's posture during walking and estimate the walking situation (S15). The analysis unit 33 identifies whether or not there is an abnormality in the posture during walking using predetermined judgment conditions (S16). For example, the difference in the reference direction between the first image and the second image obtained from the camera 5 can be used for the above identification. The analysis unit 33 displays the identification result on the terminal device 6 (S17).

[0040] Through this process, an analysis of the walking state of subject U is performed using the image information from camera 5 installed on subject U. Through the above process, the analysis unit 33 becomes able to easily identify the walking state of subject U.

[0041] Incidentally, assuming that subject U is in a healthy state with no major impairment to walking, it is estimated that their posture while walking does not change significantly from their posture at rest. However, when fatigue from exercise occurs, it becomes difficult to maintain the initial posture before the start of exercise, and the posture changes. Therefore, in this embodiment, the walking state of subject U is analyzed using image information from camera 5 before walking begins and during walking.

[0042] Referring to Figure 6, the relationship between the movement of the left arm of subject U in the embodiment and the range that can be captured by camera 5 will be explained. Figure 6 is a diagram illustrating the relationship between the movement of the left arm of subject U in the embodiment and the range that can be captured by camera 5.

[0043] Figure 6(a) shows the first image taken by camera 5 before the start of exercise and the second image taken by camera 5 after the start of exercise but before fatigue sets in. By comparing it with the panoramic image, the first image taken before the start of exercise is identified, for example, as an image in the 270-degree direction. The second image during movement will be scattered, for example, between directions slightly closer to 0 degrees and directions slightly closer to 180 degrees, with the 270-degree direction in between. Due to the swing of the arm, the vertical orientation of the second image will be uneven.

[0044] Figure 6(b) shows the first image taken by camera 5 before the start of exercise and the second image taken by camera 5 when fatigue from exercise had accumulated. By comparing it with the panoramic image, the first image taken before the start of exercise can be identified, for example, as an image in the 270-degree direction. The second image taken during exercise when fatigue has accumulated will be a scattered image, for example, closer to 0 degrees than 270 degrees. Similarly, due to arm swing, the vertical orientation of the second image will be uneven. As described above, changes in the posture of subject U cause a change in the direction of the optical axis of camera 5 attached to subject U.

[0045] The analysis unit 33 scans within the panoramic image to detect images that match the image from camera 5 or images that match within a predetermined range. The image from camera 5 may rotate around the optical axis or become distorted due to the movement of the subject U. The analysis unit 33 performs the scanning while allowing for the effects of the above-mentioned image rotation and direction changes.

[0046] Using the scanning described above, the analysis unit 33 detects the direction of the optical axis of camera 5 by identifying changes in the image from camera 5. The direction of the optical axis of camera 5 is associated with the movement of subject U's arm. By utilizing the fact that the movement of subject U's arm is affected by subject U's posture, the analysis unit 33 can identify changes in subject U's posture from the image from camera 5.

[0047] The above determination method allows for real-time processing.

[0048] The posture analysis system 1 of this embodiment includes a camera 5 positioned over the subject U to capture images of the subject U's surroundings, and an analysis unit 33 that estimates the movement of the part of the subject U where the camera 5 is positioned from images taken by the camera 5 with a time delay, and estimates the change in the subject U's posture from the estimated movement of the part. Thus, the posture of the subject U can be analyzed using images from the camera 5.

[0049] The embodiments of the present invention have been described above, but the posture analysis system 1 has a computer system inside. The series of processes related to the above-described process are stored in the form of a program on a computer-readable storage medium, and the above process is performed when the computer (CPU) reads and executes this program. Here, the computer-readable storage medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, this computer program may be distributed to a computer via a communication line through a communication interface, and the computer that receives this distribution executes the program. Furthermore, the term "computer system" here includes an OS, etc. Image information from the camera 5 may also be collected via the communication interface.

[0050] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0051] For example, some of the functions of the analysis device 3 may be implemented by the camera 5 or the terminal device 6. Furthermore, while one example of placing camera 5 on the upper arm has been shown, similar processing may be performed using a camera placed on the wrist.

[0052] Furthermore, when scanning within the panoramic image, instead of detecting a range that faithfully matches the image from camera 5, the system may be modified to detect similarities between the features of each part of the panoramic image and the features of the image from camera 5. In this case, the features of each part of the panoramic image may be coded, and the coding of angle information may be obtained by detecting the match or similarity with the features of the image from camera 5. [Explanation of Symbols]

[0053] 1. Posture Analysis System 2 Movable floor device 3 Analysis device 5 Cameras 6 Terminal devices 33 Analysis Department

Claims

1. A camera positioned on the subject to record the surrounding environment of the subject, An analysis unit estimates the movement of the part of the subject where the camera is positioned from images taken by the camera with a time delay, and estimates the change in the subject's posture from the estimated movement of the part. A posture analysis system equipped with the following features.

2. The subject's body part is at least one of the subject's upper arm, shoulder, chest, or head. The aforementioned analysis unit, Images are obtained from the subject's body part using the camera, with a time difference between each image. The movement of the body part where the camera is located is estimated from the position of the image common to the images taken with the time difference. The posture analysis system according to claim 1.

3. When the subject's body part is the subject's upper arm, The aforementioned analysis unit, The movement of the upper arm where the camera is positioned is estimated from the position of the image common to the images taken with the aforementioned time difference. The posture analysis system according to claim 2.

4. The aforementioned analysis unit, Identify common images in images taken with a time difference while the subject is walking. From the position of the common image within the image, the movement of the upper arm where the camera is positioned is estimated. The posture analysis system according to claim 3.

5. The camera is positioned on the upper arm of the subject with its optical axis directed approximately to the side of the subject during the initial stage before the subject begins walking. Image extraction unit acquires the initial reference image from the camera and acquires the first image while the subject is walking. Equipped with, The aforementioned analysis unit, Based on the position of the image common to the reference image and the first image, the movement of the upper arm where the camera is positioned during the subject's walking is estimated. The posture analysis system according to claim 2.

6. The image extraction unit acquires a first image from the camera while the subject is walking, and then acquires a second image while the subject is walking. The aforementioned analysis unit, Based on the position of an image common to the reference image, the first image, and the second image, changes in the movement of the upper arm where the camera is positioned are detected, and the subject's posture while walking is estimated using the detection result of the change in the movement of the upper arm. The posture analysis system according to claim 5.

7. A computer posture analysis method that analyzes images from a camera positioned on a subject to capture the surrounding environment of the subject, From images taken by the camera with a time delay, the movement of the part of the subject where the camera is positioned is estimated, and the change in the subject's posture is estimated from the estimated movement of the part. A posture analysis method that includes this.

Images