Posture evaluation device, posture evaluation system, posture evaluation method, and program

Abstract

Provided are: a posture evaluation device; a posture evaluation system; and a posture evaluation method that are capable of evaluating a posture with high accuracy at a relatively low cost; and a non-transitory computer-readable medium. A posture evaluation device (100) comprises: a spine extraction means (103) for extracting, on the basis of an image obtained by imaging a side of a subject's body and position information on at least a cervical spine, a hip joint, and a knee joint of the body on the image, a spine edge point group consisting of a predetermined number of points indicating a spine shape on the image; a feature amount calculation means (105) for calculating a feature amount related to at least a spine on the basis of the position information and the spine edge point group; and a state estimation means (106) for estimating a state of at least the spine on the basis of the feature amount.

Description

[Technical Field]

[0001] The present disclosure relates to a posture evaluation device, a posture evaluation system, a posture evaluation method, and program Regarding. [Background technology]

[0002] In recent years, with the spread of online training and self-training, there is a growing need for ordinary people without specialized knowledge to evaluate their own posture.

[0003] Patent Document 1 describes an exercise evaluation system that extracts features from video data acquired by capturing an image of the body using a device carried by the user, identifies the position of each part of the body, and displays the movement of the bones connecting those parts superimposed on the video. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-141806 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the exercise evaluation system described in Patent Document 1 represents the trunk as a straight line or a rectangle, making it impossible to evaluate posture based on the spinal column shape itself. Therefore, the accuracy of the posture evaluation in Patent Document 1 is insufficient compared to the accuracy of posture evaluations performed by experts such as therapists and trainers. Methods for measuring spinal column shape include those using depth cameras, acceleration sensors, and scanning probes along the spine, but all of these require expensive specialized equipment. Therefore, these methods are unsuitable for ordinary people to evaluate their own posture.

[0006] The object of the present disclosure is to provide a posture evaluation device, a posture evaluation system, a posture evaluation method, and program The purpose is to provide [Means for solving the problem]

[0007] The posture assessment device according to the present disclosure includes a spine extraction means for extracting a spine edge point cloud consisting of a predetermined number of points representing the shape of the spine on the image, based on an image obtained by capturing an image of the side of the subject's body and positional information of at least the cervical vertebrae, hip joint, and knee joint of the body on the image; a feature calculation means for calculating feature amounts related to at least the spine, based on the positional information and the spine edge point cloud; and a state estimation means for estimating at least the state of the spine, based on the feature amounts.

[0008] The posture evaluation system according to the present disclosure comprises a posture evaluation device and a subject terminal capable of communicating with the posture evaluation device, wherein the posture evaluation device comprises: a spine extraction means for extracting a spine edge point cloud consisting of a predetermined number of points representing the shape of the spine on the image based on an image of the side of the subject's body acquired by the subject terminal and position information of at least the cervical vertebrae, hip joint, and knee joint of the body on the image; a feature calculation means for calculating feature amounts related to at least the spine based on the position information and the spine edge point cloud; and a state estimation means for estimating at least the state of the spine based on the feature amounts.

[0009] The posture assessment method according to the present disclosure is a method in which a posture assessment device extracts a spinal column edge point cloud consisting of a predetermined number of points representing the shape of the spinal column on the image based on an image obtained by capturing an image of the side of the subject's body and positional information of at least the cervical vertebrae, hip joint, and knee joint of the body on the image, calculates feature amounts related to at least the spine based on the positional information and the spinal column edge point cloud, and estimates the state of at least the spine based on the feature amounts.

[0010] In accordance with the present disclosure programThe program causes a posture evaluation device to execute the following processes: extracting a spinal column edge point cloud consisting of a predetermined number of points representing the spinal column shape on an image obtained by capturing an image of the side of the body of a subject and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; calculating feature amounts related to at least the spinal column on the basis of the position information and the spinal column edge point cloud; and estimating at least the state of the spinal column on the basis of the feature amounts. In be. [Effects of the Invention]

[0011] A posture evaluation device, a posture evaluation system, and a posture evaluation method that can evaluate posture with high accuracy at a relatively low cost. program can be provided. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of a posture evaluation device according to a first embodiment. [Figure 2] FIG. 10 is a block diagram showing the configuration of a posture evaluation device according to a second embodiment. [Figure 3] FIG. 10 is a diagram showing an example of an image captured by an imaging section according to the second embodiment. [Figure 4] FIG. 10 is a diagram showing an example of key points according to the second embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a spine edge point cloud according to the second embodiment. [Figure 6] FIG. 10 is a diagram illustrating the processing of the spinal column extraction unit according to the second embodiment. [Figure 7] FIG. 10 is a diagram illustrating processing by a feature amount calculation unit according to the second embodiment. [Figure 8] FIG. 10 is a diagram illustrating processing by a feature amount calculation unit according to the second embodiment. [Figure 9] FIG. 10 is a diagram illustrating the processing of a state estimation unit according to the second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of an image displayed on a display unit according to the second embodiment. [Figure 11]FIG. 10 is a diagram showing another example of an image displayed on the display unit according to the second embodiment. [Figure 12] FIG. 10 is a diagram showing the data structure of a reference value list according to the second embodiment. [Figure 13] 10 is a flowchart showing a posture evaluation method according to the second embodiment. [Figure 14] FIG. 11 is a block diagram showing the configuration of a posture evaluation device according to a third embodiment. [Figure 15] FIG. 10 is a diagram showing another example of an image displayed on the display unit in the posture evaluation device according to the third embodiment. [Figure 16] FIG. 10 is a diagram illustrating a posture evaluation system according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Embodiment 1 The first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of a posture evaluation device 100 according to the first embodiment. The posture evaluation device 100 in the first embodiment is a device that evaluates posture based on an image obtained by capturing a side view of the body using a camera such as a smartphone. Specifically, the posture evaluation device 100 estimates the shape of the spine in the image and evaluates posture based on the spine shape. This allows posture to be evaluated relatively inexpensively and with high accuracy in situations such as online training and self-training.

[0014] As shown in FIG. 1, the posture evaluation device 100 includes a spinal column extraction unit 103, a feature amount calculation unit 105, and a state estimation unit .

[0015] The spine extraction unit 103 extracts a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image, based on an image obtained by capturing a side view of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image. Note that the subject refers to a person whose posture is to be evaluated by the posture evaluation device 100. Here, the image obtained by capturing an image of the side of the subject's body is a two-dimensional image, and may be a two-dimensional RGB image. The spine edge point cloud is a point cloud consisting of multiple points that represent the spine on an image. Each point that makes up the spine edge point cloud may be a single pixel, or an image region consisting of N pixels vertically and M pixels horizontally. N and M are positive integers, and may be equal or different. Furthermore, the position information of the cervical vertebrae on the image is, for example, the position information of any one of the seven vertebrae that make up the cervical vertebrae, for example, the position information of the vertebral column (C7). Furthermore, the position information of the knee joint on the image is the position information of any one of the lower end of the femur, the patella, the upper end of the tibia, the upper end of the fibula, and the knee joint space, for example, the position information of the lower end of the femur. Specifically, the position information of the vertebral column is, for example, the position information of a pixel located at the center of an image region on the image that corresponds to the vertebral column. Similarly, the position information of the lower end of the femur is, for example, the position information of a pixel located at the center of an image region on the image that corresponds to the lower end of the femur. Furthermore, the position information on the image is, for example, image coordinates. Here, image coordinates are coordinates for indicating the position of a pixel on a two-dimensional image, and are defined as coordinates in which, for example, the center of the pixel located at the leftmost and uppermost side of the two-dimensional image is the origin, the left-right or horizontal direction is the x-direction, and the up-down or vertical direction is the y-direction.

[0016] The feature amount calculation unit 105 calculates feature amounts relating to at least the spine based on the position information of at least the cervical vertebrae, hip joints, and knee joints on the image and the spine edge point cloud.

[0017] The state estimation unit 106 estimates at least the state of the spine based on the feature amounts calculated by the feature amount calculation unit 105.

[0018] According to the first embodiment, it is possible to provide a posture evaluation device 100 capable of evaluating posture with high accuracy at a relatively low cost. Specifically, the spine extraction unit 103 extracts a spine edge point cloud representing the spine shape on the image, and the feature calculation unit 105 calculates feature amounts related to the spine based on the position information of the cervical vertebrae, hip joints, and knee joints on the image and the spine edge point cloud. The state estimation unit 106 then estimates the state of the spine based on the feature amounts. In other words, since posture can be evaluated based on the spine shape on the image, it is possible to evaluate posture with high accuracy. Furthermore, since posture evaluation can be performed based on the spine shape without using expensive specialized equipment, it is possible to evaluate posture at a relatively low cost. Therefore, it is possible to provide a posture evaluation device 100 capable of evaluating posture with high accuracy at a relatively low cost.

[0019] Methods for measuring spinal column shape include those using a depth camera, those using an acceleration sensor, and those scanning a probe along the spine. These methods aim to accurately estimate the position and inclination of each of the multiple vertebrae that make up the spine. However, experts such as therapists and trainers evaluate the balance of the curvature of the entire spine, not the position and inclination of each individual vertebra. Therefore, ordinary people do not need to aim for the same level of accuracy as these methods when evaluating their own posture at an expert level. On the other hand, the posture evaluation device 100 according to the first embodiment can evaluate posture with the same level of accuracy as experts such as therapists and trainers, based on a spinal column edge point cloud representing the spinal column shape on an image.

[0020] Furthermore, when the trunk is represented by a straight line or a rectangle and the hip joint and spine are evaluated as a unit, as in the exercise evaluation system described in Patent Document 1, there is a problem in that it is not possible to evaluate the condition of the hip joint alone. On the other hand, the posture evaluation device 100 according to the first embodiment represents the shape of the spine on the image as a cloud of spine edge points, making it possible to evaluate the condition of the hip joint alone.

[0021] Embodiment 2 The second embodiment will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the configuration of a posture evaluation device 100A according to the second embodiment. The posture evaluation device 100A is, for example, a user terminal such as a smartphone, a tablet terminal, or a personal computer owned by a user. The term "user" includes both a subject whose posture is evaluated by the posture evaluation device 100A and an evaluator who evaluates the posture of others using the posture evaluation device 100A. When a subject evaluates his or her own posture using the posture evaluation device 100A in self-training or the like, the subject is also the evaluator. When an evaluator evaluates the posture of others using the posture evaluation device 100A, the evaluator is, for example, a therapist or a trainer.

[0022] 2, posture evaluation device 100A of the second embodiment includes an imaging unit 101, a skeleton extraction unit 102, a spine extraction unit 103, a posture determination unit 104, a feature calculation unit 105, a state estimation unit 106, an image generation unit 107, a display unit 108, an input unit 109, a storage unit 110, and a communication unit 111. The input unit 109 and the display unit 108 may be configured as a single touch panel display, or may be provided separately. The storage unit 110 stores a reference value list 112, a skeleton database (referred to as "skeleton DB" in FIG. 2) 113, a skeleton extraction model 114, etc.

[0023] The imaging unit 101 captures an image of the side of the subject's body. FIG. 3 shows an example of an image of the side of the subject's body captured by the imaging unit 101. The image shown in FIG. 3 shows the side of a person O, who corresponds to the subject bending forward. Here, the image captured by the imaging unit 101 is a two-dimensional image, and may be a two-dimensional RGB image. The imaging unit 101 inputs the captured image to the skeleton extraction unit 102 and the spine extraction unit 103. The imaging unit 101 may also acquire images by capturing a moving image of the side of the subject's body. In this case, the user may specify a time point for posture evaluation by operating the input unit 109. Then, the image at the time point specified by the user may be input to the skeleton extraction unit 102 and the spine extraction unit 103.

[0024] The skeleton extraction unit 102 extracts position information of at least the cervical vertebrae, hip joints, and knee joints (hereinafter also referred to as "position information of key points") from the image captured by the imaging unit 101. FIG. 4 shows an example of key points P1, P2, and P3 extracted by the skeleton extraction unit 102 from the image shown in FIG. 3. In FIG. 4, P1 is a key point of the vertebral column (hereinafter referred to as "vertebral column key point"), P2 is a key point of the hip joint (hereinafter referred to as "hip joint key point"), and P3 is a key point of the lower end of the femur (hereinafter referred to as "knee joint key point"). Details of the position information are as described in the first embodiment, so a description thereof will be omitted. Specifically, the skeleton extraction unit 102 extracts position information of key points from the image captured by the imaging unit 101, using a trained skeleton extraction model 114. Note that the posture evaluation device 100A performs machine learning in advance using the skeleton extraction model 114, which is a machine learning model, and the skeleton database 113, which is training data, to generate the trained skeleton extraction model 114. The skeleton extraction unit 102 inputs the position information of the extracted key points to the spine extraction unit 103 . The position information of the key points may be information expressed in three-dimensional coordinates defined by the z direction, which is the depth direction, in addition to the x direction, which is the left-right or horizontal direction, and the y direction, which is the up-down or vertical direction, of the two-dimensional image captured by the imaging unit 101. This is possible by using a skeleton extraction model 114 that extracts the position information of the key points expressed in three-dimensional coordinates from the two-dimensional image. Furthermore, the body parts from which the skeleton extraction unit 102 extracts key points are not limited to the cervical vertebrae, hip joints, and knee joints described above, but may also extract key points from, for example, ankle joints, shoulder joints, elbow joints, and wrist joints. Furthermore, the skeleton extraction unit 102 may extract eyes, ears, the center of the head, and the like as key points in addition to joints. Alternatively, the key points may be specified by displaying the image captured by the imaging unit 101 on the display unit 108 and the user operating the input unit 109. Then, the position information of the key points specified by the user may be input to the spine extraction unit 103.

[0025] The spine extraction unit 103 extracts the spine from the image as a spine edge point cloud based on the image input from the imaging unit 101 and the position information of the key points input from the skeleton extraction unit 102. An example of the spine edge point cloud P extracted by the spine extraction unit 103 from the image shown in Fig. 3 is shown in Fig. 5. Details of the spine edge point cloud are as described in the first embodiment, and therefore will not be described here.

[0026] The process of the spine extraction unit 103 will be described in detail below with reference to Fig. 6. First, the spine extraction unit 103 extracts a line segment l connecting the vertebral key point P1 and the hip joint key point P2. trunk and the line segment l connecting the hip joint key point P2 and the knee joint key point P3. thigh The length of the vertebral key point P1, the hip joint key point P2, the knee joint key point P3, the line segment l are calculated, and the size of the image input from the image capturing unit 101 is normalized. trunk and the length of the line segment l thigh The length of is also converted to a normalized value.

[0027] Next, the spine extraction unit 103 extracts the line segment l trunk and line segment l thigh Based on this, the spine extraction unit 103 identifies the line of the back of the person O in the image and extracts candidate edge points that are candidates for the spine edge point group P. Specifically, the spine extraction unit 103 extracts the line of the line segment l trunk and line segment l thigh Based on this, a bounding box that includes at least the back of person O appearing in the image is specified. Next, the spine extraction unit 103 performs edge extraction processing on the image data within the bounding box, and extracts a group of candidate edge points.

[0028] Next, the spine extraction unit 103 extracts the line segment l trunk and line segment lthigh Next, the spine extraction unit 103 calculates the exterior angle θ0 between the line segments l1 and l2 that define the head side of the spine based on the exterior angle θ0. trunk The angle θ1 between the line l2 and the line l that defines the caudal side of the spine trunk Specifically, an angle table (not shown) associating exterior angle θ0 with angle θ1 and angle θ2 is stored in advance in the storage unit 110, and the spine extraction unit 103 determines angles θ1 and θ2 based on the exterior angle θ0 by referring to the angle table. Also, an angle determination model (not shown) that has been machine-trained using the angle table as training data may be stored in advance in the storage unit 110, and the spine extraction unit 103 may use the angle determination model to determine angles θ1 and θ2 based on the exterior angle θ0.

[0029] Next, the spine extraction unit 103 determines the intersection point between the line segment l1 and the candidate edge point group as the head-side endpoint of the spine edge point group P, and determines the intersection point between the line segment l2 and the candidate edge point group as the tail-side endpoint of the spine edge point group P. This determines the range of the candidate edge point group that will become the spine edge point group P. In other words, the spine edge point group P is extracted.

[0030] Next, the spine extraction unit 103 inputs the calculated exterior angle θ0 to the posture determination unit 104. The spine extraction unit 103 also inputs the normalized vertebral prominence keypoint P1, hip joint keypoint P2, and knee joint keypoint P3, as well as the extracted spine edge point group P, to the feature calculation unit 105. The spine extraction unit 103 also inputs the normalized image, the normalized keypoints P1, P2, and P3, and the extracted spine edge point group P to the image generation unit 107.

[0031] The posture determination unit 104 determines the type of posture of the person O appearing in the image captured by the imaging unit 101, based on the exterior angle θ0 input from the spine extraction unit 103. Specifically, a posture table (not shown) that associates the exterior angle θ0 with the type of posture is stored in advance in the storage unit 110, and the posture determination unit 104 determines the type of posture based on the exterior angle θ0 by referring to the posture table. Here, examples of the types of posture include bending forward, standing, bending backward, etc. The posture determination unit 104 inputs the determined type of posture to the state estimation unit 106. The type of posture may be designated by the user operating the input unit 109. The type of posture designated by the user may then be input to the state estimation unit .

[0032] The feature amount calculation unit 105 calculates at least feature amounts related to the spine based on the key points P1, P2, and P3 and the spine edge point group P input from the spine extraction unit 103. Specifically, the feature amount calculation unit 105 calculates the spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, lumbar curvature angle, etc. as feature amounts based on the vertebral prominence key point P1, hip joint key point P2, knee joint key point P3, and the spine edge point group P.

[0033] Specifically, the feature amount calculation unit 105 calculates the spinal curvature, which is the curvature of the spine, for all points included in the spine edge point group P by fitting an n-th order power function or a spline function to the spine edge point group P. For example, the feature amount calculation unit 105 fits a cubic function to the spine edge point group P at each of all points in the spine edge point group P to calculate the spinal curvature at each point included in the spine edge point group P.

[0034] Next, the calculation process of the spinal curvature angle by the feature calculation unit 105 will be described in detail with reference to Fig. 7. As shown in Fig. 7, the feature calculation unit 105 calculates, as the spinal curvature angle, the angle θ3 formed between the normal to the spine edge point group P at an end point P4 on the cranial side of the spine edge point group P and the normal to the spine edge point group P at an end point P5 on the caudal side of the spine edge point group P. Similarly, the feature amount calculation unit 105 calculates the upper thoracic curve angle, the lower thoracic curve angle, and the lumbar curve angle. Specifically, the positions of the upper thoracic vertebrae, the lower thoracic vertebrae, and the lumbar vertebrae in the spine are determined in advance as positions 0% to A%, A% to B%, and B% to C% of the spine from the cranial side of the spine, respectively. The feature amount calculation unit 105 then calculates, as the upper thoracic curve angle, the angle formed between a normal to the spinal column edge point group P at the cranial end point of the upper thoracic vertebra and a normal to the spinal column edge point group P at the caudal end point of the upper thoracic vertebra. Similarly, the feature calculation unit 105 calculates the angle between the normal to the spinal column edge point group P at the cranial end point of the lower thoracic vertebrae and the normal to the spinal column edge point group P at the caudal end point of the lower thoracic vertebrae as the lower thoracic curvature angle. The feature amount calculation unit 105 also calculates the angle between the normal to the spinal column edge point group P at the cranial end point of the lumbar vertebrae and the normal to the spinal column edge point group P at the caudal end point of the lumbar vertebrae as the lumbar curvature angle.

[0035] Next, the hip joint angle calculation process performed by the feature calculation unit 105 will be described in detail with reference to Fig. 8. The feature calculation unit 105 obtains a tangent line l4 to the spine edge point group P at the caudal end point P5 of the spine edge point group P. The tangent line l4 represents the angle of the posterior surface of the sacrum. Next, the feature calculation unit 105 calculates the angle θ4 between the tangent line l4 and the line segment l3 connecting the hip joint key point P2 and the knee joint key point P3, as the hip joint angle.

[0036] The feature calculation unit 105 then inputs the calculated features, such as spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, and lumbar curvature angle, to the state estimation unit 106.

[0037] The state estimation unit 106 estimates at least the state of the spine based on the feature amounts input from the feature calculation unit 105. Specifically, the state estimation unit 106 estimates the states of the upper thoracic vertebrae, lower thoracic vertebrae, lumbar vertebrae, lumbar-sacral junction (L5 / S1), and hip joints based on the spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, lumbar curvature angle, etc. input from the feature calculation unit 105. Then, the state estimation unit 106 inputs the estimation result to the image generation unit 107.

[0038] For example, the state estimation unit 106 estimates the curvature of each part of the spine based on the spinal curvature input from the feature calculation unit 105. For example, if the sign of the spinal curvature at a certain point in the spine edge point group P is positive, the curvature of the spine at that point is assumed to be convex toward the front of the person O shown in the image captured by the imaging unit 101. In this case, if the sign of the spinal curvature at that point in the spine edge point group P is negative, the curvature of the spine at that point is assumed to be convex toward the rear of the person O. Therefore, the state estimation unit 106 estimates the curvature of each part of the spine based on the sign of the spinal curvature. In other words, if the sign of the spinal curvature is positive, the state estimation unit 106 estimates that the curvature of the spine at that point is convex toward the front of the person O. Furthermore, if the sign of the spinal curvature is negative, the state estimation unit 106 estimates that the curvature of the spine at that point is convex toward the rear of the person O.

[0039] Furthermore, the state estimation unit 106 estimates the state of the spine, etc., based on the type of posture input from the posture determination unit 104 and the feature amounts input from the feature amount calculation unit 105. Specifically, a reference value list 112 that associates the type of posture with the reference value of the feature amount for that posture is stored in advance in the storage unit 110. The state estimation unit 106 then estimates the state of the spine, etc., by referring to the reference value list 112, based on the type of posture input from the posture determination unit 104 and the feature amounts input from the feature amount calculation unit 105. Here, the reference value is a value within the range that the feature amount can take when a body part such as the spine is normal.

[0040] For example, as shown in FIG. 9, the state estimation unit 106 compares the reference value of the upper thoracic curve angle when the type of posture is forward bending with the upper thoracic curve angle input from the feature calculation unit 105, thereby estimating the state of the upper thoracic vertebrae as “hyperflexion.” Similarly, the state estimation unit 106 compares the reference value of the lower thoracic vertebral curvature angle when the posture type is forward bending with the lower thoracic vertebral curvature angle input from the feature calculation unit 105, and thereby estimates the state of the lower thoracic vertebrae as “insufficient flexion.” Similarly, the state estimation unit 106 estimates the state of the lumbar spine as “normal” by comparing the reference value of the lumbar curvature angle when the posture type is forward bending with the lumbar curvature angle input from the feature calculation unit 105. Similarly, the state estimation unit 106 compares the reference value of the lumbar curvature angle when the posture type is forward bending with the lumbar curvature angle input from the feature calculation unit 105, and estimates the state of the lumbar-sacral junction (L5 / S1) to be "normal." Similarly, the state estimation unit 106 compares the reference value of the hip joint angle when the posture type is forward bending with the hip joint angle input from the feature calculation unit 105, and estimates the state of the hip joint to be “insufficient flexion.”

[0041] Furthermore, a state estimation model (not shown) that has undergone machine learning may be stored in advance in the storage unit 110, and the state estimation unit 106 may estimate at least the state of the spine using the state estimation model. Specifically, the storage unit 110 may store in advance training data in which at least feature quantities related to the spine are associated with state labels such as "hyperflexion," "normal," or "insufficient flexion" as correct answer data. The storage unit 110 may then store in advance a state estimation model that has undergone machine learning using the training data.

[0042] The image generation unit 107 generates an estimation result display image to be displayed by the display unit 108 based on the normalized image, normalized key points P1, P2, P3, and extracted spine edge point group P input from the spine extraction unit 103, and the estimation result input from the state estimation unit 106. In addition, the image generation unit 107 may generate a correction image for the user to correct the spine edge point group P based on the normalized image and the extracted spine edge point group P input from the spine extraction unit 103. Then, the image generating unit 107 inputs the generated image to the display unit .

[0043] The display unit 108 displays the estimation result display image input from the image generation unit 107. The display unit 108 is configured with various display means such as an LCD (Liquid Crystal Display) or an LED (Light Emitting Diode). Fig. 10 shows an example of the estimation result display image displayed on the display unit 108. Fig. 11 shows another example of the estimation result display image displayed on the display unit 108.

[0044] 10, an image portion G1 in which a key point P6 and a line segment 15 connecting the key point P6 are superimposed on an image captured by the imaging unit 101 is displayed on the upper side of the display unit 108 of the posture evaluation device 100A, and an image portion G2 showing the estimation result of the state estimation unit 106 is displayed on the lower side of the display unit 108. Furthermore, in the image portion G2 showing the estimation result, estimation results other than "normal" may be displayed in bold, red, or the like, with emphasis.

[0045] 11, the display unit 108 of the posture evaluation device 100A displays an image G3 in which the spine edge point cloud P, which is color-coded based on the key point P7 and the estimation results of the state estimation unit 106, is superimposed on the image captured by the imaging unit 101. For example, the display unit 108 shown in Fig. 11 displays each portion A, B, C, D, and E of the spine edge point cloud P color-coded based on whether it is convex toward the front or convex toward the back of the person O appearing in the image captured by the imaging unit 101. Furthermore, the color coding of each portion A, B, C, D, and E of the spine edge point cloud P is performed based on the degree of protrusion of each portion A, B, C, D, and E (the magnitude of the value of the spinal curvature). Furthermore, the display unit 108 may divide the spinal column edge point group P into, for example, upper thoracic vertebrae, lower thoracic vertebrae, and lumbar vertebrae, and display each part in a color according to the state of each part. Furthermore, the spine edge point cloud P may be colored in a gradation that gradually changes color.

[0046] Furthermore, the display unit 108 may display the correction image input from the image generation unit 107. This allows the user to correct, for example, when the range extracted from the spine edge point group P by the spine extraction unit 103 is incorrect, by dragging the spine edge point group P displayed on the correction required screen.

[0047] The input unit 109 accepts operation instructions from a user. The input unit 109 may be configured with a keyboard or a touch panel display device. The input unit 109 may be configured with a keyboard or a touch panel connected to the main body of the posture evaluation device 100A.

[0048] The storage unit 110 stores a reference value list 112, a skeleton database 113, a skeleton extraction model 114, and the like. The storage unit 110 may also include a non-volatile memory (for example, a ROM (Read Only Memory)) in which various programs and various data required for processing are fixedly stored. The storage unit 110 may also use an HDD (Hard Disk Drive) or an SSD. The storage unit 110 may also include a volatile memory (for example, a RAM (Random Access Memory)) used as a working area. The programs may be read from a portable recording medium such as an optical disk or a semiconductor memory, or may be downloaded from a server device on a network.

[0049] Reference value list 112 is a list in which types of postures are associated with reference values ​​of feature quantities for the postures. Fig. 12 shows an example of the data structure of reference value list 112. As shown in Fig. 12, reference value list 112 is a list in which types of postures 112A, reference values ​​112B of spinal curvature for the postures, reference values ​​112C of spinal curvature angles, reference values ​​112D of hip joint angles, reference values ​​112E of upper thoracic curvature angles, reference values ​​112F of lower thoracic curvature angles, and reference values ​​112G of lumbar curvature angles are associated with each other.

[0050] The skeleton database 113 is a database in which a plurality of images obtained by capturing images of the side of the body are associated with position information of key points as correct labels.

[0051] The skeleton extraction model 114 is a machine learning model that extracts position information of key points from an image obtained by capturing an image of the side of the body. In other words, the skeleton extraction model 114 is a machine learning model that uses an image obtained by capturing an image of the side of the body as input, infers position information of key points, and outputs the position information. Note that in this specification, machine learning may be, but is not limited to, deep learning.

[0052] The communication unit 111 communicates with an external server, other terminal devices, etc. The communication unit 111 may include an antenna (not shown) for wireless communication, or may include an interface such as a NIC (Network Interface Card) for wired communication.

[0053] Next, a posture evaluation method according to the second embodiment will be described with reference to Fig. 13. First, the imaging unit 101 captures an image of the side of the body (step S101), and inputs the obtained image to the skeleton extraction unit 102 and the spine extraction unit 103.

[0054] Next, the skeleton extraction unit 102 extracts position information of key points from the image captured by the imaging unit 101 in step S101 (step S102), and inputs the extracted position information of key points to the spine extraction unit 103. For example, the skeleton extraction unit 102 extracts position information of a vertebral prominence key point P1, a hip joint key point P2, and a knee joint key point P3.

[0055] Next, the spine extraction unit 103 extracts a spine edge point cloud based on the image captured by the imaging unit 101 in step S101 and the position information of the key points P1, P2, and P3 extracted by the skeleton extraction unit 102 in step S102 (step S103). Specifically, the spine extraction unit 103 normalizes the image captured by the imaging unit 101 and extracts the line segment l trunk and line segment l thigh The spine extraction unit 103 then calculates the exterior angle θ0 between the key points P1, P2, and P3 and extracts the spine edge point group P. The spine extraction unit 103 then inputs the calculated exterior angle θ0 to the posture determination unit 104. The spine extraction unit 103 also inputs the key points P1, P2, and P3 and the spine edge point group P to the feature calculation unit 105. The spine extraction unit 103 also inputs the normalized image, the key points P1, P2, and P3, and the spine edge point group P to the image generation unit 107.

[0056] Next, the feature calculation unit 105 calculates the spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, lumbar curvature angle, etc. as feature amounts based on the key points P1, P2, and P3 and the spinal edge point group P input from the spine extraction unit 103 (step S104).The feature calculation unit 105 then inputs the calculated spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, lumbar curvature angle, etc. to the state estimation unit 106.

[0057] Furthermore, the posture determination unit 104 determines the type of posture of the person O appearing in the image captured by the imaging unit 101 based on the exterior angle θ0 input from the spine extraction unit 103 (step S105). Then, the posture determination unit 104 inputs the determined type of posture to the state estimation unit 106.

[0058] Next, the state estimation unit 106 estimates the states of the upper thoracic vertebrae, lower thoracic vertebrae, lumbar vertebrae, lumbar-sacral junction (L5 / S1), and hip joints (step S106) based on the type of posture input from the posture determination unit 104 and the spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, lumbar curvature angle, etc. input from the feature calculation unit 105. Then, the state estimation unit 106 inputs the estimation results to the image generation unit 107.

[0059] Next, the image generation unit 107 generates an image to be displayed on the display unit 108 based on the image captured in step S101 and the states of the upper thoracic vertebrae, lower thoracic vertebrae, lumbar vertebrae, lumbar-sacral junction (L5 / S1), and hip joints estimated in step S106 (step S107). The image generated by the image generation unit 107 is then input to the display unit 108.

[0060] Next, the display unit 108 displays the image generated in step S107 (step S108), and this process ends.

[0061] The key points P1, P2, and P3 may be specified by displaying the image captured by the imaging unit 101 on the display unit 108 and the user operating the input unit 109. In this case, the processing of step S102 may be omitted. Furthermore, after the processing of step S103 and before the processing of step S104, a correction image displaying the spine edge point group P extracted in step S103 may be displayed on the display unit 108, and the user may correct the spine edge point group P displayed on the correction required screen by dragging it. Furthermore, the order of the processing in step S104 and the processing in step S105 may be reversed, and the processing in step S104 and the processing in step S105 may be performed simultaneously. Furthermore, before the processing of step S106 is performed, the user may specify the type of posture by operating the input unit 109. In this case, the processing of step S105 may be omitted.

[0062] According to the second embodiment, a posture evaluation device 100A can be provided that can evaluate posture with high accuracy at a relatively low cost. Specifically, the spine extraction unit 103 extracts a spine edge point group P representing the spine shape on the image, and the feature calculation unit 105 calculates the spinal curvature, spinal curvature angle, hip joint angle, upper thoracic curvature angle, lower thoracic curvature angle, and lumbar curvature angle as feature amounts based on the key points P1, P2, and P3 on the image and the spine edge point group P. Then, the state estimation unit 106 estimates the state of the upper thoracic vertebrae, lower thoracic vertebrae, lumbar vertebrae, lumbar-sacral junction (L5 / S1), and hip joints based on the feature amounts. In other words, posture can be evaluated based on the spine shape on the image, thereby enabling highly accurate posture evaluation. Furthermore, posture can be evaluated relatively inexpensively based on the spine shape without using expensive specialized equipment. Therefore, it is possible to provide a posture evaluation device 100A that can evaluate posture with high accuracy at a relatively low cost.

[0063] Furthermore, the skeleton extraction unit 102 extracts key points P1, P2, and P3 from the image captured by the imaging unit 101. This eliminates the need for the user to specify the key points P1, P2, and P3 on the image. Furthermore, the skeleton extraction unit 102 may use the trained skeleton extraction model 114 to extract key points expressed in three-dimensional coordinates defined by the z direction, which is the depth direction, in addition to the x direction, which is the left-right or horizontal direction, and the y direction, which is the up-down or vertical direction, of the two-dimensional image captured by the imaging unit 101. This enables more precise posture evaluation.

[0064] Furthermore, the state estimation unit 106 estimates the state of the spine, etc., based on the type of posture and the feature amount by referring to the reference value list 112. Here, the reference value is a value within the range that the feature amount can take when the body part, such as the spine, is normal. Therefore, the state estimation unit 106 can estimate whether the body part, such as the spine, is normal.

[0065] Furthermore, the posture determination unit 104 determines the type of posture of the person O for which posture evaluation is to be performed, based on the key points extracted by the skeleton extraction unit 102. Therefore, the user does not need to specify the type of posture for which posture evaluation is to be performed.

[0066] Furthermore, the display unit 108 displays an estimation result display image that displays the image captured by the imaging unit 101 and the estimation result by the state estimation unit 106. This allows the user to visually grasp the state of their posture.

[0067] Furthermore, the display unit 108 displays the state estimated by the state estimation unit 106 by color-coding the spinal column edge point group P. This allows the user to visually grasp the state of the posture.

[0068] Embodiment 3 Next, a posture evaluation device 100B according to the third embodiment will be described with reference to FIG. 14. FIG. 14 is a block diagram showing the configuration of the posture evaluation device 100B according to the third embodiment. The posture evaluation device 100B according to the third embodiment differs from the posture evaluation device 100A according to the second embodiment in that it includes a first imaging unit 101A and a second imaging unit 101B. Furthermore, as shown in FIG. 15, the image generated by the image generation unit 107A, i.e., the image displayed on the display unit 108A, is also different. Therefore, among the configuration of the posture evaluation device 100B according to the third embodiment, the same components as those of the posture evaluation device 100A according to the second embodiment are denoted by the same reference numerals, and their description will be omitted.

[0069] The first imaging unit 101A captures an image of the side of the subject's body, similar to the imaging unit 101 according to embodiment 2. The first imaging unit 101A inputs the captured image to the skeleton extraction unit 102, the spine extraction unit 103, and the image generation unit 107A. Furthermore, the first imaging unit 101A may acquire an image by capturing a moving image of the side of the subject's body, similar to the imaging unit 101 according to the second embodiment. In this case, the user operates the input unit 109 to specify a time point at which posture evaluation is to be performed in the image selection area G4 (see FIG. 15 ) displayed on the display unit 108A. Then, the image at the time point specified by the user is input to the skeleton extraction unit 102 and the spine extraction unit 103. In the third embodiment, a case where the first imaging unit 101A captures a moving image of the side of the subject will be described as an example.

[0070] The second imaging unit 101B simultaneously images another surface of the subject's body with the first imaging unit 101A. Here, the other surface of the subject's body may be any surface other than the side of the subject's body. The second imaging unit 101B inputs the captured image to the image generation unit 107A. In addition, second image capturing unit 101B may capture a moving image of another side of the subject's body to acquire an image. In the third embodiment, a case where second image capturing unit 101B captures a moving image of the front of the subject's body will be described as an example.

[0071] Based on the image input from the second imaging unit 101B, the image generation unit 107A generates an image selection area G4 to be displayed on the display unit 108A. As shown in Fig. 15, the image selection area G4 is an image area in which a plurality of thumbnail images G5 of the moving image input from the second imaging unit 101B are arranged along a time scale T1, and a designation bar T2 for designating a time point for posture evaluation is displayed so as to be movable along the time scale T1. Furthermore, image generation unit 107A generates a front image area G6 that shows a front image that displays an image at a point in time designated by the user in image selection area G4 displayed on display unit 108A. In addition, based on the normalized image and key points input from the spine extraction unit 103, the image generation unit 107A generates a side image portion G7 in which the key point P8 and the line segment 16 connecting the key point P8 are superimposed on the image. Furthermore, based on the estimation result input from the state estimation unit , the image generation unit 107A generates a result image area G8 that indicates the estimation result of the state estimation unit . Then, image generation unit 107A inputs the generated image selection area G4, front image area G6, side image area G7, and resultant image area G8 to display unit 108A.

[0072] Display unit 108A displays image selection area G4, front image area G6, side image area G7, and resultant image area G8 input from image generation unit 107. Fig. 15 shows an example of an image displayed on display unit 108A.

[0073] 15, an image selection area G4 is displayed at the bottom of display unit 108A of posture evaluation device 100B, a front image area G6 is displayed at the top left of display unit 108A, a side image area G7 is displayed at the top center of display unit 108A, and a result image area G8 is displayed at the top right of display unit 108A. In the example shown in Fig. 15, the user moves designation bar T2 in image selection area G4 to designate 0 minutes 11 seconds as the time point for posture evaluation.

[0074] According to the third embodiment, the user can check information about the posture that cannot be obtained by checking the side of the body by checking the front image area G6 displayed on the display unit 108A. For example, the user can check from the front image area G6 whether the left and right sides of the body are moving evenly.

[0075] Embodiment 4 Next, a posture evaluation system 200 according to the fourth embodiment will be described with reference to Fig. 16. Fig. 16 is a diagram showing the configuration of the posture evaluation system 200 according to the fourth embodiment. As shown in Fig. 16, the posture evaluation system 200 includes a posture evaluation device 100C and a subject terminal 300 capable of communicating with the posture evaluation device 100C. The posture evaluation device 100C and the subject terminal 300 are capable of communicating with each other via a network N. Furthermore, as shown in Fig. 16, one or more subject terminals 300, ... may be capable of communicating with the posture evaluation device 100C. The subject terminal 300 is a smartphone, tablet terminal, personal computer, or the like that is owned by the subject.

[0076] The posture evaluation device 100C according to the fourth embodiment acquires an image of the side of the subject's body from the subject terminal 300. Therefore, the posture evaluation device 100C differs from the posture evaluation device 100A according to the second embodiment in that the imaging unit 101 may be omitted. Furthermore, the estimation result display image created by the image generation unit 107 of the posture evaluation device 100C may be transmitted to the subject terminal 300 and displayed on a display unit (not shown) of the subject terminal 300.

[0077] The subject terminal 300 includes an imaging unit (not shown) that captures an image of the side of the subject's body. The subject terminal 300 transmits the image to the posture evaluation device 100C.

[0078] Other embodiments Next, a brief description will be given of posture evaluation methods according to other embodiments. The posture evaluation systems according to the other embodiments are modified examples of the posture evaluation system 200. A posture evaluation device 100C according to the other embodiments acquires images of the side and other sides of the subject's body simultaneously from a subject terminal 300. In this case, the posture evaluation device 100C differs from the posture evaluation device 100B according to the third embodiment in that the first imaging unit 101A and the second imaging unit 101B may be omitted. In addition, the image selection area G4, front image area G6, side image area G7, and result image area G8 created by the image generation unit 107A of the posture evaluation device 100C may be transmitted to the subject terminal 300 and displayed on a display unit (not shown) of the subject terminal 300.

[0079] The subject terminal 300 includes a first imaging unit (not shown) that captures an image of the side of the subject's body, and a second imaging unit (not shown) that captures an image of another side of the subject's body simultaneously with the first imaging unit. The subject terminal 300 transmits the image of the side of the body and the image of the other side of the body to the posture evaluation device 100C.

[0080] According to the fourth embodiment and other embodiments, at least the side of the subject's body is imaged by the subject terminal 300, and the acquired image is transmitted to the posture evaluation device 100C via the network N, allowing posture evaluation to be performed by the posture evaluation device 100C. Therefore, for example, even if the subject and the evaluator are in different locations, the evaluator can remotely evaluate the subject's posture. The posture evaluation system 200 according to the fourth embodiment and other embodiments is particularly advantageous in situations such as remote therapy and remote training.

[0081] In the above-described embodiment, the present disclosure has been described as a hardware configuration, but the present disclosure is not limited to this. The present disclosure can also be realized by having a CPU (Central Processing Unit) execute a computer program to perform the processing steps shown in the flowchart of FIG. 13 and the processing steps described in the other embodiments.

[0082] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0083] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above. Various modifications within the scope of the invention that are understandable to those skilled in the art can be made to the configuration and details of the present invention. For example, if the subject is wearing clothing that obscures the lines of their body, the spine extraction unit 103 cannot extract the spine edge point group P by directly using the edge points extracted by the edge extraction process as the candidate edge point group. Therefore, the spine extraction unit 103 may perform a candidate edge point group estimation process based on the edge points extracted by the edge extraction process and the vertebral prominence key point P1, the hip joint key point P2, and the knee joint key point P3.

[0084] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a spine extraction means for extracting a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image based on an image obtained by capturing an image of the side of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; a feature amount calculation means for calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; a state estimation means for estimating at least the state of the spine based on the feature amount; A posture assessment device comprising: (Appendix 2) a skeleton extraction means for extracting the position information from the image; 2. The posture assessment device of claim 1. (Appendix 3) a storage means for storing the type of body posture and the reference value of the feature amount for the type of posture in association with each other; the state estimation means estimates at least a state of the spine based on the type of posture, the feature amount calculated by the feature amount calculation means, and the reference value. 3. The posture assessment device according to claim 1 or 2. (Appendix 4) a posture determination means for determining the type of posture based on the position information; 4. The posture assessment device of claim 3. (Appendix 5) a display means for displaying the image and the state estimated by the state estimation means, 5. A posture evaluation device according to any one of appendices 1 to 4. (Appendix 6) the display means displays the state by color-coding the spine edge point cloud. 6. The posture assessment device of claim 5. (Appendix 7) when the spine edge point cloud is extracted by the spine extraction means, the display means displays the spine edge point cloud together with the image in a manner that allows the user to modify the spine edge point cloud. 7. The posture assessment device according to claim 5 or 6. (Appendix 8) a first imaging means for imaging a side of the body, and a second imaging means for imaging another side of the body simultaneously with the first imaging means; the display means displays an image of the side of the body captured by the first imaging means together with an image of the other side of the body captured by the second imaging means. 8. A posture evaluation device according to any one of appendices 5 to 7. (Appendix 9) A posture evaluation device and a subject terminal capable of communicating with the posture evaluation device, The posture evaluation device includes: a spine extraction means for extracting a spine edge point cloud consisting of a predetermined number of points representing a spine shape on the image based on an image of the side of the subject's body acquired by the subject terminal and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; a feature amount calculation means for calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; a state estimation means for estimating at least the state of the spine based on the feature amount; A posture assessment system comprising: (Appendix 10) The posture evaluation device includes: a skeleton extraction means for extracting the position information from the image; 10. The posture assessment system of claim 9. (Appendix 11) The posture evaluation device includes: a storage means for storing the type of body posture and the reference value of the feature amount for the type of posture in association with each other; the state estimation means estimates at least a state of the spine based on the type of posture, the feature amount calculated by the feature amount calculation means, and the reference value. 11. The posture assessment system of claim 9 or 10. (Appendix 12) The posture evaluation device includes: a posture determination means for determining the type of posture based on the position information; 12. The posture assessment system of claim 11. (Appendix 13) The posture evaluation device includes: a display means for displaying the image and the state estimated by the state estimation means, 13. A posture evaluation system according to any one of appendices 9 to 12. (Appendix 14) the display means displays the state by color-coding the spine edge point cloud. 14. The posture assessment system of claim 13. (Appendix 15) when the spine edge point cloud is extracted by the spine extraction means, the display means displays the spine edge point cloud together with the image in a manner that allows the user to modify the spine edge point cloud. 15. The posture assessment system of claim 13 or 14. (Appendix 16) The posture evaluation device includes: a first imaging means for imaging a side of the body, and a second imaging means for imaging another side of the body simultaneously with the first imaging means; the display means displays an image of the side of the body captured by the first imaging means together with an image of the other side of the body captured by the second imaging means. 16. A posture evaluation system according to any one of appendices 13 to 15. (Appendix 17) The posture evaluation device extracting a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image based on an image obtained by capturing an image of the side of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; estimating at least the state of the spine based on the feature amount; Posture assessment methods. (Appendix 18) The posture evaluation device extracting the position information from the image; 17. A posture assessment method as described in Appendix 17. (Appendix 19) The posture evaluation device storing the type of body posture and the reference value of the feature amount for the type of posture in association with each other; estimating at least the state of the spine based on the type of posture, the feature amount, and the reference value; 19. The posture assessment method according to claim 17 or 18. (Appendix 20) The posture evaluation device determining the type of the posture based on the position information; 19. A posture assessment method as described in Appendix 19. (Appendix 21) The posture evaluation device displaying the image and the status; 21. A posture assessment method according to any one of appendices 17 to 20. (Appendix 22) The posture evaluation device The state is displayed by color-coding the spine edge point cloud. 22. A posture evaluation method according to any one of appendices 17 to 21. (Appendix 23) The posture evaluation device When the spine edge point cloud is extracted, the spine edge point cloud is displayed together with the image so that the user can modify the spine edge point cloud. 23. A posture evaluation method according to any one of appendices 17 to 22. (Appendix 24) The posture evaluation device imaging a side of the body while simultaneously imaging another side of the body; displaying an image of the side of the body captured together with an image of another side of the body captured; 24. A posture assessment method according to any one of appendices 17 to 23. (Appendix 25) Posture assessment device, extracting a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image based on an image obtained by capturing an image of the side of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; a process of calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; a process of estimating at least the state of the spine based on the feature amount; A non-transitory computer-readable medium that stores a program that causes the program to execute. (Appendix 26) The posture evaluation device includes: Extracting the position information from the image 26. A non-transitory computer-readable medium as recited in claim 25, storing a program for causing the computer to execute (Appendix 27) The posture evaluation device includes: a process of storing the type of body posture and the reference value of the feature amount for the type of posture in association with each other; a process of estimating at least a state of the spine based on the type of posture, the feature amount, and the reference value; 27. A non-transitory computer-readable medium according to claim 25 or 26, storing a program for causing the computer to execute the steps of: (Appendix 28) The posture evaluation device includes: a process of determining the type of the posture based on the position information; 28. A non-transitory computer-readable medium as recited in claim 27, storing a program for causing the computer to execute (Appendix 29) The posture evaluation device includes: displaying the image and the status; 29. A non-transitory computer-readable medium according to any one of appendices 25 to 28, storing a program for executing the above. (Appendix 30) The posture evaluation device includes: a process of displaying the state by color-coding the spine edge point cloud; 30. A non-transitory computer-readable medium according to any one of appendices 25 to 29, storing a program for executing the above. (Appendix 31) The posture evaluation device includes: When the spine edge point cloud is extracted, a process of displaying the spine edge point cloud together with the image so that the spine edge point cloud can be modified by a user; 31. A non-transitory computer-readable medium according to any one of appendices 25 to 30, storing a program for executing the above. (Appendix 32) The posture evaluation device includes: imaging a side of the body while simultaneously imaging another side of the body; displaying the captured image of the side of the body together with the captured image of the other side of the body; 32. A non-transitory computer-readable medium according to any one of appendices 25 to 31, storing a program for executing the above.

[0085] This application claims priority based on Japanese Patent Application No. 2022-058198, filed on March 31, 2022, the disclosure of which is incorporated herein in its entirety. [Industrial Applicability]

[0086] A posture evaluation device, a posture evaluation system, and a posture evaluation method that can evaluate posture with high accuracy at a relatively low cost. program can be provided. [Explanation of symbols]

[0087] 100, 100A, 100B, 100C Posture evaluation device 101 Imaging unit (imaging means) 101A First imaging unit (first imaging means) 101B second imaging unit (second imaging means) 102 skeleton extraction unit (skeleton extraction means) 103 Spinal column extraction part (vertebral column extraction means) 104 Posture judgment unit (posture judgment means) 105 Feature calculation unit (feature calculation means) 106 State estimation unit (state estimation means) 107, 107A Image generation section 108, 108A Display section (display means) 109 Input section 110 Storage unit (storage means) 111 Communications Department 112 Reference Value List 113 Skeleton DB (Skeletal Database) 114 Skeleton Extraction Model 200 Posture Assessment System 300 Target Device

Claims

1. a spine extraction means for extracting a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image based on an image obtained by capturing an image of the side of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; a feature amount calculation means for calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; a state estimation means for estimating at least the state of the spine based on the feature amount; A posture assessment device comprising:

2. a skeleton extraction means for extracting the position information from the image; The posture evaluation device according to claim 1 .

3. a storage means for storing the type of body posture and the reference value of the feature amount for the type of posture in association with each other; the state estimation means estimates at least a state of the spine based on the type of posture, the feature amount calculated by the feature amount calculation means, and the reference value. The posture evaluation device according to claim 1 .

4. a posture determination means for determining the type of posture based on the position information; The posture evaluation device according to claim 3 .

5. a display means for displaying the image and the state estimated by the state estimation means, The posture evaluation device according to any one of claims 1 to 4.

6. the display means displays the state by color-coding the spine edge point cloud. The posture evaluation device according to claim 5 .

7. when the spine edge point cloud is extracted by the spine extraction means, the display means displays the spine edge point cloud together with the image in a manner that allows the user to modify the spine edge point cloud. The posture evaluation device according to claim 5 .

8. A posture evaluation device and a subject terminal capable of communicating with the posture evaluation device, The posture evaluation device includes: a spine extraction means for extracting a spine edge point cloud consisting of a predetermined number of points representing a spine shape on the image based on an image of the side of the subject's body acquired by the subject terminal and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; a feature amount calculation means for calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; a state estimation means for estimating at least the state of the spine based on the feature amount; A posture assessment system comprising:

9. The posture evaluation device extracting a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image based on an image obtained by capturing an image of the side of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; estimating at least the state of the spine based on the feature amount; Posture assessment methods.

10. Posture assessment device, extracting a spine edge point cloud consisting of a predetermined number of points representing the spine shape on the image based on an image obtained by capturing an image of the side of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image; a process of calculating at least a feature amount related to the spine based on the position information and the spine edge point cloud; a process of estimating at least the state of the spine based on the feature amount; A program that executes the following.

Images