Posture measurement apparatus, posture measurement method, and program

Abstract

To provide a posture measurement device, a posture measurement system, a posture measurement method, and a program capable of easily and accurately measuring the shape of the back of a subject even in a scene where the shape of the back of the subject cannot be accurately grasped due to the presence of clothes of the subject.SOLUTION: A posture measurement device according to the present disclosure includes a holding unit, a calculation unit, and an optimization unit. The holding unit holds a shape model representing a shape of a back of a subject as a function of a plurality of parameters. The calculation unit calculates a value of a predetermined objective function using, as inputs, a shape model and a candidate point group that is a candidate for a point group indicating a shape of a back of a target person. The optimization unit corrects the plurality of parameters so as to minimize a value of a predetermined objective function by using constraint conditions of the plurality of parameters and the shape model as inputs.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a posture measurement device, a posture measurement method, a program, and a posture measurement system. [Background technology]

[0002] In recent years, along with the rise in the need for self-care for musculoskeletal symptoms such as lower back pain and the rise in online training, there has been a growing need for accurate and quantitative assessment of posture using images obtained by cameras installed in terminal devices such as smartphones.

[0003] Patent Document 1 describes a posture evaluation device that includes a spine extraction means, a feature calculation means, and a state estimation means, and that can evaluate posture relatively inexpensively and with high accuracy. The spine extraction means extracts a spine edge point cloud consisting of a predetermined number of points representing the shape of the spine on the image based on an image obtained by capturing a side view of the subject's body and position information of at least the cervical vertebrae, hip joints, and knee joints of the body on the image. The feature calculation means calculates feature amounts related to at least the spine based on the position information and the spine edge point cloud. The state estimation means estimates at least the state of the spine based on the calculated feature amounts. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2023 / 188796 Summary of the Invention [Problem to be solved by the invention]

[0005] When evaluating posture using images, if the person being evaluated is wearing thick clothing or is bending backward, the shape of the back of the body is hidden by the clothing. Therefore, it is not possible to accurately estimate from the image the shape of the back that would appear if the person were in the same posture without clothing, and as a result, posture cannot be evaluated accurately. The technology described in Patent Document 1 cannot solve these problems.

[0006] Therefore, it is desirable to develop a technology that can easily and accurately measure the shape of a subject's back even in situations where the shape of the subject's back cannot be accurately grasped due to the presence of the subject's clothing.

[0007] The object of the present disclosure is to provide a posture measurement device, a posture measurement method, a program, and a posture measurement system that can easily and accurately measure the shape of a subject's back even in situations where the shape of the subject's back cannot be accurately grasped due to the presence of the subject's clothing. [Means for solving the problem]

[0008] A posture measurement device according to a first aspect of the present disclosure includes a storage unit that stores a shape model that represents the shape of a subject's back as a function of multiple parameters; a calculation unit that calculates the value of a predetermined objective function using as input a candidate point cloud that is a candidate for a point cloud that represents the shape of the subject's back and the shape model; and an optimization unit that modifies the multiple parameters to minimize the value of the predetermined objective function using as input constraints for the multiple parameters and the shape model.

[0009] A posture measurement method according to a second aspect of the present disclosure includes a posture measurement device that holds a shape model that represents the shape of a subject's back as a function of multiple parameters, calculates the value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as inputs, and modifies the multiple parameters to minimize the value of the predetermined objective function using constraints on the multiple parameters and the shape model as inputs.

[0010] A program according to a third aspect of the present disclosure is a program that causes a computer to execute the following processes: retaining a shape model that represents the shape of a subject's back as a function of multiple parameters; calculating the value of a predetermined objective function using as input a candidate point cloud that is a candidate for a point cloud that represents the shape of the subject's back and the shape model; and modifying the multiple parameters so as to minimize the value of the predetermined objective function using as input constraints on the multiple parameters and the shape model. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to provide a posture measurement device, a posture measurement method, a program, and a posture measurement system that can easily and accurately measure the shape of a subject's back even in situations where the shape of the subject's back cannot be accurately grasped due to the presence of the subject's clothing. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating an example of the configuration of a posture measurement device according to the present disclosure. [Figure 2] 1 is a block diagram illustrating an example of the configuration of a posture measurement device according to the present disclosure. [Figure 3] FIG. 10 is a diagram showing an example of a back shape model stored for an image acquired by the posture measurement device according to the present disclosure. [Figure 4] FIG. 10 is a diagram showing another example of a back shape model stored for an image acquired by the posture measurement device according to the present disclosure. [Figure 5] 4 is a schematic diagram for explaining an example of a calculation method and an objective function calculation method in the back shape model of FIG. 3. FIG. [Figure 6] 10A and 10B are diagrams illustrating mathematical expressions for explaining an example of processing in the posture measurement device according to the present disclosure. [Figure 7] 10A and 10B are schematic diagrams for explaining an example of candidate point group extraction processing in the orientation measurement device according to the present disclosure. [Figure 8] 10A and 10B are schematic diagrams for explaining another example of the candidate point group extraction process in the orientation measurement device according to the present disclosure. [Figure 9] 10A and 10B are schematic diagrams for explaining an example of an end point determination process in the orientation measurement device according to the present disclosure. [Figure 10] 10A and 10B are schematic diagrams for explaining an example of edge addition processing in the posture measurement device according to the present disclosure. [Figure 11] 10A and 10B are schematic diagrams for explaining an example of a skeleton extraction process in the posture measurement device according to the present disclosure. [Figure 12] FIG. 10 is a schematic diagram showing an example of skeleton key points extracted by a skeleton extraction process in the posture measurement device according to the present disclosure. [Figure 13] FIG. 10 is a diagram illustrating an example of an image displayed on a display unit in the posture measurement device according to the present disclosure. [Figure 14] 10A and 10B are schematic diagrams for explaining an example of a shooting angle determination process in the orientation measurement device according to the present disclosure. [Figure 15] 10A and 10B are schematic diagrams for explaining an example of a shooting angle determination process in the orientation measurement device according to the present disclosure. [Figure 16] 1 is a flowchart illustrating a posture assessment method according to the present disclosure. [Figure 17] FIG. 10 is a diagram showing the results of estimating the shape of the back in a bending posture according to a comparative example. [Figure 18] FIG. 10 is a diagram showing the results of estimating the shape of the back in a backward bending posture according to a comparative example. [Figure 19] 1 is a diagram illustrating an example of the configuration of a posture evaluation system according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, the same elements in each drawing are designated by the same reference numerals, and duplicate explanations have been omitted as necessary.

[0014] Embodiment 1 An example of the configuration of the posture measurement device 1 will be described below with reference to FIG. As shown in FIG. 1, the posture measurement device 1 includes a holding unit 1a, a calculation unit 1b, and an optimization unit 1c.

[0015] The storage unit 1a stores a shape model that represents the shape of the subject's back as a function of multiple parameters. The storage unit 1a can also be called a shape model storage unit. Since this shape model is a shape model of the back, it will be referred to as a back shape model hereinafter. Since the back shape model represents the shape of the back, it also represents the shape of the spine in a pseudo manner.

[0016] The calculation unit 1b receives as input a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and a back shape model, and calculates the value of a predetermined objective function. The calculation unit 1b can also be referred to as an objective function calculation unit.

[0017] The optimization unit 1c receives the constraints on the parameters and the back shape model as inputs and modifies the parameters so as to minimize the value of the predetermined objective function. The optimization unit 1c can be called a parameter optimization unit.

[0018] The posture measurement device 1 can use such an optimized back shape model as a shape model representing the shape of the subject's back, thereby measuring the shape of the back. Since the posture measurement device 1 is a device that estimates the shape of the back through optimization, it can also be called a posture estimation device.

[0019] The back shape model can be a shape model that represents the shape of the subject's back in the sagittal plane as a function of multiple parameters. In this case, the input candidate point cloud can be a point cloud candidate that represents the shape of the subject's back in the sagittal plane. However, the back shape model and candidate point cloud are not limited to the sagittal plane, and can also be used in a plane angled relative to the sagittal plane as long as the shape of the back can be grasped.

[0020] As described above, the posture measurement device 1 can execute a posture measurement method. This posture measurement method stores a back shape model, calculates the value of a predetermined objective function using a candidate point group and the back shape model as input, and modifies multiple parameters to minimize the value of the predetermined objective function using constraints on multiple parameters and the back shape model as input. This posture measurement method can also be realized by a program and hardware that executes that program. This program refers to a program that causes a computer to execute the processing described in the posture measurement method above.

[0021] Although not shown, a posture measurement system can also be constructed by including the posture measurement device 1 and a terminal device capable of communicating with the posture measurement device 1. In this case, the posture measurement device 1 may include, in addition to the holding unit 1a, the calculation unit 1b, and the optimization unit 1c, an acquisition unit that acquires, from the terminal device, a candidate point cloud that is a candidate for a point cloud indicating the shape of the subject's back, or an image that is a source of extraction of the candidate point cloud.

[0022] According to this embodiment, the stored back shape model is optimized based on the input candidate point group, making it possible to measure the shape of the subject's back simply and accurately even in situations where the presence of the subject's clothing makes it difficult to accurately grasp the shape of the back.

[0023] Embodiment 2 An example configuration of the posture measurement device 100 will be described using Fig. 2. The posture measurement device 100 includes the posture measurement device 1 shown in Fig. 1. The posture measurement device 100 includes a back shape model storage unit 112 as an example of a storage unit 1a, an objective function calculation unit 111 as an example of a calculation unit 1b, and a parameter optimization unit 113 as an example of an optimization unit 1c.

[0024] 2 is a user terminal such as a smartphone, tablet terminal, or personal computer owned by a user. Note that the term "user" includes both a subject whose posture is to be measured by the posture measurement device 100 and a measurer who measures the posture of another person using the posture measurement device 100. Furthermore, when a subject measures their own posture using the posture measurement device 100 in self-training or the like, the subject is also the measurer. Furthermore, when a measurer measures the posture of another person using the posture measurement device 100, the measurer is, for example, a therapist or trainer.

[0025] As shown in FIG. 2 , the posture measurement device 100 includes an image acquisition unit 101, an imaging angle determination unit 102, a skeleton extraction unit 103, an input unit 104, a back shape estimation unit 105, a feature calculation unit 106, a state estimation unit 107, a display unit 108, and a communication unit 109. The input unit 104 and the display unit 108 may be integrated into a single touch panel display, or may be provided separately. The back shape estimation unit 105 may include an objective function calculation unit 111, a back shape model storage unit 112, and a parameter optimization unit 113, as well as a candidate point group extraction unit 114, an end point determination unit 115, and an edge addition unit 116. The posture measurement device 100 may also include a storage unit (not shown). This storage unit stores, for example, the back shape model stored in the back shape model storage unit 112.

[0026] The image acquisition unit 101 acquires an image of the side of the body of a subject of posture measurement. The image acquisition unit 101 may include an imaging device for capturing images. An example of an image of the side of the body of a subject captured by the image acquisition unit 101 is shown in FIG. 3. The image shown in FIG. 3 shows the side of a person O corresponding to the subject bending forward. Here, the image acquired by the image acquisition unit 101 is a two-dimensional image, and may be a two-dimensional RGB image. The image acquisition unit 101 outputs the acquired image to the skeleton extraction unit 103 or the back shape estimation unit 105, or both. Note that this output destination differs depending on the processing example adopted in this embodiment.

[0027] The image acquisition unit 101 may also acquire a video of the side of the body of the subject of posture measurement. In this case, the user may specify the time point at which posture measurement and evaluation are performed by operating the input unit 104. Then, the image acquisition unit 101 may output the image at the time point specified by the user to an output destination.

[0028] The input unit 104 accepts operation instructions from a user. The input unit 104 may be configured with a keyboard, or as described above, may be combined with the display unit 108 to configure a display with a touch panel. The input unit 104 may be configured with a keyboard or a touch panel connected to the posture measurement device 100 main body.

[0029] The display unit 108 displays captured and acquired images, measurement results, etc. The display unit 108 is configured by various display means such as an LCD (Liquid Crystal Display), an LED (Light Emitting Diode), etc.

[0030] The communication unit 109 communicates with an external server, other terminal devices, etc. The communication unit 109 may include an antenna (not shown) for wireless communication, or an interface such as a NIC (Network Interface Card) for wired communication. The image acquisition unit 101 may be configured to acquire images from an imaging device via the communication unit 109. The input unit 104 may receive operation instructions from a user via a keyboard or touch panel connected to the posture measurement device 100 main body via the communication unit 109.

[0031] First, the main components of this embodiment, that is, the objective function calculation unit 111, the back shape model storage unit 112, and the parameter optimization unit 113 in the back shape estimation unit 105, will be described.

[0032] 3 to 6, the back shape model held in the back shape model holding unit 112 in this embodiment and its optimization will be illustrated below. FIG. 3 is a diagram showing an example of a back shape model held for an image acquired by the posture measurement device 100 according to the present disclosure. FIG. 4 is a diagram showing another example of a back shape model held for an image acquired by the posture measurement device 100 according to the present disclosure. FIG. 5 is a schematic diagram for explaining an example of a calculation method for the back shape model of FIG. 3 and a calculation method for an objective function. FIG. 6 is a diagram showing mathematical formulas for explaining an example of processing by the posture measurement device according to the present disclosure.

[0033] The back shape model will be described assuming that it is a model representing the shape of the subject's back in the sagittal plane, but is not limited to this as explained in embodiment 1. The back shape model can be a model that represents the shape of the subject's spine as the shape of the subject's back using multiple parameters. In this case, the back shape estimation unit 105 can also be called a spine shape estimation unit or a spine extraction unit.

[0034] The back shape model held in the back shape model holding unit 112 reconstructs itself based on the input information, for example, the size of each part, the angle of each part in the sagittal plane, and the position information of the neck and hip joints orthogonally projected onto the sagittal plane of the subject. The back shape model reconstructed in this way is updated as an output in response to the input, and is held in the back shape model holding unit 112.

[0035] In this way, the multiple parameters of the back geometric model can include, for example, at least the size of each part constituting the back geometric model, an angle representing the posture in the sagittal plane, and position information indicating the position of the neck and hip joints in the sagittal plane. Each part constituting the back geometric model should preferably include at least a part of the spine.

[0036] The angle representing the posture in the sagittal plane may be the angle of each part constituting the back geometric model, or the angle of some of the parts constituting the back geometric model. The angle of a part may be an angle with respect to a reference plane such as a horizontal or vertical plane, or may be a relative angle with respect to an adjacent part. Furthermore, the angle representing the posture in the sagittal plane may be, for example, the angle or relative angle of a connecting portion between adjacent parts, or of a line segment connecting each part with a predetermined pair of parts, with respect to the reference plane. Furthermore, the position information may be information indicating the positions of the neck and hip joints orthogonally projected onto the sagittal plane of the subject.

[0037] The back shape model may be a model expressed with fewer parts than the number of vertebrae, as exemplified in Figure 3. This model is a simplified model, and can be called a simple model.

[0038] The illustrated simplified model begins with the hip joint keypoint P9 and is composed of parts corresponding to the pelvis keypoint P7, the lower half of the lumbar vertebrae keypoint P6, the upper half of the lumbar vertebrae keypoint P5, the lower half of the lower thoracic vertebrae keypoint P4, the upper half of the lower thoracic vertebrae keypoint P3, the lower half of the upper thoracic vertebrae keypoint P2, and the upper half of the upper thoracic vertebrae keypoint P1. Keypoint P1 may be, for example, the keypoint corresponding to the top of the upper thoracic vertebrae. Furthermore, the illustrated simplified model may also include a part connecting keypoint P1 at the top of the upper thoracic vertebrae to keypoint P8 at the base of the neck. As shown in Figure 3, which shows enlarged views of keypoints P6 and P7 and the parts between them, the illustrated simplified model may also define the connecting portions between adjacent parts as parts LP.

[0039] The simplified model can represent the shape LM of the back of the subject O using these parts. Alternatively, each of the key points P1 to P7 can be represented as a part of a point cloud that is moved a predetermined distance outside the body relative to each of the key points P1 to P7, and a part at the connection between these point clouds, as shown in the figure as the shape LM of the back.

[0040] Furthermore, the back shape model can also be a detailed model that is more detailed than the above-described simple model. The detailed model may be a model that is close to the shape of the actual vertebrae, or may be a simplified model that simulates only the vertebrae and spinous process parts as illustrated in FIG. 4. For convenience, FIG. 4 illustrates a reduced number of vertebrae. The detailed model illustrated in FIG. 4 is a model that represents the shape LM of the back of subject O using parts that have the same number of vertebrae as the actual number or a slightly reduced number, and each vertebra has a part similar to a spinous process.

[0041] Specifically, as shown enlarged in FIG. 4, each vertebra SP can be represented, for example, as follows: That is, each vertebra SP can be represented by key points Pd1, Pd2, and Pd3 representing the target vertebra SP, part LPd1 representing the connection between key points Pd2 and Pd3, and part LPd2 connecting the midpoint of part LPd1 to key point Pd1. Parts Pd1 and LPd2 represent spinous processes. The detailed model shown in FIG. 4 is a model that includes, in addition to these, a part corresponding to hip joint key point P9. For example, this detailed model can be configured starting from hip joint key point P9 and including parts corresponding to the sacrum portion of the pelvis, lumbar vertebrae 1-5, thoracic vertebrae 1-12, and cervical vertebra 7, although the vertebrae included in the configuration are not limited to this example.

[0042] However, since it is extremely difficult to accurately reproduce the shape of human vertebrae without using a radiographic technique such as CT (Computed Tomography) or MRI (Magnetic Resonance Imaging), it can be said that highly accurate modeling is difficult even when using a detailed model such as the one shown in Figure 4. Therefore, considering the amount of calculation, it can be said that a simple model such as the one shown in Figure 3 is better than a detailed model such as the one shown in Figure 4.

[0043] We will explain the specific calculation method for the back shape model when using the simplified model shown in Figure 3 as the back shape model. Consider the case where there are N parts in the simplified model. The parts are numbered part1,...,part i ,···,part N Also, part i The hip joint side of both ends is called the base (node i,base ), tip (node i,top ) In this case, part i Tip of the node i,top The position can be calculated using, for example, equation (1) in FIG.

[0044] The bold elements in equation (1) represent two-dimensional vectors (x coordinate, y coordinate). In the following explanation, for convenience, elements that cannot be shown in bold but are the same as the elements shown in bold in equation (1) will also represent two-dimensional vectors.

[0045] keypoint hip indicates the position of the hip joint. Also, since adjacent parts are connected, i,base =node i-1,top size i part i The size of the part, specifically the distance between the base and tip. 1,0 ,···,size i,0 ,···,size N,0 When the ratio r1,...,r to the size of the reference part is given, i ,···,r N Using size i =r i size i,0 It can also be defined as θ i,i+1 part i and part i+1 Here, for example, part i The same direction as θ is defined as 0 and clockwise is defined as positive. 1,2 can be defined as shown in Figure 5, and other angles can be defined in the same way.0,1 Although there is no reference part0, it can be defined, for example, using the angle with respect to the vector from the hip joint to the cervical vertebrae as the reference.

[0046] The node calculated by equation (1) i,top The edge is approximated using the position of P. Here, the edge refers to the shape of the back that appears when the same posture is taken without wearing clothes. Therefore, the candidate point group input to the objective function calculation unit 111, which will be described later, is called the edge candidate point group. Hereinafter, the approximated edge point group is called P model Let's say.

[0047] The process of approximating edges is i,top} i=1,···,N This can be performed by fitting an n-th order power function or a spline function to the point sequence corresponding to the spinous processes. When a detailed model is used, it is advisable to fit an n-th order power function or a spline function to the point sequence corresponding to the spinous processes.

[0048] In addition, the part that connects the top of the upper half of the upper thoracic vertebrae to the base of the neck, i,top If it is clear that the position information is not on the body surface, the position information may not be used for edge approximation. Also, before calculating the back shape model, the image size may be normalized. Also, the node i,top However, other parts or key point positions may be used as a reference.

[0049] A specific calculation method in the objective function calculation unit 111 will be described. The objective function calculation unit 111 calculates the back shape model and the edge candidate point group P candidate Input the edge candidate point group P candidate The edge candidate points P may be extracted from an image by a method described later. candidate is a set of points that are candidates for edges, that is, candidates for edge points. In other words, the edge candidate point set P candidate is a point cloud that indicates edges as they appear, that is, edges as they appear in the captured image, and includes sagging of clothing and the like.

[0050] Then, the objective function calculation unit 111 calculates P candidate and {node i,top} i=1,···,N From, or as a second method, P candidate and P model Calculate the value of a given objective function from P model is the edge point group approximated by the back shape model as described above.

[0051] In the first method, the objective function calculation unit 111 calculates the {node i,top} i=1,···,N For each point in P candidate The objective function calculation unit 111 may calculate, as the value of the predetermined objective function, the sum of the distances to the points that are closest to each part indicated by the back shape model among the group of edge candidate points. That is, the objective function calculation unit 111 may calculate, as the value of the predetermined objective function, the distances to the points that are closest to each part indicated by the back shape model among the group of edge candidate points, and the sum of the calculated distances.

[0052] In the second method, the objective function calculation unit 111 calculates P model For each point in P candidate The objective function calculation unit 111 may calculate, as the value of the predetermined objective function, the sum of the distances between each point of the approximation candidate point group obtained by approximating the edge candidate point group based on each part indicated by the back shape model and the closest point in the approximation candidate point group. In other words, the objective function calculation unit 111 may calculate, as the value of the predetermined objective function, the distance between each point of the approximation candidate point group obtained by approximating the edge candidate point group based on each part indicated by the back shape model and the closest point in the approximation candidate point group.

[0053] In either the first or second method described above, different weights may be assigned to each node or section in the calculation of the value. That is, for example, in the first method, the objective function calculation unit 111 may calculate, as the value of the predetermined objective function, a weighted sum of distances between each part indicated by the back shape model and the closest point in the group of edge candidate points. For example, in the second method, the objective function calculation unit 111 may calculate, as the value of the predetermined objective function, a weighted sum of distances between each point in the group of approximation candidate points obtained by approximating the group of candidate points based on each part indicated by the back shape model and the closest point in the group of candidate points.

[0054] To explain the weight consideration in the first method in detail, equation (2) in Figure 6 is the weights w1, . . . , w i ,···,w N (0≦w i ≦1), it is shown in equation (4) in Figure 6. For each node or section, P candidate The smaller the weight, the closer the back shape model is to P candidate Allows distance from P candidate By reducing the weight when there is a high possibility that the deviation from the actual back shape is large, the back shape model is adjusted to be less likely to be optimized in the wrong direction. Although the explanation is omitted here, the weight consideration in the second method can be processed in a similar way, and w is added to each element to be summed in equation (3). i The formula should be something like adding the product of

[0055] The parameter optimization unit 113 receives parameter constraints and a back shape model as input together with, for example, position information of the subject's neck in the sagittal plane, and modifies the parameters of the back shape model so as to minimize the value of the objective function.

[0056] A specific calculation method in the parameter optimization unit 113 will be described below. Information on the position of the subject's neck in the sagittal plane, which is one of the inputs, can be provided by the user as an operation input from the input unit 104, or estimated from an image acquired by the image acquisition unit 101.

[0057] Although the position information is referred to as "neck" position information for convenience, it is also possible to use position information of a predetermined vertebra among the seven vertebrae that make up the cervical spine. For example, position information of the seventh cervical vertebra, the vertebrae nodosum, can be used as the position information. Alternatively, position information indicating the midpoint between the seventh cervical vertebra and the first thoracic vertebra may be used as the neck position information. Furthermore, information indicating the midpoint between the seventh cervical vertebra and the manubrium, which is a position estimated as a key point of the "neck" in a general skeleton extraction model, can also be used as the neck position information. In this case, since there will be a deviation from the position of the vertebrae nodosum, accurate parameter estimation can be achieved by adding a part that connects the "neck" key point with the vertebrae nodosum or the upper end of the upper half of the upper thoracic vertebrae to the back shape model, and then calculating the objective function and optimizing the parameters.

[0058] The parameter constraints, which are one of the inputs, are determined in advance. As examples of the constraints, at least one of the following first to third conditions can be used.

[0059] The first condition is that the error between the neck position and the neck position calculated from the back shape model falls within a predetermined error range. In other words, the constraint conditions for the multiple parameters can include a condition that the position information as part of the multiple parameters matches the candidate position information indicating the positions of the neck and hip joints in the sagittal plane as the edge candidate point group within a predetermined error range. The predetermined error range here can be determined, for example, by the lower and upper limits of the distance between the two positions. Because the distance does not take negative values, it is desirable that the lower limit be 0 and the upper limit be as small as possible.

[0060] The second condition is that the size of each part of the back shape model falls within a predetermined range. The predetermined range here can be determined, for example, by a lower limit and an upper limit of the size for each part. The second condition may also be that the ratio to the initial value or the ratio between parts falls within a predetermined range, rather than the absolute value of the size.

[0061] The third condition is that the angles of the parts must fall within a predetermined range. This predetermined range can be determined, for example, by the lower and upper limits of the relative angles between the parts. The angles of the parts correspond to the range of motion between the vertebrae in the human spine. Therefore, this predetermined range may be set as the lower and upper limits of the range of motion of flexion and extension in the sagittal plane known in various documents, or may be set as individual values ​​that can actually be taken for several postures, such as forward bending postures and backward bending postures.

[0062] The parameter optimization unit 113 inputs such parameter constraints and the back shape model together with, for example, neck position information in the sagittal plane of the subject, and modifies the parameters of the back shape model to minimize the value of the objective function. For the modification, for example, an algorithm for solving a constrained optimization problem, such as sequential quadratic programming, can be used. Since the objective function cannot be analytically differentiated, it is preferable to use numerical differentiation to find the gradient. Then, the parameter optimization unit 113 outputs the parameters of the back shape model that minimize the objective function as a result of the modification.

[0063] As described above, the back shape estimation unit 105 inputs the edge candidate point group and position information indicating the positions of the neck and hip joints in the sagittal plane, and outputs an optimized back shape model as a result of estimating the edges that represent the shape of the back that would appear if the user were in the same posture without wearing clothes. The input position information can also be referred to as joint position information. The joint position information can include information indicating the positions of joints other than the neck and hip joints.

[0064] The above has mainly described the main components of this embodiment, namely, the objective function calculation unit 111, the shape model storage unit 112, and the parameter optimization unit 113. In this embodiment, some of the other components shown in Fig. 2 are not essential components, and various variations in the configuration of the posture measurement device 100 will be described below.

[0065] For example, posture measurement device 100 can be configured such that, in addition to the above-mentioned main components, back shape estimation unit 105 includes at least one of candidate group extraction unit 114, endpoint determination unit 115, and edge addition unit 116. A configuration example in which back shape estimation unit 105 includes candidate group extraction unit 114 will be described later as Variation 1. A configuration example in which back shape estimation unit 105 includes endpoint determination unit 115 will be described later as Variation 2. A configuration example in which back shape estimation unit 105 includes edge addition unit 116 will be described later as Variation 3.

[0066] Furthermore, the posture measurement device 100 can also be configured to include at least one of a skeleton extraction unit 103, an input unit 104, a feature calculation unit 106, and a display unit 108, in addition to the above-mentioned main components. A configuration example in which the posture measurement device 100 includes the skeleton extraction unit 103 will be described later as Variation 4. A configuration example in which the posture measurement device 100 includes the feature calculation unit 106 will be described later as Variation 5. Furthermore, if the posture measurement device 100 includes the state estimation unit 107, it will also include the feature calculation unit 106. A configuration example in which the posture measurement device 100 includes the feature calculation unit 106 and the state estimation unit 107 will be described later as Variation 6.

[0067] A configuration example in which the posture measurement device 100 includes an input unit 104 will be described later as Variation 7. A configuration example in which the posture measurement device 100 includes a display unit 108 will be described later as Variation 8. Furthermore, when the posture measurement device 100 includes the image acquisition unit 101, it also includes a candidate point group extraction unit 114 or a skeleton extraction unit 103. The posture measurement device 100 includes the skeleton extraction unit 103, the candidate point group extraction unit 114, and the image acquisition unit 101, and is therefore capable of automatically evaluating posture from captured images, that is, functioning as a posture evaluation device. A configuration example in which the posture measurement device 100 includes the image acquisition unit 101 will be described later as Variation 9. Furthermore, when the posture measurement device 100 includes the imaging angle determination unit 102, it also includes the image acquisition unit 101. A configuration example in which the posture measurement device 100 includes the image acquisition unit 101 and the imaging angle determination unit 102 will be described later as Variation 10.

[0068] <Variation 1> 7 and 8, a configuration example in which the back shape estimation unit 105 includes a candidate group extraction unit 114 will be described as Variation 1. Fig. 7 is a schematic diagram for explaining an example of candidate point group extraction processing in the posture measurement device according to the present disclosure. Fig. 8 is a schematic diagram for explaining another example of candidate point group extraction processing in the posture measurement device according to the present disclosure.

[0069] In variation 1, back shape estimation unit 105 receives the images acquired by image acquisition unit 101 and joint position information, and outputs an optimized back shape model.

[0070] An example of such a configuration that is particularly effective in cases where the subject is standing upright or bending forward will be described with reference to FIG. 7. First, the candidate point cloud extraction unit 114 inputs an image of the subject captured from the side, acquired by the image acquisition unit 101, position information of the neck in the image, and position information of the hip joints in the image. Note that capturing from the side means capturing in the sagittal plane. Then, based on the position information of the neck and hip joints in the image, the candidate point cloud extraction unit 114 specifies a rectangular area that includes at least the back of the subject in the image, such as the bounding box BB shown in FIG. 7. Note that the shape of the specified area is not limited to a rectangle. Next, the candidate point cloud extraction unit 114 performs edge extraction processing on the image data acquired by the image acquisition unit 101, and extracts a point cloud that is a candidate for an edge point cloud.

[0071] More specifically, the position and size of the bounding box BB are determined based on the neck position indicated by key point P8 in the image, the hip joint position indicated by key point P9, and the distance l between them. For example, for the side of the bounding box BB in the direction of distance l, the length l of the bounding box BB from the neck to the opposite side of the hip joint is determined. m,0 , the length of the bounding box BB from the hip joint to the opposite side of the neck is l m,1 can be calculated using the following two formulas. That is, for the edge in the direction of distance l, m,0 = l × p0, l m,1 Calculate each length using the formula: =l×p1, and then calculate the length using the formula: m,0 +l+l m,1The length of the edge can be calculated by the following equation. Also, for the edge perpendicular to the direction of distance l of the bounding box BB, the length including the edge point group LMa, which is the shape of the back, on a line parallel to the line connecting the neck and hip joints, or the length obtained by adding a predetermined distance to that length, can be obtained by calculation. Note that p0 and p1 can be manually determined parameters.

[0072] Furthermore, the edge extraction process may be applied only to the inside of the bounding box BB, or may be applied to the entire image and only the edges inside the bounding box BB may be used. The edge extraction may use the edges of the person region acquired using a machine learning model for person region segmentation that has been trained in advance by machine learning.

[0073] The knee joint position can also be used as joint position information, and in that case, instead of using the distance between the hip joint and the knee as l, a bounding box including the knee joint position can be defined and its size determined. Compared to the distance l between the neck and hip joint, the distance between the hip joint and the knee has the advantage that it changes less depending on the posture.

[0074] In this way, the candidate point cloud extraction unit 114 may input a first image obtained by capturing a sagittal plane of the subject in an upright or bent-over position, and determine a rectangular area including the subject's back based on points indicating the positions of the subject's neck and hip joints specified in the first image. The designation in the first image may be performed automatically, or may be specified by the user via the input unit 104. In this case, the candidate point cloud extraction unit 114 extracts a cloud of edge candidate points by extracting a cloud of points indicating the positions of other parts in the rectangular area.

[0075] Another example of edge extraction processing, which is particularly effective when the person is standing upright or bending backward, will be explained using Figure 8. When standing upright or bending backward, the edges of the back of the body obtained from the image often differ significantly from the actual shape of the back, so the edges of the front of the body in addition to the back, or the center line of the edges of the front and back, translated to the position of the back of the body, are used.

[0076] First, in this example as well, the candidate point group extraction unit 114 inputs an image of the subject photographed from the side, acquired by the image acquisition unit 101, position information of the neck in the image, and position information of the hip joints in the image. Then, as in the case of the bending-forward posture, the candidate point group extraction unit 114 sets a rectangular region such as the bounding box BB illustrated in Fig. 7 as a region of interest based on the position information of the neck and hip joints in the image. Note that, although not shown in Fig. 8, the rectangular region in this example is for a bending-forward posture, and therefore its shape will be significantly different from the bounding box BB for the bending-forward state illustrated in Fig. 7.

[0077] Furthermore, the candidate point cloud extraction unit 114 performs edge extraction processing on the image data acquired by the image acquisition unit 101 to extract a front body edge Fa and a back body edge Fb as shown in FIG. 8 . Next, the candidate point cloud extraction unit 114 translates the front body edge Fa to align it with a predetermined position of the back body edge Fb. Alternatively, the candidate point cloud extraction unit 114 may perform edge extraction processing on the image data acquired by the image acquisition unit 101 to obtain the center line of the edges as follows. That is, the candidate point cloud extraction unit 114 may calculate the average x coordinate for each point in the edge point cloud with the same y coordinate for the front body edge Fa and the back body edge Ba shown in FIG. 8 to obtain the midpoint, and obtain this point cloud as the center line Ca of the front and back edges. Note that the center line Ca may be a center line that is approximated to a smooth curve based on the center line obtained by the midpoint calculation in this manner. Next, the candidate point cloud extraction unit 114 translates the center line Ca to align it with a predetermined position of the back body edge Fb. An example of this translation is shown by a thick arrow in Figure 8. In either method, the candidate point group extraction unit 114 can extract a group of edge candidate points by utilizing the translation of the edges that can be extracted.

[0078] In this way, the candidate point cloud extraction unit 114 may input a first image captured in the sagittal plane of the subject in an upright or bent-back position, and detect a back edge line indicating the edge of the subject's back surface and a front edge line indicating the edge of the subject's front surface in the first image.The candidate point cloud extraction unit 114 may then translate the center line between the back edge line and the front edge line, or translate the front edge line toward the back edge line so that it coincides with a part of the back edge line.The candidate point cloud extraction unit 114 can extract a cloud of edge candidate points by such translation.

[0079] <Variation 2> 9, a configuration example in which the back shape estimation unit 105 includes an end point determination unit 115 will be described as Variation 2. Fig. 9 is a schematic diagram for explaining an example of an end point determination process in the posture measurement device according to the present disclosure.

[0080] In variation 2, the back shape estimation unit 105 inputs the edge candidate point group and joint position information, and outputs an optimized back shape model with determined endpoints. Specifically, the endpoint determination unit 115 first inputs the parameters of the back shape model optimized by the parameter optimization unit 113 and the edge candidate point group. For example, the endpoint determination unit 115 inputs the edge candidate point group LMG shown in FIG. 9 , key points P1 to P9 of each part determined by the optimized back shape model, and the edge candidate point group LMG. The endpoint determination unit 115 regards the input edge candidate point group LMG as a curve and determines endpoints based on the positions of each part determined using the back shape model, such that only a specified range of the edge candidate point group LMG, such as key points P1 to P7, is extracted as an edge. In this example, the points included in the edge candidate point group LMG that are closest to key points P1 and P7 are determined as endpoints. If the edge candidate point group LMG includes points with the same coordinates as the key points P1 and P7, the endpoints coincide with the key points P1 and P7, respectively. Then, the endpoint determination unit 115 extracts the edges by cutting out the edge point group between the two endpoints from the optimized back shape model, that is, by performing trimming, as exemplified by the edge LMGL in Fig. 9.

[0081] This edge extraction process only needs to extract the edge between key points P1 and P7. Therefore, the edge between them may be an edge that completely matches the optimized back shape model, or may be an edge extracted based on the optimized back shape model and the edge candidate point group LMG.

[0082] Of course, the specified range is not limited to this. For example, if the area between the first thoracic vertebra and the fifth lumbar vertebra is to be an edge, the endpoint determination unit 115 determines the position of each part of the spine using the optimized back shape model, and then obtains the positions corresponding to the first thoracic vertebra and the fifth lumbar vertebra. The endpoint determination unit 115 then obtains the points closest to each of the obtained positions from the group of edge candidate points and determines them as endpoints, thereby extracting a specified section whose start and end points are the end points of the section. Alternatively, the intermediate positions between adjacent vertebrae, for example, between the seventh lumbar vertebra and the first thoracic vertebra, or between the fifth lumbar vertebra and the sacrum, can be specified as the positions of the endpoints. Furthermore, by specifying a portion of the spine, such as between the first thoracic vertebra and the twelfth thoracic vertebra, only a portion of the spine can be extracted as an edge.

[0083] In this way, the optimized parameters and the group of edge candidate points can be input to the endpoint determination unit 115. Then, the endpoint determination unit 115 may determine both end points of the curve connecting the points of the group of edge candidate points as both end points of the group of edge candidate points based on the positions of each part constituting the corrected back shape model, so as to extract only a specified range of the curve connecting the points of the group of edge candidate points.

[0084] <Variation 3> 10, a configuration example in which the back shape estimation unit 105 includes an edge addition unit 116 will be described as Variation 3. Note that the edge addition unit 116 can also be simply referred to as an addition unit. Fig. 10 is a schematic diagram for explaining an example of edge addition processing in the posture measurement device according to the present disclosure.

[0085] In variation 3, the back shape estimation unit 105 receives as input a group of edge candidate points and joint position information, and outputs an optimized, weighted-averaged back shape model. Specifically, the edge adder 116 first receives as input the parameters of the optimized back shape model, the group of edge candidate points or the group of edge points after trimming by the end point determiner 115, and the weighting coefficients corresponding thereto. Then, for each predetermined section, the edge adder 116 assigns weights to the group of edge points calculated from the back shape model and the group of edge points input or acquired from an image, calculates a weighted average for each point, and outputs the averaged edge point group.

[0086] For example, in the input edge candidate point group LM shown in FIG. 10, the edges cannot be accurately captured in the region A1, which is located closer to the head than the left position in the range indicated by the double-headed arrow, but the edges can be captured relatively accurately in the region indicated by the double-headed arrow. In such cases, the influence of estimation errors of parameters potentially present in the back shape model can be reduced by taking a weighted average of the edges calculated using the back shape model and the candidate point group. In other words, if the accuracy of the estimation using the back shape model is lower than the accuracy of the edge candidate point group, it is advisable to determine the edge point group by placing emphasis on the weight of the edge candidate point group. For reference, region A1 is also shown in FIGS. 3 to 5 and 9.

[0087] In this way, the edge addition unit 116 inputs the multiple parameters corrected by the parameter optimization unit 113 and the edge candidate point group. Then, the edge addition unit 116 calculates a weighted average by applying a predetermined weight to each point in the point group indicating the position of each part constituting the back shape model indicated by the multiple corrected parameters and to the candidate points in the edge candidate point group corresponding to each point. In this way, the edge addition unit 116 calculates a point group representing the shape of the subject's back.

[0088] <Variation 4> 11 and 12, a configuration example in which the posture measurement device 100 includes a skeleton extraction unit 103 will be described as Variation 4. Fig. 11 is a schematic diagram for explaining an example of skeleton extraction processing in the posture measurement device according to the present disclosure. Fig. 12 is a schematic diagram showing an example of skeleton key points extracted by the skeleton extraction processing in the posture measurement device according to the present disclosure.

[0089] In variation 4, the back shape estimation unit 105 separately receives a group of edge candidate points input by the user from the input unit 104 and key point position information indicating the positions of skeleton key points extracted by the skeleton extraction unit 103, and outputs an optimized back shape model. To this end, first, the skeleton extraction unit 103 extracts key point position information from the image acquired by the image acquisition unit 101.

[0090] The skeleton extraction unit 103 extracts keypoint information indicating at least the positions of the neck and hip joints, such as keypoints P8 and P9 in FIG. 11, by estimating it from the image. For example, the skeleton extraction unit 103 receives an image of a subject photographed from the side from the image acquisition unit 101 and extracts keypoint information from the image using a trained skeleton extraction model that has been machine-learned to output keypoint information indicating a predetermined group of skeletal keypoints. Here, too, skeletal keypoints indicating the position of the neck can be extracted as skeletal keypoints indicating the positions of predetermined vertebrae. For example, the position of the neck can be estimated as the position of the seventh cervical vertebra, which is the vertebral column. Alternatively, the position of the neck can be estimated as the midpoint between the seventh cervical vertebra and the manubrium, which is estimated as the keypoint for the "neck" in a general skeleton extraction model.

[0091] The key point information in the skeleton extraction unit 103 can be extracted from the image as information indicating the body's skeleton key points using an existing skeleton extraction model represented by a plurality of key points P and connecting portions L between adjacent key points P, as shown in Fig. 12. However, the method for extracting the key point information in the skeleton extraction unit 103 is not limited to this and may employ, for example, the method described in Patent Document 1.

[0092] In this way, the skeleton extraction unit 103 may input a second image of the subject captured from the side, and extract keypoint position information from the second image using a trained model that has been machine-learned to output keypoint position information indicating the positions of skeleton keypoints.The back shape estimation unit 105 may then determine multiple parameters of the back shape model based on the extracted keypoint position information, and the back shape model may be updated in accordance with these determinations.This back shape model is then optimized.

[0093] <Variation 5> In Variation 5, the posture measurement device 100 includes a feature calculation unit 106, which extracts feature amounts related to the state of each part of the subject from the back shape estimated by the back shape estimation unit 105. The feature calculation unit 106 extracts feature amounts, and therefore can be referred to as a feature extraction unit. Note that in Variation 5, the configuration of the back shape estimation unit 105 does not matter, and any of the configurations of the variations may be adopted. Note that, by including the feature calculation unit 106, the posture measurement device 100 can output feature amounts indicating posture, and therefore can be said to function as a posture evaluation device.

[0094] The feature calculation unit 106 receives an edge point cloud, or an edge point cloud and joint position information. The input joint position information may include position information indicating the positions of key points other than the neck and hip joints, such as key points of the shoulders, knees, and ankles. Based on the input information, the feature calculation unit 106 extracts feature values ​​that numerically represent the state of each body part, and outputs the extracted feature values. The output destination may be the display unit 108 or the state estimation unit 107.

[0095] The extracted feature may be, for example, the curvature of the edge point cloud, i.e., a quantity representing the shape of the back that would appear if the subject were in the same posture without clothing. Specifically, the curvature is calculated for all points included in the edge point cloud by fitting an n-th power function or a spline function to the edge point cloud. Because the position of each part of the spine can be identified using the back shape model, the feature calculation unit 106 can specify a part and calculate a representative value of the curvature for each section, such as the average curvature between the first to sixth thoracic vertebrae. Furthermore, the detailed model illustrated in FIG. 4 can include all vertebrae from the vertebrae ridge to the sacrum, and can calculate a representative value of the curvature between each vertebra, such as the average or median curvature between the seventh cervical vertebra and the first thoracic vertebra.

[0096] Furthermore, as the feature to be extracted, for example, the angle of each part or the relative angle between each part, which is one of the parameters of the back shape model, can be used as is. As for the method of extracting other feature, various methods can be used, such as the method described in Patent Document 1.

[0097] In this way, the feature calculation unit 106 may extract features relating to the condition of each part of the subject's back indicated by the shape of the subject's back, based on the point cloud indicating the position of each part constituting the back shape model indicated by the multiple parameters corrected by the parameter optimization unit 113.

[0098] <Variation 6> In Variation 6, the posture measurement device 100 includes a feature calculation unit 106 and a state estimation unit 107, extracts feature amounts relating to the state of each part of the subject from the back shape estimated by the back shape estimation unit 105, and estimates the state based on the extracted feature amounts. Note that in Variation 6, the configuration of the back shape estimation unit 105 does not matter, and any of the configurations of the variations may be adopted. Note that by including the feature calculation unit 106 and the state estimation unit 107, the posture measurement device 100 functions as a posture evaluation device.

[0099] The state estimation unit 107 estimates the state of each body part of the subject based on the feature amounts extracted by the feature amount calculation unit 106, and outputs the estimated state. The output destination may be the display unit 108. For example, the state estimation unit 107 receives the feature amounts extracted by the feature amount calculation unit 106 as described in Variation 5, and evaluates the state of each body part using a list of reference values ​​for the feature amounts and a trained machine learning model. This list can be stored in a storage unit (not shown). The machine learning model can also be stored in a storage unit (not shown), and this is also true for the machine learning models described in the other variations.

[0100] For example, when evaluating the state of the upper thoracic vertebrae, the state estimation unit 107 compares the average curvature between thoracic vertebrae 1-6 with the corresponding reference value in the list. A list of reference values ​​or a machine learning model may be prepared for each posture, such as a bending forward posture, a bending backward posture, and an upright posture. In this case, the posture may be determined by the user inputting the posture or by using another machine learning model that determines the posture from features. In addition, various methods may be used for state estimation in the state estimation unit 107, such as the method described in Patent Document 1.

[0101] As a result, the state estimation unit 107 can output any one of insufficient flexion, moderate flexion, and excessive flexion for each part in a bent-forward posture. Furthermore, the state estimation unit 107 can output any one of insufficient extension, moderate extension, and excessive extension for each part in a bent-back posture. Furthermore, the state estimation unit 107 can output any one of insufficient lordosis, moderate lordosis, and excessive lordosis for each part in an upright position, or any one of insufficient kyphosis, moderate kyphosis, and excessive kyphosis for each part. Furthermore, since the thoracic vertebrae are kyphotic and the lumbar vertebrae are lordotic in an upright position, the state estimation unit 107 can output a value evaluating whether the degree of each is appropriate. The state expression is merely an example, and a three-level output or a different expression may be used. Alternatively, a two-level evaluation of appropriate or inappropriate, or a value obtained by scoring the state from insufficient to excessive, may be output.

[0102] <Variation 7> In variation 7, the posture measurement device 100 includes an input unit 104 , and outputs information input by the input unit 104 to a back shape estimation unit 105 .

[0103] The input unit 104 can accept position information of skeleton key points as input from the user. In particular, in a configuration in which the posture measurement device 100 does not include the skeleton extraction unit 103, the position information of skeleton key points can be accepted as input from the user and passed to the back shape estimation unit 105. In this example, the configuration of the back shape estimation unit 105 is not important, and any variation of the configuration may be adopted.

[0104] The position information of the skeletal key points can be accepted as coordinates in a two-dimensional coordinate space, for example, with a point in the sagittal plane of the subject as the origin and with a vertical axis (positive downward) and a horizontal axis (positive direction from the front to the back of the subject). Since joints exist in a three-dimensional space, the input unit 104 may be configured to input three-dimensional coordinates. In this case, it is preferable to use coordinates orthogonally projected onto the sagittal plane for subsequent processing.

[0105] Furthermore, when the input to the posture measurement device 100 includes an image acquired by the image acquisition unit 101, the input unit 104 may accept position information as follows. That is, the input unit 104 may accept, as position information of skeletal keypoints, coordinate information on a two-dimensional plane spanned by two axes, with the upper left corner of the input image as the origin and the downward direction of the image as positive and the right direction of the image as positive. Of course, the input unit 104 may accept only numerical coordinate information, or the input unit 104 and the display unit 108 may be integrated, and the position information of skeletal keypoints may be given on the image displayed on the display unit 108 using a mouse cursor, a touch panel, or the like.

[0106] Furthermore, by including or connecting the posture measurement device 100 to a measurement system capable of acquiring coordinates in three-dimensional space, such as a motion capture system, the position information of skeletal keypoints can also be input as three-dimensional coordinates. The posture measurement device 100 can also include or be connected to a system that combines a camera-equipped head-mounted display (HMD) capable of self-position estimation with hand tracking. This allows an evaluator, such as a therapist, to input the position information of skeletal keypoints to the input unit 104 while palpating the body of the person being evaluated while wearing the HMD. In this case, the HMD can also serve as the display unit 108. Furthermore, if three-dimensional coordinates can be acquired, the image acquired by the image acquisition unit 101 is not limited to a two-dimensional image, but may also contain three-dimensional point cloud information representing the subject's body surface. Furthermore, the position information of skeletal keypoints may be relative position information with respect to a three-dimensional polygon model or surface model representing the body surface, converted from the point cloud information by a conversion unit (not shown).

[0107] As exemplified by the input unit 104, the posture measurement device 100 can include a keypoint input unit that inputs keypoint position information indicating the positions of skeletal keypoints through a user operation. In this case, the back shape model holding unit 112 may determine a plurality of parameters to be held based on the input keypoint position information.

[0108] The input unit 104 can also accept a group of edge candidate points as input from the user. In particular, in a configuration in which the back shape estimation unit 105 does not include the candidate point group extraction unit 114, the back shape estimation unit 105 can accept a group of edge candidate points as input from the user and pass it to the back shape estimation unit 105. Input for each point included in the group of edge candidate points can be accepted in the same manner as for position information of skeletal key points. In this example, the back shape estimation unit 105 may be configured in any variation as long as it does not include the candidate point extraction unit 114. However, even in a configuration in which the candidate point extraction unit 114 is included, the input unit 104 can be used to modify the group of edge candidate points extracted by the candidate point extraction unit 114.

[0109] As exemplified by the input unit 104, the orientation measurement device 100 can include a candidate point group input unit that inputs an edge candidate point group through a user operation.

[0110] <Variation 8> An example configuration in which the posture measurement device 100 includes a display unit 108 will be described as Variation 8 using Fig. 13. Fig. 13 is a diagram showing an example of an image displayed on the display unit in the posture measurement device according to the present disclosure. The posture measurement device 100A shown in Fig. 13 is an example in which the posture measurement device 100 shown in Fig. 2 is configured as a tablet terminal.

[0111] In Variation 8, a posture measurement device 100 such as posture measurement device 100A includes a display unit 108, generates images showing various output results and progress, and displays the generated images on the display unit 108. For this reason, the posture measurement device 100 may include an image generation unit (not shown). Note that in Variation 8, the other configurations of the posture measurement device 100 do not matter, and any of the configurations of the variations may be adopted.

[0112] The display unit 108 can display, for example, an image in which measurement results and evaluation results are superimposed on an acquired image, such as image G1 shown in FIG. 13, or an image including information indicating posture evaluation results, such as image G2. Image G1 shows an example in which joint position information indicated by key points P1 to P9 and P11 to P16 is superimposed on a back shape model including an edge point cloud LM. In this way, the display unit 108 can display a point cloud indicating the position of each part constituting the back shape model indicated by multiple parameters corrected by the parameter optimization unit 113. Image G2 shows an example in which information indicating the state of each part is displayed as a posture evaluation result. However, examples of information displayed on the display unit 108 are not limited to these, and various information, such as feature value values ​​and changes in feature values ​​over time, can be displayed.

[0113] The change in a feature over time refers to the change in the feature calculated for each image acquired by the image acquisition unit 101 over time. While the images acquired by the image acquisition unit 101 are assumed to be still images, it is also possible to input a video or a sequence of images extracted from the video and process each frame. In this case, the feature is calculated for each frame, so the change in the feature over time can be displayed as a graph with the horizontal axis representing time or the angle of the trunk relative to the thigh and the vertical axis representing the value of the feature. For example, when the image acquisition unit 101 acquires a video, by displaying the change in curvature, which is one of the feature values, for each body part, the timing of the start and end of movement can be determined as the point at which the curvature begins to increase or decrease. Based on this, it is possible to evaluate whether a body part that starts moving quickly has high flexibility or whether a body part that stops moving first has low flexibility.

[0114] <Variation 9> In Variation 9, posture measurement device 100 includes image acquisition unit 101, and outputs the image acquired by image acquisition unit 101 to skeleton extraction unit 103 or candidate point cloud extraction unit 114. As described in Variation 8, posture measurement device 100 can also superimpose necessary information on the image acquired by image acquisition unit 101 and output the image to display unit 108. Furthermore, by including skeleton extraction unit 103 or candidate point cloud extraction unit 114 and image acquisition unit 101, posture measurement device 100 can function as a device that automatically measures posture from captured images.

[0115] <Variation 10> 14 and 15, a configuration example in which the orientation measurement device 100 includes an image acquisition unit 101 and an imaging angle determination unit 102 will be described as variation 10. Figs. 14 and 15 are schematic diagrams for explaining an example of imaging angle determination processing in the orientation measurement device according to the present disclosure.

[0116] The imaging angle determination unit 102 receives an image acquired by the image acquisition unit 101 or position information extracted from that image by the skeleton extraction unit 103 as input, and determines the angle at which the image was captured based on the input image or position information. The angle at which the image was captured can be the angle of the optical axis of an imaging device such as a camera. The determination is performed by estimating the angle of the optical axis of the imaging device relative to the sagittal plane using a trained machine learning model. The angle output as the determination result is the angle of the imaging device relative to the sagittal plane, but can also return a warning if, for example, the angle indicates that the angle is significantly different from the vertical direction.

[0117] The imaging angle determination unit 102 may output and display on the display unit 108 a line or the like indicating the imaging angle, superimposed on the image acquired by the image acquisition unit 101. As a result, when an image is captured from a direction perpendicular to the sagittal plane, skeletal key points on the left and right appear to overlap, as shown in the example of Figure 13. Note that only skeletal key points P11 to P16 are shown here.

[0118] On the other hand, when an image is captured looking down from above on the sagittal plane, the image appears as shown in FIG. 14, with a keypoint group LL consisting of skeletal keypoints P11 to P16 on the left side and a keypoint group RU consisting of skeletal keypoints on the right side. Also, when an image is captured slightly to the left of the sagittal plane and diagonally facing right, the image appears as shown in FIG. 14, with a keypoint group RL consisting of skeletal keypoints on the right side and a keypoint group LL on the left side. Thus, when an image is captured from a direction that is not perpendicular to the sagittal plane, the keypoints on the left and right appear to be shifted in a certain direction. It is advisable to train the machine learning model used for the determination to estimate the angle of the optical axis, which is the imaging angle, using as input the deviation in positional information of skeletal keypoints such as the shoulders and hip joints on both the left and right sides.

[0119] In this way, the imaging angle determination unit 102 may input the second image capturing the sagittal plane of the subject or keypoint position information indicating the positions of skeletal keypoints extracted from the second image, and estimate the angle of the optical axis from the input using a machine learning model. This machine learning model is a trained model that has been trained to input the second image or keypoint information and output the angle of the optical axis relative to the sagittal plane when the second image was captured.

[0120] Next, a posture evaluation method executed by posture measurement device 100 will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the posture evaluation method according to the present disclosure. Only a simple flow will be described here, but the various examples described above can also be applied. Therefore, the posture evaluation method described below can also be an example of a posture measurement method that does not go as far as evaluating posture by applying some of the examples described above.

[0121] First, the image acquisition unit 101 acquires an image of the side of the body (step S101), and inputs the acquired image to the skeleton extraction unit 103. Next, the skeleton extraction unit 103 extracts key point position information indicating the positions of skeleton key points from the image acquired in step S101 (step S102).

[0122] Next, the back shape model holding unit 112 sets or updates the parameters constituting each part based on the key point information extracted by the skeleton extraction unit 103, and holds the set or updated back shape model (step S103).

[0123] Next, the candidate point group extraction unit 114 extracts a group of points that are candidates for an edge point group, that is, an edge candidate point group, based on the image acquired by the image acquisition unit 101 and position information indicating the positions of the neck and hip joints in the image (step S104). The order of steps S102 to S103 and step S104 does not matter.

[0124] Next, the objective function calculation unit 111 receives the extracted edge candidate point group and the retained back shape model, calculates the value of a predetermined objective function, and outputs it to the parameter optimization unit 113 (step S105). The parameter optimization unit 113 receives the predetermined constraint conditions and the retained back shape model as input, and optimizes the parameters by modifying multiple parameters in the back shape model so as to minimize the value of the predetermined objective function (step S106).

[0125] Next, the edge adder 116 inputs the parameters of the optimized back shape model, the edge candidate point group or the trimmed edge point group previously determined by the end point determiner 115, and the corresponding weighting coefficients, and performs edge addition (step S107). In step S107, the edge adder 116 assigns weights to the edge point group calculated from the back shape model and the edge point group input or acquired from the image for each predetermined section, and calculates a weighted average for each point. The edge adder 116 outputs the edge point group thus averaged to the feature amount calculator 106.

[0126] Next, the feature amount calculation unit 106 extracts feature amounts based on the input edge point group and outputs them to the state estimation unit 107 (step S108). The state estimation unit 107 estimates the posture state of, for example, the upper thoracic vertebrae, lower thoracic vertebrae, and lumbar vertebrae based on the feature amounts input from the feature amount calculation unit 106 (step S109). Then, the state estimation unit 107 or an image generation unit (not shown) generates an image showing the estimation result (step S110) and passes it to the display unit 108. The display unit 108 displays the image showing the estimation result (step S111), and the process ends.

[0127] This completes the description of this embodiment. In this embodiment, as in the first embodiment, it is possible to simply and accurately measure the shape of the subject's back even in situations where the shape of the subject's back cannot be accurately grasped due to the presence of the subject's clothing.

[0128] For example, according to this embodiment, by using a back shape model, it is possible to avoid making estimations that are impossible given the body structure, thereby improving the accuracy of edge estimation. Furthermore, according to this embodiment, the back shape can be accurately estimated even if the body shape is hidden by clothing, and the accuracy of evaluation for each part can be improved even for, for example, the bending posture of a person wearing a loose T-shirt.

[0129] Furthermore, according to this embodiment, even when using only two-dimensional images obtained by a camera such as a smartphone, the shape of the back can be accurately estimated, and posture can also be accurately evaluated based on the estimated shape. In fact, with the spread of online training and self-training, there is a growing need for ordinary people to evaluate their own posture and alignment. This embodiment can also meet this need. In other words, even if you are not an expert such as a trainer or therapist, you can relatively easily and accurately estimate the shape of your back and evaluate its condition using two-dimensional images obtained by a camera such as a smartphone in situations such as online training or self-training. This embodiment can also be used for simple screening by trainers, therapists, etc., before treatment for a subject.

[0130] To further explain the effects of this embodiment, comparative examples are shown in Fig. 17 and Fig. 18. Fig. 17 is a diagram showing the results of estimating the shape of the back in a forward bending posture according to the comparative example. Fig. 18 is a diagram showing the results of estimating the shape of the back in a backward bending posture according to the comparative example.

[0131] The comparative example shown in FIG. 17 illustrates an example in which the back shape cannot be accurately estimated due to inaccurate estimation of the endpoints. As shown by the edge point group LMc in the upper diagram of FIG. 17, the edge point group representing the back shape should be accurately estimated, but as shown by the edge point group LMe in the lower diagram of FIG. 17, the back shape cannot be accurately estimated. It can be seen that the edge point group LMe does not extend to the point indicated by the arrow near the left end of the edge point group LMc. This occurs because the subject is wearing thick clothing such as a parka, and loose parts such as the hood of the parka hide the body shape, preventing accurate estimation. In contrast, this embodiment can appropriately determine the endpoints, making it possible to accurately estimate the back shape as shown by the edge point group LMc, as the back shape that would appear if the subject were wearing the same posture without clothing.

[0132] The comparative example shown in FIG. 18 illustrates an example in which the back shape in a bent-back posture cannot be accurately estimated due to loose clothing or wrinkles. As shown by the edge point group Bc representing the back shape in the left diagram of FIG. 18 , the edge point group representing the back shape should be accurately estimated even in a bent-back posture. However, as shown by the edge point group Be in the right diagram of FIG. 18 , the edge point group Be represents an edge point group obtained by a comparative example that employs an edge extraction method that separates the person from the background. This edge extraction method extracts the edges of the clothing rather than the edges of the back. In contrast, in this embodiment, the edge point group is estimated using the aforementioned information about the front of the body in a complementary manner. This allows for a relatively accurate estimation of the back shape as it would appear if the person were in the same posture without clothes, as shown by the edge point group Bc.

[0133] Furthermore, in both the comparative examples shown in Figures 17 and 18, the feature extraction, which is the subsequent processing, uses the edge shape for calculation. Therefore, as in the edge point group LMe in the comparative example of Figure 17, the feature cannot be extracted correctly because the edge endpoints are shifted, that is, shifted from the body part to be evaluated. Furthermore, as in the edge point group Be in the comparative example of Figure 18, the back shape is inaccurate, so the feature cannot be extracted correctly. In contrast, in this embodiment, feature extraction can be performed accurately, and further, the subsequent state estimation can also be performed accurately.

[0134] Embodiment 3 Next, a configuration example of a posture evaluation system 200 will be described with reference to FIG. 19. FIG. 19 is a diagram showing one configuration example of a posture evaluation system according to the present disclosure. As shown in FIG. 19, the posture evaluation system 200 includes a posture measurement device 100B and a subject terminal 300 capable of communicating with the posture measurement device 100B. The posture measurement device 100B includes the main components described in the posture measurement device 100. The posture measurement device 100B and the subject terminal 300 are capable of communicating with each other via a network N. Also, as shown in FIG. 19, one or more subject terminals 300 may be capable of communicating with the posture measurement device 100B. Furthermore, the subject terminal 300 is a smartphone, tablet terminal, personal computer, or the like owned by the subject.

[0135] The posture measurement device 100B acquires an image of the side of the body of the subject whose posture is to be measured and evaluated from the subject terminal 300. Therefore, the posture measurement device 100B differs from the posture measurement device 100 in Fig. 2 in that the image acquisition unit 101 does not need to include an imaging device. Furthermore, the estimation result display image generated by the posture measurement device 100B for display on the display unit 108 may be transmitted to the subject terminal 300 and displayed on a display unit (not shown) of the subject terminal 300.

[0136] The subject terminal 300 includes an imaging device (not shown) that captures an image of the side of the body of the subject for posture measurement and evaluation. The subject terminal 300 transmits the image to the posture measurement device 100B.

[0137] Other embodiments In the above-described embodiments, the present disclosure has been described as a hardware configuration, but the present disclosure is not limited to this. For example, the present disclosure can be realized by having a processor such as a CPU (Central Processing Unit) execute a computer program to perform the processing steps shown in the flowchart of FIG. 16 and other processing steps described in the embodiments.

[0138] Furthermore, the above-described program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), or other memory technology. By way of example and not limitation, the computer-readable medium or tangible storage medium includes a CD-ROM, a digital versatile disc (DVD), a Blu-ray disc, or other optical disk storage, a magnetic cassette, a magnetic tape, a magnetic disk storage, or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes an electrical, optical, acoustic, or other form of propagated signal.

[0139] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. In other words, the present disclosure is not limited to the above-described embodiments and can be modified as appropriate without departing from the spirit of the present disclosure. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0140] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0141] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a storage unit that stores a shape model that represents the shape of the subject's back as a function of a plurality of parameters; a calculation unit that calculates a value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as input; an optimization unit that modifies the plurality of parameters so as to minimize the value of the predetermined objective function, using the constraint conditions of the plurality of parameters and the shape model as inputs; A posture measurement device comprising: (Appendix 2) the shape model is a model representing a shape of the subject's back in a sagittal plane, the plurality of parameters include at least a size of each part constituting the shape model, an angle representing a posture in a sagittal plane, and position information representing the positions of the neck and hip joints in the sagittal plane; 2. The posture measurement device according to claim 1. (Appendix 3) the constraint conditions for the plurality of parameters include a condition that the position information as part of the plurality of parameters and candidate position information indicating positions of the neck and hip joints in a sagittal plane as the candidate point cloud match within a predetermined error range. 3. The posture measurement device according to claim 2. (Appendix 4) the calculation unit calculates, as the value of the predetermined objective function, a sum of distances between each part indicated by the shape model and the closest point in the group of candidate points; 4. The posture measuring device according to any one of Supplementary notes 1 to 3. (Appendix 5) the calculation unit calculates, as a value of the predetermined objective function, a weighted sum of distances between each part indicated by the shape model and the point that is closest to the candidate point group; 4. The posture measuring device according to any one of Supplementary notes 1 to 3. (Appendix 6) the calculation unit calculates, as a value of the predetermined objective function, a sum of distances between each point of an approximation candidate point group obtained by approximating the candidate point group based on each part indicated by the shape model and the closest point in the candidate point group; 4. The posture measuring device according to any one of Supplementary notes 1 to 3. (Appendix 7) the calculation unit calculates, as a value of the predetermined objective function, a weighted sum of distances between each point of an approximation candidate point group obtained by approximating the candidate point group based on each part indicated by the shape model and the closest point in the candidate point group; 4. The posture measuring device according to any one of Supplementary notes 1 to 3. (Appendix 8) a candidate point cloud extraction unit that inputs a first image obtained by capturing a side view of the subject in an upright or bent-over position, determines a region including the back of the subject based on points indicating the positions of the subject's neck and hip joints specified in the first image, and extracts a point cloud indicating the positions of other parts of the body in the region, thereby extracting the candidate point cloud; 8. The posture measuring device according to any one of Supplementary notes 1 to 7. (Appendix 9) a candidate point cloud extraction unit that inputs a first image obtained by capturing an image of the side of the subject in an upright or bent-back posture, detects a back edge line indicating the edge of the back surface of the subject in the first image and a front edge line indicating the edge of the front surface, and extracts the candidate point cloud by translating the center line between the back edge line and the front edge line or the front edge line toward the back edge line so that it coincides with a part of the back edge line; 8. The posture measuring device according to any one of Supplementary notes 1 to 7. (Appendix 10) an endpoint determination unit that receives the plurality of parameters modified by the optimization unit and the group of candidate points, and determines both endpoints of the curve as both endpoints of the group of candidate points based on positions of each part constituting the modified shape model, so as to extract only a specified range of a curve connecting the points of the group of candidate points; 10. The posture measuring device according to any one of Supplementary notes 1 to 9. (Appendix 11) an adder that receives the plurality of parameters modified by the optimization unit and the candidate point cloud, and calculates a weighted average by adding a predetermined weight to each point of the point cloud indicating the position of each part constituting the shape model indicated by the plurality of modified parameters and a candidate point of the candidate point cloud corresponding to each point, thereby obtaining a point cloud representing the shape of the back of the subject; 11. The posture measuring device according to any one of Supplementary notes 1 to 10. (Appendix 12) a skeleton extraction unit that receives a second image capturing a side view of the subject and extracts keypoint position information from the second image using a trained model that has been machine-learned to output keypoint position information indicating positions of skeleton keypoints; 12. The posture measuring device according to any one of Supplementary notes 1 to 11. (Appendix 13) a feature extraction unit that extracts feature amounts related to the state of each part of the subject's back indicated by the shape of the subject's back, based on a point cloud indicating the position of each part constituting the shape model indicated by the plurality of parameters corrected by the optimization unit; 13. A posture measuring device according to any one of appendices 1 to 12. (Appendix 14) a state estimation unit that estimates a state of each part of the subject based on the feature amount extracted by the feature amount extraction unit; 14. The posture measurement apparatus of claim 13. (Appendix 15) a key point input unit for inputting key point position information indicating positions of skeleton key points by a user operation; the storage unit determines the plurality of parameters to be stored based on the input key point position information. 15. A posture measuring device according to any one of appendices 1 to 14. (Appendix 16) a candidate point group input unit for inputting the candidate point group through a user operation; 16. A posture measuring device according to any one of appendices 1 to 15. (Appendix 17) a display unit that displays a point cloud indicating the positions of each part that constitutes the shape model indicated by the plurality of parameters corrected by the optimization unit, 17. A posture measuring device according to any one of appendices 1 to 16. (Appendix 18) and an imaging angle determination unit that receives input of a second image capturing a side view of the subject or key point position information indicating the positions of skeletal key points extracted from the second image, and estimates the angle of the optical axis from the second image or the key point position information using a trained model that has been machine-learned to output the angle of the optical axis with respect to the sagittal plane when the second image was captured. 18. A posture measuring device according to any one of appendices 1 to 17. (Appendix 19) The posture measurement device maintaining a shape model representing the shape of the subject's back as a function of a plurality of parameters; Calculating the value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as input; modifying the plurality of parameters so as to minimize the value of the predetermined objective function using the constraint conditions of the plurality of parameters and the shape model as inputs; Posture measurement method. (Appendix 20) maintaining a shape model representing the shape of the subject's back as a function of a plurality of parameters; Calculating the value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as input; modifying the plurality of parameters so as to minimize the value of the predetermined objective function using the constraint conditions of the plurality of parameters and the shape model as inputs; A program that causes a computer to execute a process. (Appendix 21) A posture measurement device and a terminal device capable of communicating with the posture measurement device, The posture measurement device a storage unit that stores a shape model that represents the shape of the subject's back as a function of a plurality of parameters; an acquisition unit that acquires, from the terminal device, a candidate point cloud that is a candidate for a point cloud indicating the shape of the subject's back, or an image that is a source of extraction of the candidate point cloud; a calculation unit that calculates a value of a predetermined objective function using the candidate point group and the shape model as input; an optimization unit that modifies the plurality of parameters so as to minimize the value of the predetermined objective function, using the constraint conditions of the plurality of parameters and the shape model as inputs; An attitude measurement system comprising:

[0142] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 18 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 19, 20, and 21 in the same dependency relationship as Supplementary Notes 2 to 18. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0143] 1. Posture measurement device 1a Holding part 1b Calculation section 1c Optimization section 100, 100A, 100B Attitude measurement device 101 Image acquisition unit 102 Shooting angle determination unit 103 Skeleton Extraction Unit 104 Input section 105 Back shape estimation unit 106 Feature calculation unit 107 State Estimation Unit 108 Display section 109 Communications Department 111 Objective function calculation unit 112 Back shape model holding unit 113 Parameter Optimization Unit 114 Candidate point cloud extraction unit 115 End point determination section 116 Edge Addition Unit 200 Posture Assessment System 300 Target Device

Claims

1. a storage unit that stores a shape model that represents the shape of the subject's back as a function of a plurality of parameters; a calculation unit that calculates a value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as input; an optimization unit that modifies the plurality of parameters so as to minimize the value of the predetermined objective function, using the constraint conditions of the plurality of parameters and the shape model as inputs; A posture measurement device comprising:

2. the shape model is a model representing a shape of the subject's back in a sagittal plane, the plurality of parameters include at least a size of each part constituting the shape model, an angle representing a posture in a sagittal plane, and position information representing the positions of the neck and hip joints in the sagittal plane; The posture measurement device according to claim 1 .

3. the constraint conditions for the plurality of parameters include a condition that the position information as part of the plurality of parameters and candidate position information indicating positions of the neck and hip joints in a sagittal plane as the candidate point cloud match within a predetermined error range. The posture measurement device according to claim 2 .

4. the calculation unit calculates, as a value of the predetermined objective function, a sum or a weighted sum of distances between each part indicated by the shape model and the point that is closest to the candidate point group; The posture measurement device according to claim 1 or 2.

5. the calculation unit calculates, as the value of the predetermined objective function, a total sum or a weighted sum of distances between each point of an approximate candidate point group obtained by approximating the candidate point group based on each part indicated by the shape model and the closest point in the candidate point group; The posture measurement device according to claim 1 or 2.

6. a candidate point cloud extraction unit that inputs a first image obtained by capturing a side view of the subject in an upright or bent-over position, determines a region including the back of the subject based on points indicating the positions of the subject's neck and hip joints specified in the first image, and extracts a point cloud indicating the positions of other parts of the body in the region, thereby extracting the candidate point cloud; The posture measurement device according to claim 1 or 2.

7. a candidate point cloud extraction unit that inputs a first image obtained by capturing a side view of the subject in an upright or backward-leaning posture, detects a back edge line indicating the edge of the back surface of the subject in the first image and a front edge line indicating the edge of the front surface, and extracts the candidate point cloud by translating a center line between the back edge line and the front edge line or the front edge line toward the back edge line so as to coincide with a part of the back edge line; The posture measurement device according to claim 1 or 2.

8. an endpoint determination unit that receives the plurality of parameters modified by the optimization unit and the group of candidate points, and determines both endpoints of the curve as both endpoints of the group of candidate points based on positions of each part constituting the modified shape model, so as to extract only a specified range of a curve connecting the points of the group of candidate points; The posture measurement device according to claim 1 or 2.

9. The posture measurement device maintaining a shape model representing the shape of the subject's back as a function of a plurality of parameters; Calculating the value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as input; modifying the plurality of parameters so as to minimize the value of the predetermined objective function using the constraint conditions of the plurality of parameters and the shape model as inputs; Posture measurement method.

10. maintaining a shape model representing the shape of the subject's back as a function of a plurality of parameters; Calculating the value of a predetermined objective function using a candidate point cloud that is a candidate point cloud representing the shape of the subject's back and the shape model as input; modifying the plurality of parameters so as to minimize the value of the predetermined objective function using the constraint conditions of the plurality of parameters and the shape model as inputs; A program that causes a computer to execute a process.

Images