Body measurement method executed by a computer

Abstract

To provide a body size measuring method executed by a computer of measuring physical features of a target person with high precision.SOLUTION: A method according to the present invention includes a step 11 of acquiring a captured image including a body region and a background region, a step 12 of generating a reference two-gradation image corresponding to the captured image, and a step 15 of generating a three-dimensional model of a target person from the reference two-gradation image by using body type template data and a CNN. The method further includes a step of estimating the skeleton of the target person from the captured image or the reference two-gradation image to correct a shape of the body region included in the captured image or the reference two-gradation image by using the estimated skeleton.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a body measurement method executed by a computer.

Background Art

[0002] With the recent development of computer technology, attempts have been made to automatically measure the body dimensions of a person from a video or image of the person. Such an automatic measurement technology from a video or image has attracted attention, particularly in the clothing field, as a method for measuring a customer's body dimensions over a network.

[0003] Currently well-known automatic measurement methods typically generate a three-dimensional model representing the body of a subject from a two-dimensional video or image of the subject using a convolutional neural network (CNN), and measure the dimensions from this three-dimensional model. As an example of such an automatic measurement technology, BodyNet disclosed in Non-Patent Document 1 is widely known. In BodyNet, after performing skeleton estimation on a subject photographed in a two-dimensional video or image, a three-dimensional model is generated by skinning the estimated skeleton.

[0004] On the other hand, as another example of automatic measurement technology, HS-Nets disclosed in Non-Patent Document 2 are also known. In HS-Nets, instead of performing skeleton estimation and skinning like BodyNet, a three-dimensional model is directly generated from a plurality of two-dimensional images of a subject photographed in a predetermined posture and orientation. In this method, the shooting state of the base two-dimensional image is specified, but a three-dimensional model can be generated with higher accuracy compared to BodyNet of Patent Document 1.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

[0006] However, the three-dimensional models obtained by the methods of Non-Patent Documents 1 and 2 are insufficient in accuracy for use in obtaining accurate measurement data for the fine features of the body, and there is a problem that it is difficult to obtain high-precision measurement data from the three-dimensional models obtained by these methods.

[0007] Also, although HS-Nets of Non-Patent Document 2 has the advantage that it can generate a three-dimensional model with relatively high accuracy compared to BodyNet of Non-Patent Document 1 if ideal captured images can be used, it is difficult for an individual to capture ideal captured images for HS-Nets. Therefore, it is difficult to obtain a three-dimensional model with high accuracy by this method, and there is a problem that high-precision measurement data cannot be obtained.

[0008] Therefore, there is a demand for an automatic measurement method that can perform measurement with higher accuracy based on the captured two-dimensional images.

[0009] In view of the above problems, an object of the present invention is to provide a body measurement method executed by a computer that measures the body characteristics of a subject with high accuracy. [Means for Solving the Problems]

[0010] A method executed by a computer for measuring the body of a subject according to a first aspect of the present invention includes: an image acquisition step of acquiring at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region; a binarization step of converting each of the captured images to generate a reference binary image corresponding to the captured image, the reference binary image including a body region represented by a first color corresponding to the body region of the captured image and a background region represented by a second color corresponding to the background region of the captured image; a definition step of defining each of the reference binary images as an image for model generation; a model output step of outputting a three-dimensional model of the subject, the model output step including: a sub-step of generating a three-dimensional model of the subject from the image for model generation using body shape template data and a convolutional neural network; a sub-step of generating a comparison binary image corresponding to each of the images for model generation based on the three-dimensional model of the subject; a sub-step of comparing each body region of the reference binary image with each body region of the comparison binary image to calculate a difference value therebetween; a sub-step of, when the difference value is greater than a predetermined threshold and the number of executions of this model output step is less than a predetermined number of times, correcting the comparison binary image, changing the image for model generation to the corrected comparison binary image, and re-executing this model output step; and a model output step; and a step of measuring one or more dimensions related to the body characteristics of the subject based on the finally generated three-dimensional model of the subject when the finally calculated difference value is less than or equal to the predetermined threshold. A method including these steps.

[0011] According to such a configuration, when outputting a three-dimensional model of a subject, the generation of the three-dimensional model is repeated while correcting the binary image that is the basis for generating the three-dimensional model until a three-dimensional model is obtained in which the error between the comparison binary image generated from the three-dimensional model and the reference binary image based on the captured image is sufficiently small. Thereby, based on the three-dimensional model in which the body of the subject is modeled with high accuracy, the body characteristics of the subject can be measured with high accuracy.

[0012] A method executed by a computer for measuring the body of a subject according to a second aspect of the present invention is as follows: An image acquisition step of acquiring at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region; A binarization step of converting each of the captured images to generate the same number of reference binary images as the captured images, including a body region represented by a first color corresponding to the body region of the captured image and a background region represented by a second color corresponding to the background region of the captured image; A model output step of generating a three-dimensional model of the subject from the reference binary image using body shape template data and a convolutional neural network; including Before the step of generating the reference binary image, estimating the skeleton of the subject from the captured image and using the estimated skeleton to correct the shape of the body region included in the captured image, or Before the step of generating the three-dimensional model of the subject, estimating the skeleton of the subject from the captured image or the reference binary image and using the estimated skeleton to correct the shape of the body region included in the reference binary image The method further includes.

[0013] According to such a configuration, when generating a reference binary image that serves as a basis for generating a three-dimensional model of a subject, based on the skeleton of the subject estimated from the captured image or its binary image, the distortion of the body region of the subject that may occur when acquiring the captured image is corrected, and a reference binary image suitable for generating the three-dimensional model is generated. Thereby, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body characteristics of the subject can be measured with high accuracy.

[0014] A method executed by a computer for measuring the body of a subject according to a third aspect of the present invention is as follows. An image acquisition step of acquiring at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region. A binarization step of converting each of the captured images to generate the same number of reference binary images as the captured images, including a body region represented by a first color corresponding to the body region of the captured image and a background region represented by a second color corresponding to the background region of the captured image. A model output step of generating a three-dimensional model of the subject from the reference binary image using body shape template data and a convolutional neural network. including The body shape template data is constituted by a body shape template generated from a group of people having one or more attributes possessed by the subject.

[0015] According to such a configuration, by using body shape template data constituted by a body shape template generated from a group of people close to the subject, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body characteristics of the subject can be measured with high accuracy.

Advantages of the Invention

[0016] According to the method executed by a computer for measuring the body of a subject of the present invention, the body characteristics of the subject can be measured with high accuracy.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0018] Hereinafter, with reference to the drawings, embodiments of a method executed by a computer for measuring the body of a subject of the present invention will be described. However, the following embodiments are merely examples. The present invention is not limited to the following embodiments at all.

[0019] The method executed by a computer for measuring the body of a subject of the present invention is an image acquisition step of acquiring at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region; a binarization step of converting each of the captured images to generate a reference binary image corresponding to the captured image, the reference binary image including a body region represented by a first color corresponding to the body region of the captured image and a background region represented by a second color corresponding to the background region of the captured image; a definition step of defining each of the reference binary images as an image for model generation; A model output step for outputting a three-dimensional model of the subject; A measurement step of measuring one or more dimensions related to the physical characteristics of the subject based on the three-dimensional model of the subject.

[0020] The first feature of the method of the present invention is that when outputting a three-dimensional model of a subject, until a three-dimensional model with a sufficiently small error between a comparison grayscale image generated from the three-dimensional model and a reference grayscale image based on a photographed image is obtained, the generation of the three-dimensional model is repeated while correcting the grayscale image that is the basis for generating the three-dimensional model as necessary. Specifically, the first feature of the method of the present invention is that The model output step for outputting the three-dimensional model of the subject includes A sub-step of generating a three-dimensional model of the subject from the model generation image using body shape template data and a convolutional neural network; A sub-step of generating a comparison grayscale image corresponding to each of the model generation images based on the three-dimensional model of the subject; A sub-step of comparing each of the body regions of the reference grayscale image with each of the body regions of the comparison grayscale image and calculating a difference value therebetween; When the difference value is greater than a predetermined threshold value and the number of executions of this model output step is less than a predetermined number of times, the comparison grayscale image is corrected, the model generation image is changed to the corrected comparison grayscale image, and this model output step is executed again. And includes The three-dimensional model of the subject used in the measurement step is the three-dimensional model finally output in the model output step.

[0021] Thereby, based on a three-dimensional model in which the body of the subject is modeled with high accuracy, the physical characteristics of the subject can be measured with high accuracy.

[0022] The second feature of the method of the present invention is to correct the distortion of the body area of the subject that may occur when acquiring a photographed image. Specifically, the second feature of the method of the present invention is before the binarization step, estimating the skeleton of the subject from the photographed image, and using the estimated skeleton to correct the shape of the body area included in the photographed image, or before the definition step, estimating the skeleton of the subject from the photographed image or the reference binary image, and using the estimated skeleton to correct the shape of the body area included in the reference binary image, including a binary image correction step.

[0023] Thereby, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body characteristics of the subject can be measured with high accuracy.

[0024] The third feature of the method of the present invention is that the body shape template data is composed of a body shape template generated from a group of people having one or more attributes possessed by the subject.

[0025] Thereby, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body characteristics of the subject can be measured with high accuracy.

[0026] The method of the present invention may separately include each of these first to third features, or may include any combination of these features. When the method of the present invention includes a plurality of combinations of these first to third features, it becomes possible to measure the body characteristics of the subject with higher accuracy.

[0027] FIG. 1 shows Method 1 which is an embodiment of a body measurement method executed by a computer of the present invention. The method of this embodiment includes the first to third features of the present invention described above. Method 1 can be executed, for example, by a computer system 3 described later.

[0028] In Method 1, first, in Step 11, at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region is acquired (image acquisition step).

[0029] Specifically, in this Step 11, for one or more body orientations and postures required for model generation performed in the subsequent model output step 14, captured images corresponding thereto are acquired. In this embodiment, two captured images including a captured image (front captured image) of a subject in a neutral posture generally called the "A pose" taken from the front and a captured image (side captured image) of the subject taken from the side are acquired.

[0030] This Step 11 can be acquired by the subject being photographed by a camera provided in or connected to the computer used for photographing.

[0031] Here, when the computer or camera used for photographing has a function of correcting the distortion of the captured image, it is desirable to acquire the captured image corrected for distortion using that function. For example, when the computer or camera used for photographing is equipped with an inclination sensor, distortion processing may be performed using the inclination data. For example, the distortion of the vertical dimension of the body region caused by the inclination of the camera when the subject as the object is photographed can be corrected based on the inclination data.

[0032] Also, when the computer or camera used for photographing is equipped with a depth sensor, it is preferable to acquire the depth data of the subject at the time of photographing. Based on the depth information obtained based on the depth data, it is possible to confirm whether there is a large error in the body information such as the height of the subject acquired in advance. Furthermore, the depth information can also be used to accurately discriminate the boundary between the two when distinguishing the body region and the background region in the subsequent binarization step 12.

[0033] Next, in step 12, each of the captured images is converted to generate a reference binary image corresponding to the captured image, which includes a body region represented by a first color corresponding to the body region of the captured image and a background region represented by a second color corresponding to the background region of the captured image (binarization step).

[0034] By the binarization conversion of the captured image performed in this step 12, in the generated binary image, the details included in the body region and the background region are omitted, and the body region and the background region are clearly distinguished. In the binary image generated in this way, the body region corresponding to the subject's body is clearly separated from the background region, so that when generating a model in the subsequent model output step 15, it becomes easier to recognize the body region corresponding to the subject's body. Note that the binary image obtained in this step 11 will be used as a reference image (reference binary image) for confirming the accuracy of the three-dimensional model generated in the subsequent model output step 15.

[0035] In this step 12, binarization conversion is performed on each of the one or more captured images obtained in the aforementioned image acquisition step 11, and a corresponding binary image is generated. In this embodiment, the front captured image and the side captured image obtained in the image acquisition step 11 are each binarized to generate a reference front binary image and a reference side binary image.

[0036] The binarization conversion of the captured image in this step 12 can be executed by any conventionally known method. As such a method, a binarization technique using deep learning technologies such as global contrast normalization (GCN), semantic segmentation, and instance segmentation (e.g., DeepMask) can be adopted. These methods may be adopted alone or in combination of multiple ones.

[0037] In the reference binary image, the first color and the second color representing the body region and the background region may be different from each other, and it is preferable that they are colors with significantly different colors and contrasts, which facilitates the distinction of the boundaries of those regions. Usually, the first color and the second color are selected from white and black, and the first color may be white and the second color may be black, or vice versa.

[0038] Next, in step 13, the skeleton of the subject is estimated from the captured image or the reference binary image, and using the estimated skeleton, the shape of the body region included in the reference binary image is corrected (binary image correction step). This step 13 corresponds to the second feature of the present invention.

[0039] Specifically, in this step 13, the skeleton of the subject in the body region included in the captured image or the reference binary image is estimated, and based on the estimated skeleton, the reference binary image is corrected so that it becomes an image suitable for model generation performed in the subsequent model output step 15. The estimation of the subject's skeleton is usually performed based on the reference binary image after binarization conversion, but if the subject's skeleton can be estimated with sufficient accuracy, it may be performed based on the captured image before binarization conversion.

[0040] The method for estimating the subject's skeleton is not particularly limited, and any conventionally known skeleton estimation method such as a two-dimensional skeleton estimation method like OpenPose or PoseNet, or a three-dimensional skeleton estimation method can be adopted.

[0041] The correction of the shape of the body region included in the reference grayscale image is performed, for example, by correcting the posture represented in the captured image of the body region or the body region of the reference grayscale image to a posture suitable for model generation performed in the subsequent model output step 15. For example, due to causes such as distortion of the subject's body, the subject may not assume a posture suitable for model generation when being captured, and in the posture of the subject represented in the body region, one shoulder may be lower than the other shoulder, or the neck may be bent. In such a case, based on the estimated skeleton, it is conceivable to correct the body region of the reference grayscale image so as to be a skeleton representing a posture suitable for model generation.

[0042] Note that in this embodiment, the estimation of the skeleton and subsequent distortion correction are performed on the reference grayscale image in the grayscale image correction step 13. However, instead of or in addition to this grayscale image correction step 13, the estimation of the skeleton and subsequent distortion correction may be performed on the captured image before binarization. Specifically, the method 1 of this embodiment may include, instead of or in addition to the grayscale image correction step 13, a step 13A of estimating the skeleton of the subject from the captured image and correcting the shape of the body region included in the captured image using the estimated skeleton (captured image correction step). This captured image correction step 13A is performed before the binarization step 12, unlike the grayscale image correction step 13 performed after the binarization step 12.

[0043] Next, in step 14, each of the reference grayscale images is defined as an image for model generation (definition step).

[0044] In this step 14, each of the reference grayscale images corrected in the grayscale image correction step 13 is defined as the first model generation image as the image used when first generating the three-dimensional model of the subject in the next model output step 15. In this embodiment, two images are generated: a front image for model generation and a side image for model generation.

[0045] Next, in step 15, a three-dimensional model of the subject is output (model output step).

[0046] As described above, in this embodiment, in step 15, the generation of the three-dimensional model is repeated while correcting the grayscale image that is the basis for generating the three-dimensional model until a three-dimensional model is obtained in which the error between the comparison grayscale image generated from the three-dimensional model and the reference grayscale image based on the photographed image is sufficiently small, and the first feature of the present invention is provided. This step 15 includes sub-steps 21 to 24 shown in FIG. 2, and by repeating these as necessary, a three-dimensional model with high accuracy can be generated. These sub-steps 21 to 24 will be described in detail below.

[0047] In the model output step 15 of this embodiment, first, in sub-step 21, a three-dimensional model of the subject is generated from a model generation image, which is a two-dimensional image, using body shape template data and a convolutional neural network (model generation sub-step).

[0048] In this sub-step 21, as a method for generating a three-dimensional model using body shape template data and a convolutional neural network, for example, HS-Nets described in Patent Document 2 above can be used. In three-dimensional modeling using HS-Nets, based on the body shape template data, a three-dimensional model corresponding to the orientation and posture of the subject included in those images can be directly generated from a model generation front image and a model generation side image each including a body region corresponding to the subject photographed from the front and the side.

[0049] The body shape template data is learned data generated from a predetermined group of people. Specifically, the body shape template data can be a template mesh created using machine learning from three-dimensional scan data of a plurality of bodies obtained from a predetermined group of people. At this time, before the three-dimensional scan data is used for machine learning, it is desirable that the raw three-dimensional scan data be preprocessed, and further, a plurality of feature points (landmarks) be manually specified in the preprocessed three-dimensional scan data. The preprocessing that can be performed here includes, for example, removal of noise components in the three-dimensional scan data, manual correction of missing parts, and smoothing processing of the surface of the three-dimensional scan data.

[0050] As the third feature of the present invention described above, this embodiment has a feature that the body shape template data is composed of a body shape template generated from a group of people having one or more attributes possessed by the subject. Here, the one or more attributes possessed by the subject are the race, nationality, age, gender, body composition value, body shape such as BMI, etc. of the subject, and for example, the race or nationality of the subject. General HS-Nets use a body shape template learned from data of a wide range of people with less bias in order to model the body shapes of a wide range of people. However, as a trade-off for being able to generate a model targeting a wide range of human figures, there has been a problem that sufficient accuracy cannot be obtained. In this embodiment, by using, as the body shape template, one generated from a group of people whose body shape tendency is close to that of the subject, such as having the same race or nationality as the subject, it becomes possible to generate a highly accurate model that can be used for such applications.

[0051] The three-dimensional model generated in this sub-step 21 is composed of, for example, about 10,000 polygons. However, the number of polygons constituting the three-dimensional model is not particularly limited and can be appropriately determined according to the required computing resources and the smoothness of the curves in the required three-dimensional model.

[0052] Next, in sub-step 22, based on the three-dimensional model of the subject, a binary comparison image corresponding to each of the images for model generation is generated (comparison image generation sub-step).

[0053] Specifically, in this sub-step 22, for each of the posture and orientation of the subject's body represented in the body region of the image for model generation, a comparison binary image is generated respectively. These comparison binary images are images for comparison with the reference binary image in the subsequent image comparison sub-step 23, and include a body region corresponding to the posture and orientation of the three-dimensional model of the subject with the posture and orientation corresponding to the body region included in the reference binary image, and a background region corresponding to the region other than the body region. The body region and the background region included in the comparison binary image are respectively represented by the same first color and second color as the body region and the background region included in the reference binary image.

[0054] In this embodiment, since a three-dimensional model is generated using HS-Nets, in the aforementioned model generation sub-step 21, a front three-dimensional model having the posture and orientation corresponding to the body region included in the front image for model generation, and a side three-dimensional model having the posture and orientation corresponding to the body region included in the side image for model generation are respectively generated. Therefore, in this embodiment, by binarizing these front three-dimensional model and side three-dimensional model, a comparison front binary image and a comparison side binary image having the posture and orientation corresponding to the body region included in the reference binary image can be generated.

[0055] Next, in sub-step 23, each of the body regions of the reference binary image is compared with each of the body regions of the comparison binary image to calculate the difference value between the two (comparison sub-step).

[0056] The comparison of the images and the calculation of the difference value in this sub-step 23 can be performed, for example, by overlapping the reference binary image and the comparison binary image respectively. At this time, in order to measure the difference between the two images more accurately, it is desirable to adjust and overlap the two images so that the area of the region where the body region in the reference binary image and the body region in the comparison binary image do not overlap is minimized, and then calculate the difference value. For example, when adjusting the positions of two images, it is desirable to adjust them so that the vertical direction (height direction) of the body regions included in these images is fitted, and then overlap them. Also, the two images may be adjusted to reduce or eliminate the influence of the way the hands and feet are opened in the body region, or the lengths of the torso and legs in the body region may be corrected based on the skeleton estimated in the aforementioned step 13 (or step 13A), and then the overlapping may be performed. Also,

[0057] The difference value calculated by comparing two images can be determined, for example, by calculating the difference in the distance of the body regions of the two images and integrating the difference.

[0058] In this sub-step 23, the two images may be compared as a whole to calculate one difference value, or they may be compared for each specific region within the two images to calculate a plurality of difference values. For example, the difference value may be calculated for each of a plurality of predetermined body parts in a plurality of body regions. For example, by comparing two images, the difference values for each of the body parts such as the chest, torso, right arm, left arm, right leg, left leg, ··· etc. may be calculated.

[0059] Next, in sub-step 24, when the difference value is greater than a predetermined threshold value and the number of executions of this model output step 15 is less than a predetermined number of times, the comparison grayscale image is corrected and the model generation image is changed to the corrected comparison grayscale image (image correction sub-step).

[0060] Specifically, in this sub-step 24, if the difference value calculated in the above-described comparison sub-step 23 is equal to or less than a predetermined threshold value, the present model output step 15 is terminated and the process proceeds to the next measurement step 15. On the other hand, if the difference value calculated in the above-described comparison sub-step 23 is greater than the predetermined threshold value, in other words, if the difference between the reference grayscale image and the comparison grayscale image is large and the three-dimensional model generated by the model generation sub-step 21 does not sufficiently and accurately represent the body of the subject, the comparison grayscale image is corrected in order to re-execute the present model output step 15 using another model generation image, and the model generation image is changed to the corrected comparison grayscale image.

[0061] The above-mentioned predetermined threshold value can be arbitrarily determined according to the desired accuracy of the generated three-dimensional model. In addition, when a plurality of difference values are calculated, if at least one of the plurality of difference values is greater than the predetermined threshold value, it may be determined to re-execute the present model output step 15, or if the average of the plurality of difference values is greater than the predetermined threshold value, it may be determined to re-execute the present model output step 15. Further, different threshold values may be defined for each of the plurality of regions in which the difference values are calculated within the image, and the difference values may be compared with the threshold values for each of those regions.

[0062] However, depending on the reference grayscale image used, there may be a case where the difference value does not become equal to or less than the predetermined threshold value even if the present model output step 15 is repeated a large number of times and it is considered that no further expectation can be had. In order to stop the repetition of this step 15 in such a situation, in this sub-step 24, even if the difference value calculated in the above-described comparison sub-step 23 is greater than the predetermined threshold value, if the number of executions of the present model output step 15 reaches a predetermined number of times, the repetition of the present model output step 15 is terminated and the process proceeds to the next measurement step 15. Here, the predetermined number of executions of this step 15 set to determine the end of the repetition of this step 15 is arbitrarily determined according to the computing resources and required time allowed for the generation of the three-dimensional model. Also, if you do not want to stop the repetition as much as possible, you can set the predetermined number of executions to a sufficiently large value (for example, the maximum value that can be set).

[0063] The comparison binary image may be corrected by any method. For example, the correction of the comparison binary image can be performed by removing the noise component estimated based on the difference value from the comparison binary image. Alternatively, the correction of the comparison binary image can also be performed by correcting the dimensions of the comparison binary image. Here, the dimension correction may be performed on the entire comparison binary image, or may be performed on one or more specific body parts in the body region within the comparison binary image. For example, in the above-described comparison sub-step 23, when the body region corresponding to the right arm of the subject is compared and the calculated difference value is relatively large, the dimension of the region corresponding to the right arm of the body region in the comparison binary image may be corrected to be larger or smaller so as to approach the reference image. Note that the methods listed here may be performed alone or in combination of multiple methods.

[0064] After correcting the comparison binary image in this sub-step 24, the corrected comparison binary image is set as a new image for model generation, and this model output step 15 is executed again. The model output step 15 executed again starts from sub-step 21 again, generates a new three-dimensional model from this newly set image for model generation, and evaluates this in sub-steps 22 to 24.

[0065] In this way, in the model output step 15, the model generation term image used for generating the three-dimensional model is changed and the generation of the three-dimensional model is repeated until it can be confirmed that the generated three-dimensional model accurately represents the body of the photographed subject with the desired accuracy. As a result, the method 1 of the present embodiment can generate a three-dimensional model with high accuracy.

[0066] Thereafter, in step 16, when the finally calculated difference value is equal to or less than a predetermined threshold value, one or more dimensions related to the body characteristics of the subject are measured based on the finally generated three-dimensional model of the subject (measurement step).

[0067] The dimensions related to the body characteristics of the subject measured in this step 16 may be any dimensions such as length, height, perimeter, etc. related to any body part of the subject, and for example, may be measured as the dimensions of clothing sizes that fit the subject's body. The dimensions can be, for example, the subject's shoulder width, chest circumference, torso circumference, neck circumference, arm circumference, waist circumference, wrist circumference, thigh circumference, waist, hip, back length, sleeve length, body width, crotch length, inseam length, etc.

[0068] In measuring the dimensions, the body information of the subject acquired in advance may be referred to. For example, when the value of the subject's height has been given in advance, the dimensions can be calculated based on the ratio to the height of the three-dimensional model of the subject.

[0069] As described above, in the method 1 of the present embodiment, the model output step 15 for outputting the three-dimensional model of the subject repeats the sub-steps 21 to 24 as necessary (the first feature) until a three-dimensional model with a sufficiently small error between the comparison grayscale image generated from the three-dimensional model and the reference grayscale image based on the photographed image is obtained, so that a three-dimensional model in which the body of the subject is modeled with high accuracy can be output, and based on such a three-dimensional model, the body characteristics of the subject can be measured with high accuracy.

[0070] Furthermore, in Method 1 of the present embodiment, after the binarization step 12, a grayscale image correction step 13 is executed to correct the distortion of the subject's body area that may occur when acquiring a captured image (the second feature), so that in the model output step 15, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body features of the subject can be measured with high accuracy.

[0071] Furthermore, in Method 1 of the present embodiment, the body shape template data is composed of a body shape template generated from a group of people having one or more attributes possessed by the subject (the third feature), so that in the model output step 15, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body features of the subject can be measured with high accuracy.

[0072] As a result, a three-dimensional model of the subject can be modeled with high accuracy, and based on such a three-dimensional model, the body features of the subject can be measured with high accuracy.

[0073] FIG. 3 shows an embodiment of a system 3 including a computer capable of executing the body measurement method shown in FIG. 1. Note that the body measurement method shown in FIG. 1 does not necessarily have to be executed by the system 3 and can be executed by any computer system.

[0074] The system 3 includes a user terminal 31 and a server 32 as a computer that executes the body measurement method shown in FIG. 1.

[0075] The user terminal 31 can be any terminal that is operated by the subject or another person at a position away from the server 32 and can transmit and receive data to and from the server 32 via a network. For example, the user terminal 1 may be a desktop computer, a mobile computer, a tablet computer, a mobile phone, a smartphone, a wearable device, a handheld device, any embedded device, or the like.

[0076] Server 32 can be any server capable of performing computing processing on data transmitted from user terminal 31 via a network. For example, server 32 may be an on-premises server composed of one or more computer resources, or may be a cloud server implemented on a cloud network.

[0077] User terminal 31 and server 32 can arbitrarily share and execute each step executed in the body measurement method of the present invention described in detail below by transmitting and receiving data between them via a network. For example, in user terminal 31, while executing steps for acquiring data that can be acquired near the subject, such as photographing the subject, and steps for performing processing with a relatively small computational load, server 32, which has more computing resources than normal user terminal 2, may execute steps for performing processing with a large computational load.

[0078] Specifically, in system 3 of the present embodiment, user terminal 31 includes an image acquisition module 311, a binarization module 312, and a communication module 310. User terminal 31 further includes a camera 315 and a depth sensor 316. Image acquisition module 311 is configured to execute the above-described image acquisition step 11. Specifically, it is configured to acquire a photographed image by camera 315 provided in the user terminal. In addition, image acquisition module 311 is also configured to perform distortion correction processing on the photographed image using the depth data acquired by depth sensor 316. Binarization module 312 is configured to execute the above-described binarization step 12. Communication module 310 is configured to transmit and receive data with communication module 320 of server 32. As an example, it transmits the reference binary image generated by binarization module 312 to communication module 320 of server 32 for subsequent processing to be performed on server 32.

[0079] In addition, the server 32 includes a communication module 320, a halftone image correction module 323, a definition module 324, a model output module 325, and a measurement module 326. The communication module 320 is configured to transmit and receive data with the communication module 310 of the user terminal 31. As an example, it receives a reference halftone image transmitted from the communication module 310 of the user terminal 31. The halftone image correction module 323 is configured to execute the above-described halftone image correction step 13. The definition module 324 is configured to execute the above-described definition step 14. The model output module 325 is configured to execute the above-described model output step 15. The measurement module 326 is configured to execute the above-described measurement step 16.

[0080] Here, as shown in FIG. 4, the model output module 325 includes a model generation sub-module 421, a comparison image generation sub-module 422, a comparison sub-module 423, and an image correction sub-module 424. These sub-modules 421 to 424 are configured to execute the above-described sub-steps 21 to 24, respectively.

[0081] As described above, in the system 3 of the present embodiment, the user terminal 31 is configured to execute the image acquisition step 11 capable of acquiring a photographed image from the subject and the binarization step 12 with a relatively small computational load among the respective steps included in the method 1 of the present invention, while the server 32 is configured to execute the halftone image correction step 13 and the model output step 15 with a large computational load. Therefore, the system 3 of the present embodiment can efficiently distribute the computational load between the user terminal 31 and the server 32.

[0082] In the system 3 of the present embodiment, the binary image correction module 323 configured to execute the binary image correction step 13 is provided in the server 32. However, instead of or in addition to this, the user terminal 31 may be provided with a module 313 (not shown) configured to execute step 13A.

[0083] FIG. 5 shows an example of a shooting screen 50 displayed on an application 5 used when shooting a captured image acquired in the image acquisition step of the method of the present invention (for example, in step 11 of the above embodiment). The application 5 can operate, for example, on the user terminal 31 of the system 3. Hereinafter, it is assumed that the user terminal 31 is a smartphone and the application 5 is a smartphone application for explanation.

[0084] As shown in FIGS. 5(a) and (b), a guide frame 51 corresponding to the posture and orientation of the subject required for shooting is displayed on the shooting screen 50 of the application 5. Here, the posture shown in FIGS. 5(a) and (b) is the A pose, the orientation shown in FIG. 5(a) is the front-facing direction, and the orientation shown in FIG. 5(b) is the side-facing direction.

[0085] When the photographer shoots the subject using the application 5, an image reflected in the lens of the camera 315 provided in the user terminal 31 on which the application operates is displayed on the shooting screen 50, and the guide frame 51 continues to be displayed on the shooting screen 50 so as to overlap with the image. When the photographer shoots the body of the subject, the photographer adjusts the orientation of the camera 315 (and, if necessary, the user terminal 31 on which the camera 315 is provided), the distance to the subject, the magnification of the image displayed on the shooting screen 50 in the shooting screen 50, etc. so that the body of the subject is generally within the frame of the guide frame 51. After the body of the subject is sufficiently within the frame of the guide frame 51 on the shooting screen 50 by such adjustment, the photographer shoots the body of the subject.

[0086] In this way, the guide frame 51 displayed on the shooting screen 50 of the application 5 enables easy shooting of a subject in the required posture and orientation in a later model output step (for example, step 15). Therefore, it is possible to easily obtain a shooting image with less distortion suitable for model generation.

[0087] When the application 5 is used for the photographer to shoot a subject, if the subject's body does not fit well within the frame of the guide frame 51 on the shooting screen 50, shooting is not permitted. Only when the subject's body fits well within the frame of the guide frame 51, the touch area within the range where the character "Shoot" is displayed on the screen of the application 5 is activated to permit shooting. When the application 5 is configured as described above, the photographer can more easily shoot a subject in the required posture and orientation.

[0088] The application 5 is mainly used to shoot at least one shooting image obtained in the image acquisition step, but may also be used to obtain other useful information for implementing the body measurement method of the present invention. For example, the application 5 may further include an interface for the subject to directly input useful information that can be referred to in body measurement based on a three-dimensional model, such as the subject's height, weight, age, etc.

[0089] Note that the body measurement method executed by the computer according to this embodiment is not limited to the configuration of the above embodiment. Also, the body measurement method executed by the computer according to the present invention is not limited by the above-described effects. The body measurement method executed by the computer according to the present invention can be variously modified without departing from the gist of the present invention.

[0090] For example, in Method 1 shown in FIG. 1, it includes both sub-steps 21 to 24 that repeat the generation of the three-dimensional model according to the above-described first feature of the present invention, and step 13 (or step 13A) that corrects an image based on the skeleton estimation according to the above-described second feature of the present invention. However, the method of the present invention does not necessarily have both of these features.

[0091] For example, in the body measurement method of the present invention, if it includes sub-steps 21 to 24 that repeat the generation of the three-dimensional model, it does not necessarily have to perform step 13 (or step 13A) that corrects an image based on the skeleton estimation. Similarly, if one executes step 13 (or step 13A) that corrects an image based on the skeleton estimation, one may only execute the generation of the three-dimensional model once without performing sub-steps 21 to 24 that repeat the generation of the three-dimensional model. Even if the body measurement method of the present invention includes only one of the steps related to these features, it can measure the body features of the subject with high accuracy.

[0092] Also, although not repeating more detailed explanations here, even for matters not directly described above, for matters of conventionally known techniques regarding the body measurement method, they can also be appropriately adopted in the present invention.

Claims

1. A method executed by a computer for measuring the body of a subject, comprising: an image acquisition step of acquiring at least one captured image including a body region corresponding to the body of the subject and a background region corresponding to a region other than the body region; a binarization step of converting each of the captured images to generate the same number of reference binary images as the captured images, including a body region represented by a first color corresponding to the body region of the captured image and a background region represented by a second color corresponding to the background region of the captured image; a model output step of generating a three-dimensional model of the subject from the reference binary images using body template data and a convolutional neural network; including before the binarization step of generating the reference binary images, estimating the skeleton of the subject from the captured image, and using the estimated skeleton to correct the shape of the body region included in the captured image, or before the model output step of generating the three-dimensional model of the subject, estimating the skeleton of the subject from the captured image or the reference binary image, and using the estimated skeleton to correct the shape of the body region included in the reference binary image further including a method.

2. In the captured image correction step of correcting the shape of the body region included in the captured image or the binary image correction step of correcting the shape of the body region included in the reference binary image, the method according to claim 1, wherein a convolutional neural network is used to estimate the skeleton of the subject.

3. The method according to claim 1 or 2, wherein the at least one captured image includes an image of the subject taken from the front and an image of the subject taken from the side.

4. The method according to any one of claims 1 to 3, wherein the at least one captured image is an image captured by a user terminal of the subject.

5. The method according to claim 4, wherein the at least one captured image is a distortion-corrected image based on depth data acquired simultaneously with the capture of the captured image by a depth sensor provided in the user terminal.

6. The method according to claim 4 or 5, wherein the at least one captured image is an image captured on an application operating on the user terminal, and a guide frame corresponding to the posture and orientation of the subject required for capturing is displayed on the capture screen of the application.

7. The method according to any one of claims 1 to 6, wherein the body shape template data is constituted by a body shape template generated from a group of persons having one or more attributes possessed by the subject.

8. The method according to claim 7, wherein one or more attributes possessed by the subject include one or more of the nationality or race of the subject.

Images