Body information estimation method and body information estimation system

Abstract

To provide a body information estimation method for specifying a plurality of feature points in the body of a user on the basis of an image including the front and side face of the user and estimating abdomen information using the plurality of feature points.SOLUTION: A body information estimation method includes: an acquisition step for acquiring an image including the front and side face of a user; a specification process for specifying the position of a plurality of feature points in the body of the user on the basis of the image; a position specification process for specifying a specific position determined by positional relationships between the feature points and other feature points, and included in the abdomen of the user; and an estimation process for estimating abdomen information related to the size or the shape of the abdomen related to the specified specific position.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a physical information estimation method for estimating abdominal information from an image including a front and side view of a user. [Background technology]

[0002] There is a method for estimating the width of the front and the thickness of the sides of the torso of a subject near the navel. This method estimates the length of the area near the navel by placing a pair of hands on the area (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-198800 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the navel area of ​​the torso was determined using both hands of the person being estimated, so there was room for improvement in terms of operability when taking images. Also, because the navel area of ​​the torso was determined by the person being estimated, there was a problem that the position to be estimated varied depending on the person being estimated.

[0005] The present invention has been made in consideration of the above-mentioned problems, and relates to a physical information estimation method that identifies multiple feature points on a user's body based on an image including the front and side of the user, and estimates abdominal information using the multiple feature points. [Means for solving the problem]

[0006] The present invention relates to a physical information estimation method including an acquisition step of acquiring an image including a front and a side of a user, a specification step of specifying the positions of a plurality of feature points on the user's body based on the image, a position specification step of specifying a specific position that is determined by the positional relationship between the feature points and other feature points and is included in the abdomen of the user based on the plurality of feature points, and an estimation step of estimating abdominal information related to the size or shape of the abdomen related to the specified specific position.

[0007] The present invention also relates to a physical information estimation system that estimates physical information of a user using a control device, including: an acquisition means that acquires an image including the front and side of the user; an identification means that identifies the positions of a plurality of feature points on the user's body based on the image; a position identification means that identifies a specific position that is determined by the positional relationship between the feature point and other feature points and is included in the user's abdomen based on the plurality of feature points; and an estimation means that estimates abdominal information related to the size or shape of the abdomen related to the identified specific position. [Effects of the Invention]

[0008] According to the method provided by the present invention, abdominal information can be estimated with a simple operation by the user. [Brief explanation of the drawings]

[0009] [Figure 1] (a) is an image diagram of when a user is imaged from the front, and (b) is an image diagram of when a user is imaged from the side. [Figure 2] (a) is an image showing feature points, and (b) is an image after segmentation processing. [Figure 3] 10 is a process flow of a physical information estimation method. [Figure 4] 10A is a diagram showing a specific position and the length in the width direction of the front surface, and FIG. 10B is a diagram showing the length in the width direction of the side surface. [Figure 5]FIG. 10 is a diagram showing an experimental method for determining the relationship between a feature point and its positional relationship with other feature points and a specific position, which was conducted on subjects. [Figure 6] This is a diagram showing an ellipse with a major axis radius that is 1 / 2 of the front length shown in Figure 4 and a minor axis radius that is 1 / 2 of the side length. [Figure 7] FIG. 1 is a block diagram of a physical information estimation system. [Figure 8] FIG. 10 is a diagram showing an experimental method (another example) for determining the relationship between a feature point, its positional relationship with other feature points, and a specific position, which was conducted on a subject. DETAILED DESCRIPTION OF THE INVENTION

[0010] Preferred embodiments of the present invention will be described below with reference to the drawings. The drawings of the present embodiments are intended to explain the technical concept, configuration, and operation of the present invention, and are not intended to specifically limit the configuration. In all drawings, similar components are denoted by similar reference numerals, and redundant descriptions will be omitted where appropriate. In this specification, a "system" includes one or more information processing devices. For example, a system can be configured by a single information processing device, or multiple information processing devices working together to perform functions such as a web server can also constitute a system. Furthermore, a "system" may include an information processing device functioning as a web server, one or more terminal devices, and one or more imaging devices and inspection devices.

[0011] An overview of the physical information estimation method according to this embodiment (hereinafter sometimes referred to as the present method) will be described. The physical information estimation method of this embodiment includes an acquisition step of acquiring an image including the front and side of a user; an identification step of identifying the positions of multiple feature points on the user's body based on the image; a position identification step of identifying a specific position included in the user's abdomen, which is determined by the positional relationship between the feature point and other feature points, based on the multiple feature points; and an estimation step of estimating abdominal information related to the size or shape of the abdomen related to the identified specific position.

[0012] In this method, as shown in FIG. 1, an image capturing device 20 captures images of the user's entire body in the front direction (FIG. 1(a)) and the side direction (FIG. 1(b)). While FIG. 1(a) shows the user being captured with their arms extended horizontally, the user's pose during image capture is not limited to this. In this embodiment, in order to estimate abdominal information, an image is captured with the user's arms extended horizontally as shown in FIG. 1(a), which makes it possible to capture an abdominal image in the side direction without the arms overlapping, as shown in FIG. 1(b). Furthermore, the user may remain clothed during image capture. Although details will be described later, for example, thick clothing such as a down jacket is likely to result in estimation errors, so it is preferable to wear thin clothing such as a T-shirt, a jersey, or sports pants. FIG. 2 is a conceptual diagram illustrating the location of multiple feature points on a user's body based on a captured image. FIG. 2(a) illustrates the extracted skeletal points of a user extracted from a captured image. The user's skeletal points are extracted using a known skeletal information acquisition technology, such as MediaPipe Pose, a library that infers a person's posture from an image. Skeletal points are extracted as shown in FIG. 2(a). The positions and number of extracted skeletal points depend on the skeletal point acquisition technology used. Here, the skeletal points specifically used in this embodiment are extracted: the top of the head 10, both shoulders 11 (both acromions), both buttocks 12 (anterior superior iliac spine (pelvis)), and heels 13. The user is then segmented using a known technology, such as Selfie Segmentation, a library that segments a specific object (e.g., a person) from an image. FIG. 2(b) illustrates the result of segmenting the acquired image and separating the person from the background. In this embodiment, MediaPipe Pose is used when extracting human skeleton points from an image, but this is not limiting and other methods such as VISION POSE and OPEN POSE may also be used. These images and the extracted skeleton points are used as feature points, and abdominal information is estimated based on the feature points.

[0013] The processing flow of this method is shown in Fig. 3. This method includes an acquisition step (step S100), a determination step (step S200), a position determination step (step S300), and an estimation step (step S600).

[0014] Step S100 is an "acquisition step" in which an image including the front and side of the user is acquired. An "image including the front and side views of the user" refers to, for example, the images shown in FIGS. 1(a) and 1(b). The front image shown in FIG. 1(a) and the side image shown in FIG. 1(b) may be separate images (each image), or a single image (including both the front and side views). A "user" can be any gender, age, or height. The term "front" refers to a direction in which the user's face is facing straight ahead, a direction in which the user can view the image capture device 20 from a standing position, a direction in which the user's navel is approximately parallel to the image capture device 20 from a standing position, a direction in which the user's chest is located, and also includes a direction in which the back of the user's head is located and a direction in which the user's back is located. The term "side" refers to a direction in which the user's front is rotated approximately 90 degrees around a vertical axis, and includes both a right-side view, which is the user's right side, and a left-side view, which is the user's left side. Furthermore, the "image" may be either a still image or a moving image. For example, when a user is taking images by themselves using a camera built into a smartphone, it is preferable to use a moving image in order to easily obtain estimated abdominal information related to the size or shape of the abdomen. In this case, the user stands in front of the camera that starts capturing the moving image, captures a frontal image as described above, and then rotates their body around a vertical axis in front of the camera so that a side image can be captured. Furthermore, the imaging device 20 may be a general RGB camera, a monochrome camera, or a spectral camera, and the performance and specifications of the imaging device 20 are not limited. Examples of the imaging device 20 include a video camera, a camera built into a smartphone, a camera built into a tablet device, and a webcam that can be attached to a personal computer or the like via a connection means such as a cable. "Acquire" means to acquire images, and the source of the time-series images does not matter, such as acquiring from a specified medium, acquiring via a network, or acquiring directly from a photographing device (importing directly from the photographing device).

[0015] Step S200 is an "identification step" that identifies the positions of a plurality of feature points on the user's body on the image based on the acquired image. "Multiple feature points on the body" are points (positions) that can identify a specific position on the body, such as skeletal points such as the top of the head, neck, acromion, wrists, and ankles, points related to the positions of bones such as joints, and points related to the positions of body parts such as the corners of the eyes, the tip of the nose, and the tips of the fingers. In this embodiment, MediaPipe Pose is used to extract the following skeletal points: those used to identify height (e.g., top of the head, heels, etc.); those highly correlated with height (e.g., fingertips, shoulders, neck, pelvis, knees, heels, etc.); those that tend to differ depending on gender (e.g., pelvis, shoulders, rib cage, etc.); those related to the upper limbs (e.g., neck, shoulders, elbows, wrists, etc.); and those related to the lower limbs (e.g., buttocks, knees, ankles, heels, etc.). In this embodiment, the skeletal points used to identify height (top of the head 10, heels 13), those related to the upper limbs (shoulders 11), and those related to the lower limbs (buttocks 12) are extracted, and each of the extracted skeletal points is designated as a feature point. The coordinates (X (horizontal) and Y (vertical)) of each extracted feature point on the image are also identified. Furthermore, the user's height distance m (see FIG. 4) is calculated from the coordinates of the top of the head 10, which is one of the identified feature points, and the coordinates of the heel 13, which is also one of the feature points, and the number of pixels per predetermined unit (1 cm in this embodiment) on the image is calculated from the calculated height distance m and the user's actual height. The number of pixels per predetermined unit is used in the processing described below.

[0016] Step S300 is a "position specifying step" that specifies a specific position that is determined by the feature points and their positional relationships with other feature points and that is included in the abdomen of the user, based on a plurality of feature points. The phrase "determined by the positional relationship between a feature point and other feature points" refers to a determination based on the positions of multiple feature points, and examples thereof include an interpolated position between a feature point and other feature points and an extrapolated position between a feature point and other feature points. Specifically, examples include a predetermined position on a virtual line connecting a feature point to other feature points, a predetermined ratio position, a central position, and a position away from the feature point by a predetermined ratio of the distance from the feature point to the other feature points. Examples other than the interpolated position or extrapolated position include the intersection position of a virtual line at a predetermined angle from a feature point and a virtual line at a predetermined angle from the other feature point. Also, a position on the perpendicular bisector of a virtual line connecting a feature point to other feature points, which is a predetermined multiple of the distance between the two points, may be used. Furthermore, in addition to the feature point and other feature points, there may be additional feature points (three or more feature points). For example, the center of gravity of all feature points or an average value obtained by weighting the coordinates of each feature point may be used. Alternatively, it may be the intersection position of a virtual line connecting a feature point (referred to as a first feature point) to another feature point (referred to as a second feature point) and a virtual line connecting a feature point (referred to as a third feature point) different from the first feature point and the other feature point and another feature point (referred to as a fourth feature point) different from the first feature point and the other feature point. The "abdomen" refers to the lower half of the user's torso, and in this embodiment, refers to the area around the navel in the lower half of the torso. Furthermore, the "specific position included in the abdomen" refers to a position determined by the feature point and other feature points, and is a position included in the abdomen. The specific position in this embodiment will be described with reference to FIG. 4. In this embodiment, the feature point is the shoulder 11, and the other feature point is the buttocks 12. As shown in FIG. 4(a), a position 2 / 3 of the way from the shoulder 11 is determined with respect to a virtual line 21 connecting the shoulder 11 and the buttocks 12, and this position, which is included in the abdomen, is designated as the specific position 22. This relational expression was derived as a result of extensive research into extracting various feature points from images of multiple subjects and determining the positional relationship of the extracted feature points that indicates the abdomen position. In other words, the present inventors identified multiple feature points correlated with the abdomen position and discovered a relational expression that shows the relationship between the multiple feature points. Note that, for convenience of explanation, the virtual line 21 is depicted in FIG. 4(a), but the virtual line 21 is not necessary because the coordinates of each acquired feature point are used. In the above, a position 2 / 3 from the shoulder 11 is determined on the imaginary line 21 connecting the shoulder 11 and the buttocks 12, and the position included in the abdomen is set as the specific position 22, but this is not limited to this, and the position 22 can be changed appropriately to between 3 / 5 and 4 / 5 from the shoulder 11 on the imaginary line 21 connecting the shoulder 11 and the buttocks 12, and it is preferable to set it to between 2 / 3 and 3 / 4.

[0017] The measurement method used to derive the relational equation for multiple subjects will be explained using FIG. As shown in FIG. 5, the subject is asked to point to their navel with a finger or the like, and an image of this is obtained. This image is preferably the same as the image from which the user's skeletal points are extracted using MediaPipe Pose or the like. Then, of the extracted skeletal points, the percentage of the distance (on virtual line 21) along the Y axis from shoulder 11 to buttocks 12 at which the subject pointed to the navel was calculated. As a result of this measurement, the position at which the subject pointed to the navel was between 7 / 10 and 3 / 4 of the way from shoulder 11 to buttocks 12. It was also found that when the virtual line 25 connecting the buttocks 12 (left hip joint) and the buttocks 12 (right hip joint) is used as the base, the navel position is located at the vertex of the virtual equilateral triangle that does not include the virtual line 25. Therefore, if the feature point is the user's buttocks 12 (left hip joint), and the other feature point is the buttocks 12 (right hip joint), and virtual lines 26 and 27 are drawn at an angle of 60 degrees in the interpolation direction from both feature points, the intersection of these lines will be the navel position, and the position determined by this relationship is the abdomen, so it may be determined as the specific position. It should be noted that, although a plurality of feature points correlated with abdominal position are identified and a relational equation showing the relationship between the plurality of feature points is described, the information is not limited to a relational equation and may be a table showing the relationship. Any format of information may be used as long as it shows the correlation statistically obtained from feature points extracted by imaging a plurality of subjects and specific positions of the plurality of subjects (in this embodiment, the abdomens related to the specific positions). In this way, while it is easy to identify feature points such as skeletal points from an acquired image, the abdomen does not have any feature points like skeletal points, so in order to identify the abdomen, it was necessary to capture an image with the navel visible or with the user placing their hand on the abdomen. However, with this method, the abdomen can be identified from the positional relationship between the identified feature points and other feature points, so abdominal information can be estimated with a simple operation by the user, without requiring them to put on or take off excessive clothing that would expose their skin.

[0018] Step S600 is an "estimation step" in which abdominal information relating to the size or shape of the abdomen relating to the specific position is estimated. "Abdominal size" includes not only the abdominal circumference (perimeter), frontal length, lateral length, and abdominal cross-sectional area, but also the rank of each of the abdominal circumference (perimeter), frontal length, lateral length, and abdominal cross-sectional area (for example, in the case of circumference (perimeter), the rank is A: 60 cm or less, B: 61 cm to 75 cm, C: 76 cm to 90 cm, D: 91 cm to 105 cm, E: 106 cm or more). Furthermore, "abdominal shape" refers to terms that represent the shape, such as circle, horizontally elongated ellipse, or vertically elongated ellipse, as well as images and illustrations that show the estimated shape. Furthermore, as described above, "abdominal information" can be any information that represents the estimation result, such as numerical values, ranks, terms, images, and illustrations, and is not limited to specific numerical values. For example, an image in which a mark indicating the abdomen or a circumferential line indicating the abdominal circumference is drawn on the acquired image is also abdominal information.

[0019] The above-mentioned estimation process may include a "front length calculation process" which calculates a front length A, which is the length of the abdomen in the front width direction at a specific position shown in Figure 4, and a "side length calculation process" which calculates a side length B, which is the length of the abdomen in the side width direction at a specific position, and abdominal information may be estimated based on the calculated front length A and side length B. The "frontal length, which is the length of the abdomen in the front width direction at a specific position" is the frontal length A at the specific position shown in Figure 4(a). The frontal length A of the user is calculated by calculating the number of pixels at the specific position of the front image of the user, and then calculating the frontal length A based on the calculated number of pixels and the number of pixels per cm that has been calculated in advance. The "side length, which is the length in the side width direction of the abdomen at a specific position" is the side length B at the specific position shown in Figure 4(b). The side length B of the user is calculated by calculating the number of pixels at a specific position in the side image of the user, and calculating the side length B based on the calculated number of pixels and the number of pixels per cm that has been calculated in advance.

[0020] Based on the calculated front length A and side length B, the abdominal cross-sectional area S and abdominal circumference are estimated as abdominal information as follows. <Abdominal cross-sectional area S> Generally, the cross-sectional area parallel to the width direction of the abdomen is elliptical, so the cross-sectional area S of the abdomen is calculated assuming that the front length A is an ellipse with the major axis length A and the side length B is an ellipse with the minor axis length B. Figure 6 is a diagram showing an ellipse whose major axis is 1 / 2 of the front length A and whose minor axis is 1 / 2 of the side length B. The area S of the ellipse shown in Figure 6 is calculated by the following formula (1).

number

[0021] <Abdominal circumference> The abdominal circumference L is the circumference of the ellipse shown in FIG. 6, and can be calculated using the front length A and the side length B using the following formula (2).

number

[0022] Under the condition of B>A, the approximation of the above formula (2) is expressed as the following formula (3).

number

[0023] Next, a method for improving the estimation accuracy of abdominal information will be described. One factor that may reduce the accuracy of abdominal information estimation is the thickness of the clothing worn by the user. This method aims to enable users to easily measure their abdomens in their daily lives, so it is assumed that the user will capture images while wearing their own clothing. Since the clothing is the user's own clothing, it is expected that the thickness and material of the clothing will vary. Therefore, the estimation process preferably sets the thickness of the clothing according to the type of clothing worn by the user, and preferably also includes a process for correcting the estimated abdominal information according to the thickness of the clothing worn by the user. A table containing the thickness of clothing for each type of clothing (e.g., thin T-shirts, dress shirts, cut-and-sew tops) is prepared in advance, and the thickness of the clothing is determined by having the user specify the type of clothing. The estimated abdominal information is then corrected according to the determined clothing thickness. The correction method involves measuring in advance how much the estimated nude measurements should be increased for each type of clothing, and then subtracting that percentage from the estimated value. Furthermore, when measuring how much the estimated nude measurements should be increased for each type of clothing, it is preferable to measure how the clothing is worn in advance. The way in which clothing is worn, for example, the amount of surcharge to nude measurements varies depending on whether the same T-shirt is worn over pants or tucked into them. Therefore, it is more effective to require the user to specify the way in which the clothing is worn in addition to the type of clothing. Since it is assumed that some users wear multiple pieces of clothing, accuracy can be improved by having the user specify this if multiple pieces are worn. Instead of requiring the user to specify the type of clothing, the user may also be required to specify the fashion style being worn. The fashion style indicates the clothing and the way in which it is worn, such as a style in which a T-shirt is worn over a jersey, a style in which a shirt is worn over jeans, or a style in which a T-shirt is tucked into jeans. The user may specify the fashion style using an image depicting the fashion style. The amount of surcharge to nude measurements is then measured in advance for each fashion style. This enables corrections to be made taking into account the type, way in which it is worn, and the number of pieces of clothing.Alternatively, rather than the user specifying the type of clothing, machine learning may be used to analyze the acquired image of the user to determine the type of clothing, how it is worn, the number of pieces, or fashion style, and determine the amount of surcharge for nude measurements. In this manner, in this embodiment, highly accurate estimation can be performed on the feature points and other feature points of the acquired image, even if the abdomen is covered with clothing.

[0024] As described above, in this embodiment, the user's height distance m is calculated from the coordinates from the vertex 10 to the heel 13, and the number of pixels per predetermined unit on the image is calculated based on the user's actual height. Therefore, it is preferable to provide a step ("physical feature acquisition step") of acquiring height from the user as a physical feature other than the abdomen before calculating the number of pixels per predetermined unit (e.g., before step S200 in FIG. 3). Other physical features that can be acquired include weight, age, and gender in addition to height. By acquiring data that correlates with the positional relationship between the estimated body part, the feature point, and other feature points (e.g., a correlation between the positional relationship between the estimated body part, the feature point, and other feature points and age, or a correlation between the positional relationship between the estimated body part, the feature point, and other feature points and gender), it is possible to improve estimation accuracy. In other words, this embodiment preferably further includes a physical feature acquisition step of acquiring the user's physical features other than the abdomen, and the estimation step preferably estimates the abdominal information using the physical features.

[0025] <Physical Information Estimation System> FIG. 7 shows a conceptual diagram of a physical information estimation system 400 (hereinafter, sometimes referred to as "this system"). This system is composed of an acquisition unit 330, an identification unit 340, a position identification unit 350, and an estimation unit 360. It also includes an information processing terminal 300 capable of executing various processes, and the information processing terminal 300 is equipped with the acquisition unit 330, the identification unit 340, the position identification unit 350, and the estimation unit 360. The information processing terminal 300 is equipped with an input device such as a keyboard or a pointing device, a processing unit, a memory unit, etc. It is preferable that the information processing terminal 300 is equipped with a display device (display unit 380), but this may be provided external to the information processing terminal and connected via a network. The physical information estimation system 400 may also include an internal or external imaging device 20.

[0026] The acquisition unit 330 is a means for acquiring an image including the front and side of the user, and corresponds to an acquisition means. The identification unit 340 is a means for identifying the positions of a plurality of feature points on the user's body based on the acquired image, and corresponds to an identification means. The position specifying unit 350 is a means for specifying a specific position that is determined by the feature points and their positional relationships with other feature points and is included in the abdomen of the user based on a plurality of feature points, and corresponds to a position specifying means. The estimation unit 360 corresponds to an estimation means for estimating abdominal information relating to the size or shape of the abdomen related to the identified specific position. This system is capable of executing the processing content of the physical information estimation method described above. From the above, the present invention discloses a physical information estimation system that estimates physical information of a user using a control device, including: an acquisition means that acquires an image including the front and side of the user; an identification means that identifies the positions of a plurality of feature points on the user's body based on the image; a position identification means that identifies a specific position that is determined by the positional relationship between the feature point and other feature points and is included in the user's abdomen based on the plurality of feature points; and an estimation means that estimates abdominal information related to the size or shape of the abdomen related to the identified specific position.

[0027] Furthermore, the physical information estimation system 400 may further include a body fat mass estimation unit (body fat mass estimation means) that estimates the body fat mass contained in the user's abdomen from relationship information that indicates the relationship between the estimated abdominal information and the body fat mass contained in the abdomen. In this case, the body fat mass estimation unit may be provided in the information processing terminal 300. A method for estimating body fat mass from the acquired abdominal information will be described below.

[0028] Focusing on the fact that body fat mass correlates with abdominal information, the body fat mass is estimated using relational information indicating the relationship between abdominal information estimated by this system and body fat mass. "Relationship information showing the relationship between abdominal information and the amount of body fat contained in the abdomen" is information showing the correlation between abdominal information and the amount of body fat contained in the abdomen, such as an arithmetic formula or table showing the relationship between the front length and lateral length of abdominal information and the amount of body fat, or an arithmetic formula or table showing the relationship between the circumference of abdominal information and the amount of body fat. Table 1 is an example of relationship information and is a table showing the relationship. Table 1 associates the major axis A (corresponding to "frontal length A") and the minor axis B (corresponding to "lateral length B") with the cross-sectional area S of body fat (also referred to as cross-sectional area S) that occupies the cross-sectional area of ​​the abdomen. The table shown in Table 1 divides the expected numerical range of the frontal length and the lateral length into multiple stages, and associates the cross-sectional area S of body fat with the frontal length and the lateral length of each stage. The table shown in Table 1 was created in advance from the measurements of the frontal length and the lateral length of multiple subjects, and the cross-sectional area S indicating the body fat mass of these multiple subjects was measured using a method such as CT scan. The ranges (stages) of the frontal length and the lateral length shown in Table 1 may be determined depending on the measurement accuracy and the accuracy of the cross-sectional area of ​​the body fat mass to be estimated. [Table 1]

[0029] Table 2 is another example of relationship information, a table showing the relationship. Table 2 associates circumference L (corresponding to "abdominal circumference") with the cross-sectional area S of body fat (also called cross-sectional area S) that occupies the cross-sectional area of ​​the abdomen. The table shown in Table 2 divides the expected numerical range of circumference into multiple stages in advance, and associates the cross-sectional area S of body fat with the circumference of each stage. The table shown in Table 2 was created in advance from measurements of circumferences of multiple subjects and the cross-sectional areas S indicating the body fat mass of these multiple subjects, measured using a method such as CT scan. The ranges (stages) of circumference shown in Table 2 may be determined depending on the measurement accuracy and the accuracy of the cross-sectional area of ​​the body fat mass to be estimated. [Table 2]

[0030] As described above, this system can estimate body fat mass using relationship information indicating the relationship between the estimated abdominal information and body fat mass. While the above description has been given for a standard body type with an oval abdominal cross-section, it is known that the location of body fat affects body shape (abdominal cross-sectional shape). Specifically, the following applies: The "standard body type" has an oval abdominal cross-section, with gradually increasing abdominal thickness (lateral length). The "visceral obesity type" has a large amount of visceral fat and an abdominal cross-section that is closer to a circle than an oval. The "subcutaneous obesity type" has a large amount of subcutaneous fat, fat on the sides of the back (both sides), and different frontal lengths on the navel side and back side. The "athlete type" tends to have less body fat and a large abdominal pressure due to well-developed abdominal muscles, back muscles, and iliopsoas muscles. Because the relationship between abdominal circumference and body fat mass differs depending on the type (depending on the type), it is preferable to estimate body fat mass using the estimated abdominal circumference as well as the abdominal cross-sectional shape, which is one piece of abdominal information. In this case, the relationship between abdominal circumference and body fat mass (e.g., Table 2) is determined for each type, the type is identified based on the cross-sectional shape, and body fat mass is estimated based on the relationship between abdominal circumference and body fat mass for the identified type. This enables more accurate estimation of body fat mass based on estimated abdominal information. In the case of the athlete type, it is more preferable to correct the estimated body fat mass based on muscle mass. In this case, the circumference of a part of the body where muscle accounts for a high proportion of the cross-sectional area, such as the upper arms or thighs, may be estimated, and the estimated body fat mass may be corrected using the estimated circumference. Specifically, since it can be inferred that the larger the circumference of the upper arms or thighs, the greater the muscle mass, the relationship between the circumference and the percentage by which the estimated abdominal body fat mass should be reduced may be determined in advance, and correction may be performed based on this relationship.

[0031] As described above, the present invention has been described by showing specific embodiments, but the present invention is not limited to the above-described embodiments and includes various modifications, improvements, and other aspects as long as the object of the present invention is achieved. <Modification> In this embodiment, the number of pixels per predetermined unit on the image is calculated from the user's height distance in the image and the user's actual height, and abdominal information is estimated using the number of pixels per predetermined unit, but this is not limited to this. For example, abdominal information may be estimated from the length of a specific position in the image based on the distance between the image capture device 20 and the user when capturing an image of the user. Alternatively, a measurement value (actual length) of another part may be used instead of height. While similar results can be obtained in this way, it is preferable to use actual height considering the burden on the user when capturing an image (measuring the distance between the image capture device 20 and the user, or capturing an image at a predetermined distance, or measuring another part).

[0032] The feature point may be the shoulder 11, and the other feature point may be the knee 15. A diagram showing the positional relationship between the shoulder 11 and the knee 14 is shown in FIG. As shown in Fig. 8, it was found that the navel is located between 2 / 5 and 9 / 25 from the shoulder 11 with respect to the imaginary line 28 connecting the shoulder 11 and the knee 15. Therefore, the relationship found by setting the shoulder 11 as the feature point and the knee 15 as the other feature point may be used.

[0033] In this embodiment, abdominal information is estimated, but information on other parts may also be estimated. Other parts can be estimated in the same way as long as they have a correlation with the feature points and their positional relationships with other feature points.

[0034] In this embodiment, the feature point is the user's shoulder 11 (shoulder joint) in the acquired image, and the other feature point is the user's buttocks 12 (hip joint) in the acquired image. This is because, when using known skeletal information acquisition technology, it is easy to accurately acquire the user's skeletal points, especially joints, in the acquired image, so by using joints as feature points, it is possible to accurately identify specific positions included in the abdomen. However, it is also possible to make highly accurate estimations using skeletal points other than joints.

[0035] The estimation process of this embodiment includes a frontal length calculation process and a lateral length calculation process, and estimates the abdominal circumference and abdominal cross-sectional area as abdominal information using the frontal length A calculated in the frontal length calculation process and the lateral length B calculated in the lateral length calculation process, but is not limited to this. For example, the estimation process may include an information calculation process in which multiple frames of images are acquired and a quantitative value that can be converted into abdominal information is calculated for each frame. The estimation process may then estimate abdominal information using the quantitative value for each frame calculated in the information calculation process.

[0036] In the present embodiment, the longer of the front length and the lateral length is described as the front length A and the shorter of the front length is described as the lateral length B, but this is not limited thereto. As described above, the cross-sectional shape of the abdomen may be circular rather than elliptical due to the location of body fat or the large amount of muscle mass of athletes. In this case, since the longer side cannot be defined as the front length A and the shorter side cannot be defined as the lateral length B, the specified lengths may be stored in association with information indicating the direction in which the user was imaged (for example, information extracted from the orientation of the user's face in the acquired image), and the cross-sectional shape may be estimated using the information indicating the image-capturing direction.

[0037] In the present embodiment, as shown in the cases where the abdomen is identified based on the positional relationship between the feature point (shoulder 11) and another feature point (buttocks 12) and the case where the abdomen is identified based on the positional relationship between the feature point (shoulder 11) and another feature point (knee 15), two feature points are used to identify the abdomen in both cases. However, this is not limited to this. For example, the specific position (abdomen) may be identified by averaging the results of both the positional relationship between the shoulder 11 (right shoulder) and the buttocks 12 (right buttocks) and the positional relationship between the shoulder 11 (left shoulder) and the buttocks 12 (left buttocks). That is, the specific position (abdomen) may be identified by averaging the positional relationship between the feature points present on the right half of the body and other feature points and the positional relationship between the feature points present on the left half of the body and other feature points. Many users have different skeletal positions (heights) between the right and left halves of their bodies due to factors such as lifestyle habits. Therefore, the estimation accuracy of abdominal information can be improved by identifying the abdomen based on the average value of the results for the right and left halves of the body. Furthermore, for example, the abdomen may be identified based on the positional relationship between the shoulder 11 (right shoulder) and the buttocks 12 (left buttocks) and the positional relationship between the shoulder 11 (left shoulder), the buttocks 12 (right buttocks), and the buttocks 12 (left buttocks). The abdomen is located near the navel and is therefore approximately at the center of the user's body in the left-right direction. Therefore, the specific position (abdomen) may be identified based on the intersection of the positional relationship between the shoulder 11 (right shoulder) and the buttocks 12 (left buttocks) (a virtual line connecting the shoulder 11 (right shoulder) and the buttocks 12 (left buttocks)) and the positional relationship between the shoulder 11 (left shoulder), the buttocks 12 (right buttocks), and the buttocks 12 (left buttocks) (a virtual line connecting the shoulder 11 (left shoulder), the buttocks 12 (right buttocks), and the buttocks 12 (left buttocks). In other words, the specific position (abdomen) may be identified based on the positional relationship between a feature point on the right half of the body and another feature point on the left half of the body.

[0038] In this embodiment, when identifying the abdomen based on the positional relationship between the feature point (shoulder 11) and another feature point (buttocks 12), a position between 3 / 5 and 4 / 5 from the shoulder 11 on the virtual line 21 connecting the feature point (shoulder 11) and another feature point (buttocks 12) is determined, and the position that is included in the abdomen is set as the specific position 22. However, this is not limited to this. For example, a Y coordinate value between 3 / 5 and 4 / 5 from the shoulder between the Y coordinate value of the feature point (shoulder 11) and the Y coordinate value of the other feature point (buttocks 12) may be determined, and the position that is included in the abdomen and has that Y coordinate value may be set as the specific position 22. In either case, abdominal information can be estimated with high accuracy. Similarly, when identifying the abdomen based on the positional relationship between the feature point (shoulder 11) and another feature point (knee 15), a position between 2 / 5 and 9 / 25 from the shoulder 11 on the virtual line 28 connecting the feature point (shoulder 11) and another feature point (knee 15) is determined, and the position that is included in the abdomen is regarded as the specific position, but this is not limited to this. For example, a Y coordinate value between 2 / 5 and 9 / 25 from the shoulder between the Y coordinate value of the feature point (shoulder 11) and the Y coordinate value of another feature point (knee 15) may be determined, and the position that is included in the abdomen and has that Y coordinate value may be regarded as the specific position. In either case, abdominal information can be estimated with high accuracy.

[0039] The present invention has been described above with reference to the embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the disclosure in this specification. Note that, in the present invention, the preferred examples of the physical information estimation device, the physical information estimation program, and the physical information estimation system are similar to those described above in relation to the physical information estimation method. [Explanation of symbols]

[0040] 10 top of head 11 Shoulders 12 Butt 13 Heel 14 Elbow 15 Knee 20 Imaging device 21 Virtual Line 22 Specific position 25 Virtual Line 26 Virtual Line 27 Virtual Line 28 Virtual Line 300 Information processing terminal 330 Acquisition Department 340 Specific part 350 Location identification part 360 Estimation Department 380 Display 400 Physical Information Estimation System

Claims

1. an acquisition step of acquiring an image including a front and a side of the user; an identifying step of identifying positions of a plurality of feature points on the user's body based on the image; a position specifying step of specifying a specific position, which is determined by a positional relationship between the feature point and other feature points and is included in the abdomen of the user, based on the plurality of feature points; and an estimation step of estimating abdominal information relating to the size or shape of the abdomen relating to the identified specific position.

2. The estimation step includes: a frontal length calculation step of calculating a frontal length, which is the length of the abdomen in the front width direction at the specified specific position; a side length calculation step of calculating a side length, which is the length in the side width direction of the abdomen at the specified specific position, 2. The method for estimating physical information according to claim 1, wherein the abdominal information is estimated based on the calculated front length and side length.

3. 3. The physical information estimation method according to claim 1, wherein the feature point is a shoulder joint of the user, and the other feature point is a hip joint of the user.

4. further comprising a physical feature acquiring step of acquiring a physical feature of the user other than the abdomen, 3. The physical information estimating method according to claim 1, wherein the estimation step estimates the abdominal information using the physical characteristics as well.

5. 3. The physical information estimating method according to claim 1, wherein the estimation step corrects the abdominal information in accordance with a thickness of clothing worn by the user.

6. The method for estimating physical information according to claim 5 , wherein the estimation step sets a thickness of the clothing depending on the type of the clothing.

7. A physical information estimation system that estimates physical information of a user using a control device, an acquisition means for acquiring an image including a front and a side of the user; an identification means for identifying positions of a plurality of feature points on the user's body based on the image; a position specifying means for specifying a specific position that is determined by a positional relationship between the feature point and other feature points and that is included in the abdomen of the user based on the plurality of feature points; and an estimation means for estimating abdominal information relating to the size or shape of the abdomen relating to the identified specific position.

8. The physical information estimation system according to claim 7, further comprising a body fat mass estimation means for estimating the body fat mass contained in the abdomen of the user from relationship information indicating the relationship between the abdominal information and the body fat mass contained in the abdomen.

Images