Physique determination device and physique determination method

Abstract

Included are: a captured image acquisition unit (11) that acquires a captured image in which a subject is captured by an image-capturing device that captures an image of a range to be captured in which at least the upper body of the subject and a backrest of a seat in which the subject is seated are to be present; a first feature calculation unit (12) that, on the basis of the captured image acquired by the captured image acquisition unit (11), calculates a first feature that is a feature in the captured image relating to an outline of the seat; a second feature calculation unit (13) that, on the basis of the captured image acquired by the captured image acquisition unit (11), calculates a second feature that is a feature in the captured image relating to the subject; and a physique determination unit (14) that determines physique of the subject on the basis of the first feature calculated by the first feature calculation unit (12) and the second feature calculated by the second feature calculation unit (13).

Description

Physique determination device and physique determination method

[0001] The present disclosure relates to a physique determination device and a physique determination method.

[0002] 2. Description of the Related Art Conventionally, there is known a technique for determining the physique of a subject seated in a seat based on a captured image of the subject (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-104680

[0004] When a seat in which a subject is seated is moved forward or backward relative to an imaging device capturing an image of the subject, the subject appears larger or smaller in the captured image as the seat moves. Conventional technologies have had the problem of erroneously determining the subject's physique when the seat is moved forward or backward relative to the imaging device. Patent Document 1 states that the accuracy of detecting the occupant's physique decreases when the seat state changes (e.g., sliding or reclining). However, Patent Document 1 does not disclose a specific method for detecting the physique of an occupant when the position of the seat changes as the seat is moved, such as when the seat is slid forward or backward. Therefore, the technology disclosed in Patent Document 1 still cannot solve the above-mentioned problem.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a physique determination device that performs physique determination regardless of the position of the seat in which the subject is sitting.

[0006] The physique determination device according to the present disclosure includes an image acquisition unit that acquires an image of a subject using an imaging device that captures an image of a target range in which at least the upper body of the subject and the back of the seat on which the subject is seated should be located; a first feature calculation unit that calculates a first feature, which is a feature in the image related to the contour of the seat, based on the image acquired by the image acquisition unit; a second feature calculation unit that calculates a second feature, which is a feature in the image related to the subject, based on the image acquired by the image acquisition unit; and a physique determination unit that determines the physique of the subject based on the first feature calculated by the first feature calculation unit and the second feature calculated by the second feature calculation unit.

[0007] According to the present disclosure, it is possible to perform a physical build determination that is not affected by the position of the seat in which the subject is seated.

[0008] 3A, 3B, and 3C are diagrams for explaining an example of a ratio of the area of ​​a skeleton point region to the area of ​​a seat region, which changes depending on the occupant's physique. FIGS. 5A, 5B, and 5C are diagrams for explaining a ratio of the first inter-skeleton point distance to the first inter-seat point distance, which changes depending on the occupant's physique. FIGS. 5A, 5B, and 5C are diagrams for explaining a ratio of the first inter-skeleton point distance to the first inter-seat point distance, which changes depending on the occupant's physique. FIGS. 5A, 5B, and 5C are diagrams for explaining a first inter-skeleton point distance to the first inter-seat point distance, which changes depending on the occupant's physique. FIGS. 5B, 5C, and 5C are diagrams for explaining a .... FIGS. 5B, 5C, and 5C are diagrams for explaining a first inter-skeleton point distance to the first inter-s 7A, 7B, and 7C are diagrams for explaining the ratio of the area of ​​the skeleton point region to the second seat-to-seat point distance, which varies depending on the occupant's physique. In the first embodiment, figures for explaining the first feature amount calculated by the first feature amount calculation unit, the second feature amount calculated by the second feature amount calculation unit, and another example of physique determination by the physique determination unit. In the first embodiment, figures for explaining the first seat-to-seat skeleton ... second seat-to-seat skeleton distance, which varies depending on the occupant's physique. 13A, 13B, and 13C are diagrams for explaining another example of the first feature amount calculated by the first feature amount calculation unit, the second feature amount calculated by the second feature amount calculation unit, and the physique determination by the physique determination unit in Embodiment 1. The third seat skeleton distance varies depending on the physique of the occupant.15A, 15B, and 15C are diagrams for explaining the fourth seat-to-frame distance, which varies depending on the occupant's physique. 17A, 17B, and 17C are diagrams for explaining the ratio of the area of ​​the human body region to the area of ​​the seat region, which varies depending on the occupant's physique. 17B is a flowchart for explaining the operation of the physique determination device 1 according to the first embodiment. 17C is a diagram showing an example of an occupant's physique class that needs to be determined when controlling an airbag or a seat belt, as defined in the FMVSS or the like. 17A and 17B are diagrams showing an example of the hardware configuration of the physique determination device according to the first embodiment.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Embodiment 1. In Embodiment 1, a physique determination device determines the physique of a subject based on an image captured by an imaging device that captures at least an area (hereinafter referred to as the "target area") in which the upper body of the subject whose physique is to be determined and the backrest of the seat in which the subject is seated should be located. In the following Embodiment 1, as an example, the subject is a vehicle occupant. The vehicle occupant includes the driver, a passenger seated in the front passenger seat, and a rear passenger seated in the rear seat. In the following Embodiment 1, when the term "occupant" is used simply, this refers to the vehicle occupant. Note that in the following Embodiment 1, it is assumed that the occupant is properly seated in a seat within the vehicle cabin. In Embodiment 1, "proper seating" refers to the occupant sitting deeply in the seat in a normal, upright position without any poor posture.

[0010] FIG. 1 is a diagram illustrating an example of the configuration of a physique determination device 1 according to a first embodiment. In the first embodiment, it is assumed that the physique determination device 1 is mounted on a vehicle. The physique determination device 1 is connected to an imaging device 2 mounted on the vehicle. The imaging device 2 is, for example, a near-infrared camera or a visible light camera, and captures an image of an occupant present in the vehicle cabin. The imaging device 2 may be, for example, a device commonly used with a so-called "driver monitoring system (DMS)." The imaging device 2 is installed so as to be able to capture an image of a range within the vehicle cabin that includes at least the area where the upper body of the occupant and the backrest of the seat on which the occupant is seated should be present. That is, in the first embodiment, the target range is the area where at least the upper body of the occupant and the backrest of the seat on which the occupant is seated should be present. The target range is, for example, an area corresponding to the seat back and the space near the front of the headrest. In the first embodiment, as an example, it is assumed that the imaging device 2 is installed in the center of the vehicle in the vehicle width direction. Note that in the first embodiment, the "center" in the vehicle width direction is not limited to being strictly "center" but also includes "approximately center." Specifically, it is assumed that the imaging device 2 is installed, for example, in the overhead console, center console, or near the center of the dashboard where a car navigation system or the like is provided.

[0011] The installation position of the imaging device 2 as described above is merely an example. The imaging device 2 may be installed in any position so as to be able to capture an image of the target area. However, it is assumed that the imaging device 2 captures an image of the target area from a position forward of the occupant. Also, although only one imaging device 2 is shown in FIG. 1 , this is merely an example. A vehicle may be equipped with a plurality of imaging devices 2, and the plurality of imaging devices 2 may be connected to the physique determination device 1. For example, the vehicle may be equipped with two imaging devices 2, one for capturing an image of the driver and the other for capturing an image of the passenger in the front passenger seat.

[0012] The physique determination device 1 determines the physique of an occupant based on an image of the occupant captured by the imaging device 2. In the first embodiment, the physique determined by the physique determination device 1 is one of "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small woman," "equivalent to a standard man," and "equivalent to a large man." Note that this is merely an example, and the classification of the physique determined by the physique determination device 1 can be set as appropriate. For example, the classification of the physique is set in advance (e.g., before the vehicle is shipped from the factory) by an administrator or the like.

[0013] After determining the physique of an occupant, the physique determination device 1 outputs the determination result of the occupant's physique (hereinafter referred to as the "physique determination result") to various devices connected to the physique determination device 1. The various devices are, for example, an airbag control device (not shown), a seatbelt control device (not shown), or a display device (not shown). For example, the airbag control device controls an airbag based on the physique determination result output from the physique determination device 1. For example, the seatbelt control device appropriately controls the belt force of the seatbelt, taking into account the physique of the occupant, based on the physique determination result output from the physique determination device 1. For example, the display device displays an icon indicating that an infant or child is present among the occupants.

[0014] 1 , the physique determination device 1 includes an image acquisition unit 11, a first feature amount calculation unit 12, a second feature amount calculation unit 13, a physique determination unit 14, and a determination result output unit 15. The second feature amount calculation unit 13 includes a skeleton detection unit 131.

[0015] The captured image acquisition unit 11 acquires a captured image of the occupant captured by the imaging device 2. The captured image acquisition unit 11 outputs the acquired captured image to the first feature amount calculation unit 12 and the second feature amount calculation unit 13.

[0016] The first feature amount calculation unit 12 calculates a feature amount (hereinafter referred to as a "first feature amount") relating to the contour of the seat in the captured image based on the captured image acquired by the captured image acquisition unit 11. For example, the first feature amount calculation unit 12 detects the contour of the seat in the captured image output from the captured image acquisition unit 11 and calculates a first feature amount relating to the detected contour of the seat in the captured image. The first feature amount calculation unit 12 may detect the contour of the seat in the captured image using a known technique such as edge detection. Note that this is merely an example, and the first feature amount calculation unit 12 may detect the contour of the seat in the captured image using other methods. For example, the contour of the seat in the captured image may be detected by pattern matching. In the first embodiment, the contour of the seat refers to a part or the entire outer shape of the seat. The first feature amount calculation unit 12 detects the contour of the seat using coordinates in the captured image. In addition, the first feature calculation unit 12 can identify points on the seat contour from the detected shape of the seat contour, for example, points indicating the four corners of the headrest (both ends of the upper edge and both ends of the lower edge) or both ends of the upper edge of the backrest.

[0017] For example, if an obstruction (e.g., a passenger's body) exists between the image capture device 2 and the seat contour, the first feature amount calculation unit 12 may be unable to detect the entire seat contour in the captured image. Therefore, for example, the first feature amount calculation unit 12 may have a function of estimating the undetected portion from the detected portion and interpolating the undetected portion when only a portion of the seat contour in the captured image is detected. Specifically, when the first feature amount calculation unit 12 is unable to detect the entire seat contour in the captured image and the detected seat contour is equal to or greater than a predetermined threshold (hereinafter referred to as the "first contour detection threshold"), the first feature amount calculation unit 12 interpolates the undetected portion to detect the seat contour. The first contour detection threshold is set in advance by an administrator or the like and stored in a storage unit (not shown). For example, the administrator or the like may set the first contour detection threshold to the length of the seat contour in the captured image that is estimated to be interpolatable even if a portion of the seat contour is not detected.

[0018] The first feature calculation unit 12 may use known techniques such as spline interpolation, polynomial interpolation, or linear interpolation to estimate the portions of the seat contour that could not be detected, and interpolate the edges to interpolate the portions of the seat contour that could not be detected.

[0019] Furthermore, the first feature amount calculation unit 12 may interpolate the portion of the seat contour that could not be detected using, for example, a trained model in machine learning (hereinafter referred to as a “machine learning model”). The machine learning model (herein referred to as a “contour detection model”) is a model that receives a captured image as input and outputs information indicating the contour of the seat in the captured image. The machine learning model is generated in advance by an administrator or the like and stored in a storage unit (not shown). The administrator or the like may perform training in advance using, for example, captured images of the interior of the vehicle when no occupants are in the vehicle as training data to generate the contour detection model and store the model in a storage unit (not shown). The first feature amount calculation unit 12 acquires the contour detection model from the storage unit, inputs the captured image acquired by the captured image acquisition unit 11 into the acquired contour detection model, and obtains information indicating the contour of the seat in the captured image, thereby estimating the portion of the seat contour that could not be detected and interpolating the portion of the seat contour that could not be detected.

[0020] Furthermore, the first feature calculation unit 12 may, for example, compare information (hereinafter referred to as "seat position contour correspondence information") that associates all seat positions with the contours of seats on the captured image, which information is generated in advance by an administrator or the like and stored in a memory unit (not shown), with the partial seat contours that can be detected from the captured image acquired by the captured image acquisition unit 11, identify the contours of seats that are close to the partial seat contours that can be detected from the seat position contour correspondence information, estimate the parts of the seat contours that could not be detected, and interpolate the parts of the seat contours that could not be detected.

[0021] When the first feature amount calculation unit 12 detects the contour of the seat in the captured image output from the captured image acquisition unit 11, it calculates a first feature amount on the captured image related to the detected contour of the seat. More specifically, the first feature amount calculation unit 12 calculates the first feature amount in accordance with the first feature amount calculation conditions included in the physique determination conditions. Details of the physique determination conditions and the first feature amount will be described later with specific examples. The first feature amount calculation unit 12 outputs the calculated first feature amount to the physique determination unit 14.

[0022] Here, the first feature amount calculation unit 12 detects the contour of the seat in the captured image output from the captured image acquisition unit 11 and then calculates the first feature amount related to the detected contour of the seat in the captured image. However, this is merely an example. For example, the first feature amount calculation unit 12 may calculate the first feature amount related to the contour of the seat in the captured image without detecting the contour of the seat in the captured image. For example, if the position of the seat when the occupant sits is fixed, the positional relationship between the image capture device 2 and the seat in which the subject sits is fixed, and the position of the contour of the seat in the captured image is also fixed. For example, information indicating the contour of the seat in the captured image (hereinafter referred to as "image contour information") may be generated in advance by an administrator or the like and stored in a storage unit (not shown). The first feature amount calculation unit 12 may calculate the first feature amount related to the contour of the seat in the captured image output from the captured image acquisition unit 11 from the image contour information. In this case, the first feature amount calculation unit 12 can calculate the first feature amount without performing a process of detecting the contour of the seat. Furthermore, for example, it is assumed that the seat position contour correspondence information as described above is stored in a storage unit (not shown), and the first feature amount calculation unit 12 acquires seat information indicating the position of the seat where the subject is seated from a seat sensor (not shown), and calculates, from the seat information and the seat position contour correspondence information, a first feature amount on the captured image related to the contour of the seat in the captured image output from the captured image acquisition unit 11. In this case as well, the first feature amount calculation unit 12 can calculate the first feature amount without performing a process of detecting the contour of the seat.

[0023] The second feature amount calculation unit 13 calculates feature amounts (hereinafter referred to as "second feature amounts") related to the occupant in the captured image based on the captured image acquired by the captured image acquisition unit 11. Specifically, the skeleton detection unit 131 of the second feature amount calculation unit 13 detects multiple skeleton coordinate points of the occupant that indicate body parts of the occupant in the captured image. Then, the second feature amount calculation unit 13 calculates the second feature amount based on the multiple skeleton coordinate points detected by the skeleton detection unit 131. More specifically, the second feature amount calculation unit 13 calculates the second feature amount in accordance with second feature amount calculation conditions included in the physique determination conditions. More specifically, the second feature amount calculation unit 13 calculates the second feature amount in accordance with the physique determination conditions. Details of the physique determination conditions and the second feature amount will be described later with specific examples.

[0024] Here, the skeleton detection unit 131 will be described in detail. As described above, the skeleton detection unit 131 detects a plurality of skeleton coordinate points of the occupant, which indicate parts of the occupant's body, based on the captured image acquired by the captured image acquisition unit 11. More specifically, the skeleton detection unit 131 detects the occupant's skeleton coordinate points, which indicate joint points determined for each part of the occupant's body, based on the captured image acquired by the captured image acquisition unit 11. The skeleton detection unit 131 detects the coordinates of the occupant's skeleton coordinate points and which parts of the occupant's body the skeleton coordinate points indicate. The skeleton coordinate points are points in the captured image and are represented by coordinates in the captured image. Examples of joint points determined for each body part of the occupant include nose joint points, neck joint points, shoulder (right shoulder and left shoulder) joint points, elbow (right elbow and left elbow) joint points, waist (right hip and left hip) joint points, groin (right groin and left groin) joint points, wrist joint points, knee joint points, ankle joint points, etc. The skeletal coordinate points indicating which body parts correspond to which joint points are to be detected by the skeletal detection unit 131 are determined in advance by an administrator or the like, and information indicating the joint points to be detected as skeletal coordinate points is held in the skeletal detection unit 131.

[0025] For example, the skeleton detection unit 131 detects the skeletal coordinate points of the occupant using a machine learning model (herein referred to as a "skeletal point detection model") that receives an image of the occupant as input and outputs information about the skeletal coordinate points in the image. The information indicating the skeletal coordinate points includes the coordinates of the skeletal coordinate points in the image and information that can identify which part of the body the skeletal coordinate points indicate. Note that this is just one example, and the skeleton detection unit 131 may detect the skeletal coordinate points of the occupant using various known image processing techniques.

[0026] Furthermore, when detecting a skeleton coordinate point, the skeleton detection unit 131 can also detect which occupant the detected skeleton coordinate point belongs to. In the first embodiment, the skeleton detection unit 131 determines the occupant based on their seating position. The skeleton detection unit 131 does not need to perform personal authentication of the occupant. For example, in a captured image, an area corresponding to each seat (hereinafter referred to as a "seat-corresponding area") is set in advance. The seat-corresponding area is set in advance according to the installation position and angle of view of the imaging device 2. The skeleton detection unit 131 determines which seat the occupant in whose seat the skeleton coordinate point belongs, depending on whether or not any of the detected skeleton coordinate points is included in the seat-corresponding area. To give a specific example, if any of the detected skeleton coordinate points is included in the seat-corresponding area corresponding to the driver's seat, the skeleton detection unit 131 determines that the driver is sitting in the driver's seat. The skeleton detection unit 131 then determines that the skeleton coordinate points detected in the seat corresponding area corresponding to the driver's seat are the skeleton coordinate points of the driver. Furthermore, for example, if the skeleton detection unit 131 detects any skeleton coordinate points included in the seat corresponding area corresponding to the passenger seat, it determines that a passenger is seated in the passenger seat. The skeleton detection unit 131 then determines that the skeleton coordinate points detected in the seat corresponding area corresponding to the passenger seat are the skeleton coordinate points of the passenger.

[0027] The skeleton detection unit 131 outputs information about the detected multiple skeleton coordinate points (hereinafter referred to as "skeleton coordinate point information") to the second feature calculation unit 13. For example, in the skeleton coordinate point information, for each skeleton coordinate point, the information about the skeleton coordinate point, information indicating which body part the skeleton coordinate point represents, and information identifying the occupant are associated. The information about the skeleton coordinate point is specifically the coordinate of the skeleton coordinate point on the captured image. The information identifying the occupant may be, for example, information about the seat in which the occupant is seated. Then, the second feature calculation unit 13 calculates the second feature based on the multiple skeleton coordinate points detected by the skeleton detection unit 131.

[0028] After calculating the second feature amount based on the captured image acquired by the captured image acquisition unit 11 , the second feature amount calculation unit 13 outputs the calculated second feature amount to the physique determination unit 14 .

[0029] The physique determination unit 14 determines the physique of the occupant based on the first feature calculated by the first feature calculation unit 12 and the second feature calculated by the second feature calculation unit 13. More specifically, for example, the physique determination unit 14 determines the physique of the occupant based on the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination conditions. In the first embodiment, the physique determination conditions are conditions for determining the physique classification of the occupant based on the relationship between the first feature and the second feature. The physique determination conditions are set in advance, for example, by an administrator or the like. The administrator or the like stores information indicating the set physique determination conditions (hereinafter referred to as "physique determination condition information") in a storage unit (not shown). The physique determination conditions may be updated, for example, at appropriate times. The physique determination conditions include a judgment criterion condition, a first feature quantity calculation condition, and a second feature quantity calculation condition. That is, the physique determination condition information includes information indicating the judgment criterion condition (hereinafter referred to as "judgment criterion condition information"), information indicating the first feature quantity calculation condition (hereinafter referred to as "first feature quantity calculation condition information"), and information indicating the second feature quantity calculation condition (hereinafter referred to as "second feature quantity calculation condition information"). The judgment criterion condition is a condition that defines the relationship between the first feature quantity, the second feature quantity, and the physique. The first feature quantity calculation condition is a condition that defines how to calculate the first feature quantity. The second feature quantity calculation condition is a condition that defines how to calculate the second feature quantity. Note that if the calculation methods for the first feature quantity and the second feature quantity are uniquely determined, it is not essential that the first feature quantity calculation condition and the second feature quantity calculation condition be set in the physique determination conditions. A specific example of the occupant physique determination method performed by the physique determination unit 14 based on the first feature amount, the second feature amount, and the physique determination conditions will be described below.

[0030] <Physique Determination Method Example (1)> For example, the physique determination conditions define the ratio of a second feature value to a first feature value and the physique corresponding to the ratio, the first feature value calculation unit 12 calculates the area of ​​a region surrounded by the outline of the seat in the captured image (hereinafter referred to as the "seat region") as the first feature value, the second feature value calculation unit 13 calculates the area of ​​a region formed by three or more skeleton coordinate points (hereinafter referred to as the "skeleton point region") among the multiple skeleton coordinate points detected by the skeleton detection unit 131 as the second feature value, and the physique determination unit 14 determines the occupant's physique from the ratio of the area of ​​the skeleton point region to the area of ​​the seat region. Note that the "skeleton point region formed by three or more skeleton coordinate points" refers to a region having three or more skeleton coordinate points as vertices.

[0031] In this case, the physique determination conditions define "the ratio of the second feature amount to the first feature amount, and the physique corresponding to that ratio" in the determination criteria conditions. For example, the physique determination criteria conditions correspond to a range of the ratio of the second feature amount to the first feature amount, and a physique, in this case, "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small woman," "equivalent to a standard man," or "equivalent to a large man."

[0032] In addition, in the conditions for determining physical build, the first characteristic amount calculation condition defines a condition that "the area of ​​the seat area in the captured image is used as the first characteristic amount."

[0033] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "Three or more skeletal coordinate points from among the multiple skeletal coordinate points are selected as skeletal coordinate points for forming a skeletal point region (hereinafter referred to as "region-forming skeletal coordinate points"), and the area of ​​the skeletal point region formed by the region-forming skeletal coordinate points in the captured image is used as the second feature. The skeletal coordinate points to be used as the region-forming skeletal coordinate points are selected in accordance with the region-forming skeletal point selection conditions." In the first embodiment, the region-forming skeletal point selection conditions are conditions that determine which skeletal coordinate points are to be used as the region-forming skeletal coordinate points. The region-forming skeletal point selection conditions are set in advance by an administrator or the like, and information indicating the region-forming skeletal point selection conditions (hereinafter referred to as "region-forming skeletal point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines that three or more appropriate skeletal coordinate points from the skeletal coordinate points detected by the skeleton detection unit 131 are to be used as the region-forming skeletal coordinate points. However, it is preferable that the region-forming skeletal coordinate points correspond to body parts that move less. Area-forming skeleton coordinate points with little movement are, for example, the skeleton coordinate points of both shoulders and the base of both feet. For example, the area-forming skeleton point selection conditions may be set to a condition that all skeleton coordinate points detected by the skeleton detection unit 131 are to be area-forming skeleton coordinate points. The area-forming skeleton point selection conditions may also be updated as appropriate. The area-forming skeleton point selection conditions may be included in the second feature amount calculation conditions. Here, as an example, it is assumed that the area-forming skeleton point selection conditions are set to a condition that the four skeleton coordinate points of both shoulders and the base of both feet are to be area-forming skeleton coordinate points.

[0034] FIG. 2 is a diagram illustrating the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (1) in the first embodiment. Here, the vehicle is assumed to be a right-hand drive vehicle, and for ease of explanation, FIG. 2 illustrates only the outline of the passenger seat and the passenger seat occupant seated in the passenger seat from the captured image captured by the imaging device 2. The physique determination unit 14 determines the physique of the passenger seat occupant as shown in FIG. 2, but for convenience, the passenger seat occupant will be simply referred to as the "occupant." In FIG. 2, "C1" indicates the occupant, and "S" indicates the outline of the seat. Also, in FIG. 2, "P11a" indicates the skeletal coordinate point of the right shoulder, "P12a" indicates the skeletal coordinate point of the left shoulder, "P13a" indicates the skeletal coordinate point of the base of the right foot, and "P14a" indicates the skeletal coordinate point of the base of the left foot.

[0035] When the physique determination conditions are as described above, the first feature amount calculation unit 12 calculates the area of ​​the seat area as the first feature amount. In FIG. 2 , the first feature amount calculation unit 12 calculates the area of ​​the area indicated by "201a" as the area of ​​the seat area, i.e., the first feature amount. The first feature amount calculation unit 12 may calculate the area of ​​the seat area using a known polygonal area calculation technique, such as a method using a point inclusion determination algorithm or a method using a geometric formula.

[0036] The second feature amount calculation unit 13 selects area-forming skeleton coordinate points from among the multiple skeleton coordinate points detected by the skeleton detection unit 131. Here, the second feature amount calculation unit 13 selects four skeleton coordinate points, one for each shoulder and one for each hip, as area-forming skeleton coordinate points from among the multiple skeleton coordinate points detected by the skeleton detection unit 131. The second feature amount calculation unit 13 then calculates the area of ​​the skeleton point area formed by the four skeleton coordinate points, one for each shoulder and one for each hip, as the second feature amount. In FIG. 2 , the second feature amount calculation unit 13 selects the skeletal coordinate point of the right shoulder indicated by "P11a," the skeletal coordinate point of the left shoulder indicated by "P12a," the skeletal coordinate point of the right hip indicated by "P13a," and the skeletal coordinate point of the left hip indicated by "P14a" as area-forming skeleton coordinate points, and calculates the area of ​​the area indicated by "301a" as the area of ​​the skeleton point area, i.e., the second feature amount. The second feature calculation unit 13 may calculate the area of ​​the skeleton point region using a known polygon area calculation technique, such as a method using a point inclusion determination algorithm or a method using a geometric formula.

[0037] The physique determination unit 14 calculates the ratio of the area of ​​the skeleton point area (area indicated by "301a" in FIG. 2) calculated by the second feature amount calculation unit 13 to the area of ​​the seat area (area indicated by "201a" in FIG. 2) calculated by the first feature amount calculation unit 12. Then, the physique determination unit 14 determines the physique of the occupant by comparing the calculated ratio with the determination criteria conditions. For example, if the area of ​​the seat area calculated by the first feature amount calculation unit 12 is 1000 pixels, ２ , the area of ​​the skeleton point region calculated by the second feature amount calculation unit 13 is 500 pixels ２In this case, the physique determination unit 14 calculates the ratio of the area of ​​the skeleton point region to the area of ​​the seat region as 500 / 1000 = 0.5. Then, the physique determination unit 14 compares the ratio "0.5" with the determination criterion condition. For example, in the determination criterion condition, a physique of "average male" is associated with a range of "0.4 to 0.7" for the ratio of the second feature amount to the first feature amount. In this case, the physique determination unit 14 determines that the occupant's physique is "average male." Note that in the first embodiment, the length of a line on a captured image is acquired as a distance based on, for example, the spacing between pixels.

[0038] The area of ​​the skeleton point region of the occupant increases as the occupant's physique increases. On the other hand, the area of ​​the seat region does not change depending on the occupant's physique. Therefore, the larger the occupant's physique, the greater the ratio of the area of ​​the skeleton point region to the area of ​​the seat region.

[0039] 3A, 3B, and 3C are diagrams illustrating the ratio of the area of ​​the skeleton point region to the area of ​​the seat region, which varies depending on the occupant's physique. FIG. 3A is the same as FIG. 2. Here, the occupant indicated by "C1" is assumed to be an adult male with a physique equivalent to a "standard male." FIG. 3B illustrates a seat having a contour indicated by "S" in FIG. 3A, in place of the occupant indicated by "C1," in which an adult female occupant with a physique equivalent to a "petite woman" is seated. In FIG. 3B, the occupant is indicated by "C2." In FIG. 3B, "P11b," "P12b," "P13b," and "P14b" respectively indicate the skeletal coordinate points of the right shoulder, left shoulder, right hip, and left hip of the occupant indicated by "C2." In addition, in FIG. 3B, "301b" indicates the skeleton point region of the occupant indicated by "C2." FIG. 3C shows a child occupant with a physique equivalent to a 6-year-old child sitting in the seat having the contour indicated by "S" in FIG. 3A, instead of the occupant indicated by "C1." In FIG. 3C, this occupant is indicated by "C3." In FIG. 3C, "P11c," "P12c," "P13c," and "P14c" indicate the skeletal coordinate points of the right shoulder, left shoulder, right hip, and left hip of the occupant indicated by "C3," respectively. In addition, in FIG. 3C, "301c" indicates the skeleton point region of the occupant indicated by "C3."

[0040] As shown in Figures 3A, 3B, and 3C, the area of ​​the skeleton point region of the occupant increases as the occupant's physique increases. Specifically, the area of ​​the skeleton point region indicated by "301b" is larger than the area of ​​the skeleton point region indicated by "301c," and the area of ​​the skeleton point region indicated by "301a" is larger than the area of ​​the skeleton point region indicated by "301b." On the other hand, the size of the area of ​​the seat region indicated by "201a" does not change depending on the occupant's physique. In other words, the larger the occupant's physique, the larger the ratio of the area of ​​the skeleton point region to the area of ​​the seat region. If the seat position is moved forward / backward or up / down, or if the reclining angle of the seat is changed, the positional relationship between the imaging device 2 and the seat changes, and the seat may appear larger or smaller in the captured image. Note that in Embodiment 1, the "position of the seat" includes the position of the seat back, in other words, the reclining angle. Even in such a case, the ratio of the area of ​​the skeleton point region to the area of ​​the seat region remains unchanged because the occupant sitting in the seat also moves as the seat moves. Note that "remains unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the occupant's physique from the ratio between the first feature amount related to the seat contour and the second feature amount related to the occupant, a relationship which is maintained at a constant rate regardless of the seat position, thereby enabling physique determination that is not affected by the position of the seat in which the occupant is sitting.

[0041] <Example of Physical Build Determination Method (2)> Furthermore, for example, the conditions for physical build determination may define a ratio of a second feature value to a first feature value and a physical build corresponding to the ratio, the first feature value calculation unit 12 may calculate the distance between two points on the contour of the seat in the captured image (hereinafter referred to as the "seat point distance") as the first feature value, the second feature value calculation unit 13 may calculate the distance between two skeleton coordinate points (hereinafter referred to as the "skeleton point distance") among the multiple skeleton coordinate points detected by the skeleton detection unit 131 as the second feature value, and the physical build determination unit 14 may determine the occupant's physical build from the ratio of the skeleton point distance to the seat point distance.

[0042] In this case, the physique determination conditions define the "ratio of the second feature amount to the first feature amount, and the physique for that ratio" in the determination criteria conditions. For example, the physique determination criteria conditions correspond to a range of the ratio of the second feature amount to the first feature amount, and the physique, in this case, "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small woman," "equivalent to a standard man," or "equivalent to a large man."

[0043] Furthermore, in the physical build determination conditions, the first feature amount calculation conditions define the following condition: "Two points (hereinafter referred to as "seat points") on the contour of the seat in the captured image are selected as representative seat points (hereinafter referred to as "first representative seat points"), and the seat point distance between the two first representative seat points (hereinafter referred to as "first seat point distance") is set as the first feature amount. The two seat points to be used as the first representative seat points are selected in accordance with the first representative seat point selection conditions." In the first embodiment, the first representative seat point selection conditions are conditions that determine which seat points are to be used as the first representative seat points. The first representative seat point selection conditions are set in advance by an administrator or the like, and information indicating the first representative seat point selection conditions (hereinafter referred to as "first representative seat point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines two appropriate seat points from among the seat points on the contour of the seat to be used as the first representative seat points. For example, the first representative seat point selection condition may be set to select any two points on the contour of the seat as the first representative seat points. The first representative seat point selection condition may be updated as appropriate. The first representative seat point selection condition may be included in the first feature quantity calculation condition. Here, as an example, it is assumed that the first representative seat point selection condition is set to select the seat points corresponding to both ends of the upper edge of the headrest as the first representative seat points.

[0044] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "Two of the multiple skeletal coordinate points are selected as representative skeletal coordinate points (hereinafter referred to as "first representative skeletal coordinate points"), and the skeletal point-to-skeletal point distance (hereinafter referred to as "first inter-skeletal point distance") between the two first representative skeletal coordinate points in the captured image is set as the second feature. The skeletal coordinate points to be used as the first representative skeletal coordinate points are selected in accordance with the first representative skeletal point selection conditions." In the first embodiment, the first representative skeletal point selection conditions are conditions that determine which skeletal coordinate points are to be used as the first representative skeletal coordinate points. The first representative skeletal point selection conditions are set in advance by an administrator or the like, and information indicating the first representative skeletal point selection conditions (hereinafter referred to as "first representative skeletal point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines which two appropriate skeletal coordinate points from the skeletal coordinate points detected by the skeleton detection unit 131 are to be used as the first representative skeletal coordinate points. However, it is desirable that the first representative skeleton coordinate point be a skeleton coordinate point corresponding to a body part that moves little. Here, as an example, it is assumed that the first representative skeleton point selection condition specifies that the two skeleton coordinate points on both shoulders are to be the first representative skeleton coordinate points.

[0045] 4 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (2) in the first embodiment. In Fig. 4, the same reference numerals are used for the points and the like that are described with reference numerals in Fig. 2, and redundant description will be omitted. In Fig. 4, "P21a" indicates the seat point at the right end of the upper side of the headrest, and "P22a" indicates the seat point at the left end of the upper side of the headrest.

[0046] When the physique determination conditions are as described above, the first feature amount calculation unit 12 selects first representative seat points from among the seat points on the seat contour. Here, the first feature amount calculation unit 12 selects the seat points corresponding to both ends of the upper edge of the headrest as the first representative seat points. The first feature amount calculation unit 12 then calculates the first inter-seat point distance between the two first representative seat points as the first feature amount. In FIG. 4 , the first feature amount calculation unit 12 calculates the distance indicated by "202a" as the first inter-seat point distance, i.e., the first feature amount.

[0047] The second feature amount calculation unit 13 selects a first representative skeleton coordinate point from among the multiple skeleton coordinate points detected by the skeleton detection unit 131. Here, the second feature amount calculation unit 13 selects the skeleton coordinate points of both shoulders as first representative skeleton coordinate points from among the multiple skeleton coordinate points detected by the skeleton detection unit 131. Then, the second feature amount calculation unit 13 calculates the first inter-skeleton point distance between the skeleton coordinate points of both shoulders as the second feature amount. In FIG. 2 , the second feature amount calculation unit 13 calculates the distance indicated by "302a" as the first inter-skeleton point distance, i.e., the second feature amount.

[0048] The physique determination unit 14 calculates a ratio of the first inter-skeleton point distance (distance indicated by "302a" in FIG. 4 ) calculated by the second feature amount calculation unit 13 to the first inter-seat point distance (distance indicated by "202a" in FIG. 4 ) calculated by the first feature amount calculation unit 12. The physique determination unit 14 then determines the occupant's physique by comparing the calculated ratio with the determination criterion condition. For example, assume that the first inter-seat point distance calculated by the first feature amount calculation unit 12 is 80 pix and the first inter-skeleton point distance calculated by the second feature amount calculation unit 13 is 160 pix. In this case, the physique determination unit 14 calculates the ratio of the first inter-skeleton point distance to the first inter-seat point distance as 160 / 80=2. The physique determination unit 14 then compares the ratio "2" with the determination criterion condition. For example, if the range of the ratio of the second feature amount to the first feature amount is set to 1.8 to 2.2, the physique determination unit 14 determines that the occupant's physique is equivalent to that of a standard male.

[0049] The first inter-skeleton point distance increases as the occupant's physique increases. On the other hand, the first inter-seat point distance does not change depending on the occupant's physique. Therefore, the larger the occupant's physique, the greater the ratio of the first inter-skeleton point distance to the first inter-seat point distance.

[0050] Here, Figures 5A, 5B, and 5C are diagrams for explaining the ratio of the first inter-skeleton point distance to the first inter-seat point distance, which changes depending on the occupant's physique. Figure 5A is the same as Figure 4. Here, the occupant indicated by "C1" is assumed to be an adult male with a physique equivalent to that of a "standard male." Figure 5B is a diagram showing an occupant indicated by "C2" sitting in place of the occupant indicated by "C1" in a seat having an outline indicated by "S" in Figure 5A. The occupant indicated by "C2" in Figure 5B is the same occupant as the occupant indicated by "C2" in Figure 3B. In Figure 5B, "302b" indicates the first inter-skeleton point distance between the first representative skeleton coordinate point indicated by "P11b" (skeleton coordinate point of the right shoulder) and the first representative skeleton coordinate point indicated by "P12b" (skeleton coordinate point of the left shoulder). Figure 5C shows an occupant indicated by "C3" seated in place of the occupant indicated by "C1" in the seat having the contour indicated by "S" in Figure 5A. The occupant indicated by "C3" in Figure 5C is the same occupant as the occupant indicated by "C3" in Figure 3C. In Figure 5C, "302c" indicates the first inter-skeleton point distance between the first representative skeleton coordinate point indicated by "P11c" (skeleton coordinate point of the right shoulder) and the first representative skeleton coordinate point indicated by "P12c" (skeleton coordinate point of the left shoulder).

[0051] As shown in FIGS. 5A, 5B, and 5C, the first inter-skeleton point distance increases as the occupant's physical size increases. Specifically, the first inter-skeleton point distance indicated by "302b" is greater than the first inter-skeleton point distance indicated by "302c," and the first inter-skeleton point distance indicated by "302a" is greater than the first inter-skeleton point distance indicated by "302b." Meanwhile, the magnitude of the first inter-seat point distance indicated by "202a" does not change depending on the occupant's physical size. That is, the larger the occupant's physical size, the greater the ratio of the first inter-seat point distance to the first inter-seat point distance. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the image capture device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such cases, the occupant sitting in the seat also moves as the seat moves, so the ratio of the first inter-skeleton point distance to the first inter-seat point distance remains unchanged. The term "unchanged" here does not necessarily mean being exactly the same, but includes being approximately the same. The physique determination unit 14 determines the physique of the occupant from the ratio between the first feature amount related to the contour of the seat and the second feature amount related to the occupant, the relationship of which is maintained at a constant rate regardless of the position of the seat, thereby making it possible to perform a physique determination that is not affected by the position of the seat in which the occupant is seated.

[0052] <Physique Determination Method Example (3)> For example, the physique determination conditions may define a ratio of a second feature value to a first feature value and a physique corresponding to the ratio, the first feature value calculation unit 12 may calculate the seat point distance between two points on the contour of the seat in the captured image as the first feature value, the second feature value calculation unit 13 may calculate the area of ​​a skeleton point region as the second feature value, and the physique determination unit 14 may determine the occupant's physique from the ratio of the skeleton point region to the seat point distance. The skeleton point region has already been described, so a redundant description will be omitted.

[0053] In this case, the physique determination conditions define the "ratio of the second feature amount to the first feature amount, and the physique for that ratio" in the determination criteria conditions. For example, the physique determination criteria conditions correspond to a range of the ratio of the second feature amount to the first feature amount, and the physique, in this case, "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small woman," "equivalent to a standard man," or "equivalent to a large man."

[0054] Furthermore, in the physical build determination conditions, the first feature amount calculation conditions define the following condition: "Two seat points on the contour of the seat in the captured image are selected as representative seat points (hereinafter referred to as "second representative seat points"), and the seat point distance between the two second representative seat points (hereinafter referred to as "second seat point distance") is set as the first feature amount. The two seat points to be used as the second representative seat points are selected in accordance with the second representative seat point selection conditions." In the first embodiment, the second representative seat point selection conditions are conditions that set which seat points are to be used as the second representative seat points. The second representative seat point selection conditions are set in advance by an administrator or the like, and information indicating the second representative seat point selection conditions (hereinafter referred to as "second representative seat point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines two appropriate seat points from among the seat points on the contour of the seat to be used as the second representative seat points. For example, the second representative seat point selection condition may be set to select any two points on the contour of the seat as the second representative seat points. The second representative seat point selection condition may be updated as appropriate. The second representative seat point selection condition may be included in the first feature quantity calculation condition. Here, as an example, it is assumed that the second representative seat point selection condition is set to select the seat points corresponding to both ends of the upper edge of the seat back as the second representative seat points.

[0055] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "Three or more skeletal coordinate points from among the multiple skeletal coordinate points are selected as region-forming skeletal coordinate points, and the area of ​​the skeletal point region formed by the region-forming skeletal coordinate points in the captured image is taken as the second feature. The skeletal coordinate points to be used as the region-forming skeletal coordinate points are selected in accordance with the region-forming skeletal point selection conditions." Here, as an example, it is assumed that the region-forming skeletal point selection conditions set a condition that the four skeletal coordinate points at both shoulders and the base of both feet are used as the region-forming skeletal coordinate points.

[0056] 6 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (3) in the first embodiment. In Fig. 6, the same reference numerals are used for the points and the like that are described with reference numerals in Fig. 2, and redundant description will be omitted. In Fig. 6, "P23a" indicates the rightmost seat point on the upper side of the backrest, and "P24a" indicates the leftmost seat point on the upper side of the backrest.

[0057] When the physique determination conditions are as described above, the first feature amount calculation unit 12 selects second representative seat points from among the seat points on the seat contour. Here, the first feature amount calculation unit 12 selects the seat points corresponding to both ends of the upper edge of the seat back as the second representative seat points. The first feature amount calculation unit 12 then calculates the second inter-seat point distance between the two second representative seat points as the first feature amount. In FIG. 6 , the first feature amount calculation unit 12 calculates the distance indicated by "203a" as the second inter-seat point distance, i.e., the first feature amount.

[0058] The second feature amount calculation unit 13 selects the four skeletal coordinate points of both shoulders and the base of both feet as region-forming skeletal coordinate points, and calculates the area of ​​a skeleton point region formed by the four region-forming skeletal coordinate points of both shoulders and the base of both feet as the second feature amount. In Fig. 6, the second feature amount calculation unit 13 calculates the area of ​​the region indicated by "301a" as the area of ​​the skeleton point region, i.e., the second feature amount.

[0059] The physique determination unit 14 calculates the ratio of the area of ​​the skeleton point region (the region indicated by "301a" in FIG. 6) calculated by the second feature amount calculation unit 13 to the second inter-seat point distance (the distance indicated by "203a" in FIG. 6) calculated by the first feature amount calculation unit 12. Then, the physique determination unit 14 determines the physique of the occupant by comparing the calculated ratio with the determination criterion conditions. For example, if the second inter-seat point distance calculated by the first feature amount calculation unit 12 is 100 pixels and the area of ​​the skeleton point region calculated by the second feature amount calculation unit 13 is 500 pixels, the physique determination unit 14 determines the physique of the occupant. ２ In this case, the physique determination unit 14 calculates the ratio of the area of ​​the skeleton point region to the second inter-seat point distance as 500 / 100=5. Then, the physique determination unit 14 compares the ratio "5" with the determination criterion condition. For example, in the determination criterion condition, the range of the ratio of the second feature amount to the first feature amount, "4.5 to 6," is associated with a physique of "average male." In this case, the physique determination unit 14 determines that the occupant's physique is "average male."

[0060] The area of ​​the skeleton point region increases as the occupant's physique increases. On the other hand, the second seat point distance does not change depending on the occupant's physique. Therefore, the ratio of the area of ​​the skeleton point region to the second seat point distance increases as the occupant's physique increases.

[0061] Here, Figures 7A, 7B, and 7C are diagrams for explaining the ratio of the area of ​​the skeleton point region to the second seat point distance, which changes depending on the occupant's physique. Figure 7A is the same as Figure 6. Here, the occupant designated "C1" is assumed to be an adult male with a physique equivalent to that of a standard male. Figure 7B is a diagram showing an occupant designated "C2" sitting in place of the occupant designated "C1" on a seat having a contour designated "S" in Figure 7A. The occupant designated "C2" in Figure 7B is the same occupant designated "C2" as the occupant designated "C2" in Figure 3B. Figure 7C is a diagram showing an occupant designated "C3" sitting in place of the occupant designated "C1" on a seat having a contour designated "S" in Figure 7A. The occupant designated "C3" in Figure 7C is the same occupant designated "C3" as the occupant designated "C1" in Figure 3C.

[0062] As shown in Figures 7A, 7B, and 7C, the area of ​​the skeleton point region increases as the occupant's physique increases. Meanwhile, the magnitude of the second seat-point distance indicated by "203a" does not change depending on the occupant's physique. That is, the larger the occupant's physique, the greater the ratio of the area of ​​the skeleton point region to the second seat-point distance. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the imaging device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such cases, the occupant seated in the seat also moves as the seat moves, so the ratio of the area of ​​the skeleton point region to the second seat-point distance does not change. Note that "unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the physique of the occupant from the ratio between a first feature value related to the contour of the seat and a second feature value related to the occupant, the relationship of which is maintained at a constant rate regardless of the position of the seat, thereby making it possible to perform a physique determination that is not affected by the position of the seat in which the occupant is seated.

[0063] <Example of Physical Build Determination Method (4)> Furthermore, for example, the physical build determination conditions may define the distance between a point indicated by a first feature amount and a point indicated by a second feature amount, and the physical build corresponding to the distance, the first feature amount calculation unit 12 may calculate, as the first feature amount, the coordinates of a seat point that is a point on the contour of the seat in the captured image, the second feature amount calculation unit 13 may calculate, as the second feature amount, the coordinates of a certain skeleton coordinate point among the multiple skeleton coordinate points detected by the skeleton detection unit 131, and the physical build determination unit 14 may determine the physical build of the occupant from the distance between the point indicated by the first feature amount and the point indicated by the second feature amount (hereinafter referred to as the “first seat skeleton distance”).

[0064] In this case, the physique determination conditions include a definition of the first seat-skeleton distance between the point indicated by the first feature amount and the point indicated by the second feature amount, and a physique corresponding to the first seat-skeleton distance. For example, the physique determination conditions include a correspondence between the first seat-skeleton distance and a physique, such as "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small female," "equivalent to a standard male," or "equivalent to a large male."

[0065] Furthermore, in the physical build determination conditions, the first feature amount calculation condition defines the following condition: "A representative seat point on the contour of the seat in the captured image (hereinafter referred to as the "third representative seat point") is selected, and the coordinates of the third representative seat point are calculated as the first feature amount. The seat point to be used as the third representative seat point is selected in accordance with the third representative seat point selection condition." In the first embodiment, the third representative seat point selection condition is a condition that determines which seat point is to be used as the third representative seat point. The third representative seat point selection condition is set in advance by an administrator or the like, and information indicating the third representative seat point selection condition (hereinafter referred to as the "third representative seat point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate seat point from among the seat points on the contour of the seat to be used as the third representative seat point. For example, the third representative seat point selection condition may be set to determine an arbitrary point on the contour of the seat as the third representative seat point. The third representative seat point selection condition may also be updated as appropriate. The third representative seat point selection condition may be included in the first feature amount calculation condition. Here, as an example, it is assumed that the third representative seat point selection condition specifies that the seat point corresponding to the right end of the upper side of the backrest is to be the third representative seat point.

[0066] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "A representative skeletal coordinate point (hereinafter referred to as the "second representative skeletal coordinate point") is selected from among a plurality of skeletal coordinate points, and the coordinates of the second representative skeletal coordinate point are calculated as the first feature. The skeletal coordinate point to be used as the second representative skeletal coordinate point is selected in accordance with the second representative skeletal point selection conditions." In the first embodiment, the second representative skeletal point selection conditions are conditions that determine which skeletal coordinate point is to be used as the second representative skeletal coordinate point. The second representative skeletal point selection conditions are set in advance by an administrator or the like, and information indicating the second representative skeletal point selection conditions (hereinafter referred to as the "second representative skeletal point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate skeletal coordinate point from among the skeletal coordinate points detected by the skeleton detection unit 131 to be used as the second representative skeletal coordinate point. However, it is desirable that the second representative skeletal coordinate point be a skeletal coordinate point corresponding to a body part that moves less. For example, when the skeleton detection unit 131 detects only a certain skeleton coordinate point, the second representative skeleton point selection condition may be set to a condition that the skeleton coordinate point detected by the skeleton detection unit 131 is to be the second representative skeleton coordinate point. The second representative skeleton point selection condition may be updated as appropriate. The second representative skeleton point selection condition may be included in the second feature amount calculation condition. Here, as an example, it is assumed that the second representative skeleton point selection condition is set to a condition that the skeleton coordinate point of the right shoulder is to be the second representative skeleton coordinate point.

[0067] 8 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (4) in the first embodiment. In Fig. 8, the same reference numerals are used for the points and the like that are described with reference numerals in Fig. 2, and redundant description will be omitted. In Fig. 8, "P23a" indicates the rightmost seat point on the upper side of the backrest.

[0068] When the physique determination conditions are as described above, the first feature amount calculation unit 12 selects a third representative seat point from among the seat points on the seat contour. Here, the first feature amount calculation unit 12 selects the seat point corresponding to the right end of the upper edge of the seat back as the third representative seat point. The first feature amount calculation unit 12 then calculates the coordinates of the third representative seat point as the first feature amount. The first feature amount calculation unit 12 detects the seat contour using coordinates on the captured image. That is, (x, y) coordinates are assigned to each point on the seat contour. The first feature amount calculation unit 12 may calculate the coordinates of the third representative seat point as the first feature amount by identifying the coordinates of the point on the detected seat contour that corresponds to the third representative seat point. In FIG. 8, the first feature amount calculation unit 12 sets the seat point "P23a" as the third representative seat point, and calculates the coordinates of the seat point "P23a" as the coordinates of the third representative seat point, i.e., the first feature amount.

[0069] The second feature amount calculation unit 13 selects a second representative skeleton coordinate point from among the multiple skeleton coordinate points detected by the skeleton detection unit 131. Here, the second feature amount calculation unit 13 selects the skeleton coordinate point of the right shoulder from among the multiple skeleton coordinate points detected by the skeleton detection unit 131 as the second representative skeleton coordinate point. Then, the second feature amount calculation unit 13 calculates the coordinate of the right shoulder skeleton coordinate point as the second feature amount. Note that the skeleton coordinate point information includes the coordinate of the skeleton coordinate point on the captured image. The second feature amount calculation unit 13 may specify the coordinate of the second representative skeleton coordinate point based on the skeleton coordinate point information, thereby calculating the coordinate of the second representative skeleton coordinate point as the second feature amount. In FIG. 8 , the second feature amount calculation unit 13 calculates the coordinate of the right shoulder skeleton coordinate point indicated by "P11a" as the coordinate of the second representative skeleton coordinate point, i.e., the second feature amount.

[0070] The physique determination unit 14 calculates a first seat-skeleton distance (distance indicated by "401a" in FIG. 8 ) between a point indicated by the coordinates of the third representative seat point calculated by the first feature amount calculation unit 12, i.e., the third representative seat point (seat point indicated by "P23a" in FIG. 8 ), and a point indicated by the coordinates of the second representative skeleton coordinate point calculated by the second feature amount calculation unit 13, i.e., the second representative skeleton coordinate point (skeleton coordinate point indicated by "P11a" in FIG. 8 ). The physique determination unit 14 then determines the physique of the occupant by comparing the calculated first seat-skeleton distance with the determination criterion conditions. For example, suppose the calculated first seat-skeleton distance is 20 pixels. Furthermore, suppose the determination criterion conditions associate a "standard male" physique with a first seat-skeleton distance range of "18 to 21." In this case, the physique determination unit 14 determines that the occupant's physique is "equivalent to a standard male."

[0071] The position of the third representative seat point on the captured image basically remains the same (if the seat is not moved), regardless of the occupant's physique. Note that "remains the same" here does not necessarily mean being exactly the same, but also includes being approximately the same. On the other hand, the position of the second representative skeleton coordinate point on the captured image changes depending on the occupant's physique. If the seat does not move, it is assumed that the position of the second representative skeleton coordinate point on the captured image approaches the contour of the seat as the occupant's physique increases. In other words, the first seat-to-skeleton distance decreases as the occupant's physique increases.

[0072] 9A, 9B, and 9C are diagrams illustrating the first seat-skeleton distance, which varies depending on the occupant's physique. FIG. 9A is the same as FIG. 8. Here, the occupant designated "C1" is assumed to be an adult male with a physique equivalent to a standard male. FIG. 9B illustrates a seat having a contour designated "S" in FIG. 9A, in which an occupant designated "C2" is seated in place of the occupant designated "C1." The occupant designated "C2" in FIG. 9B is the same occupant designated "C2" as the occupant designated "C2" in FIG. 3B. In FIG. 9B, "401b" indicates the first seat-skeleton distance between the seat point corresponding to the right end of the upper edge of the seat back designated "P23a," which is the third representative seat point, and the skeletal coordinate point of the right shoulder designated "P11b," which is the second representative skeletal coordinate point. Figure 9C shows an occupant indicated by "C3" seated in place of the occupant indicated by "C1" in the seat having the contour indicated by "S" in Figure 9A. The occupant indicated by "C3" in Figure 9C is the same occupant as the occupant indicated by "C3" in Figure 3C. In Figure 9C, "401c" indicates the first seat skeleton distance between the seat point corresponding to the right end of the upper edge of the backrest indicated by "P23a," which is the third representative seat point in this case, and the skeleton coordinate point of the right shoulder indicated by "P11c," which is the second representative skeleton coordinate point in this case.

[0073] In the examples shown in Figures 9A, 9B, and 9C, the larger the occupant's physique, the higher the seat height, and the skeletal coordinate point of the right shoulder, i.e., the second representative skeletal coordinate point, is located higher on the seat. As shown in Figures 9A, 9B, and 9C, the larger the occupant's physique, the smaller the first seat-to-skeleton distance. Specifically, the first seat-to-skeleton distance indicated by "401b" is smaller than the first seat-to-skeleton distance indicated by "401c," and the first seat-to-skeleton distance indicated by "401a" is smaller than the first seat-to-skeleton distance indicated by "401b." In other words, the larger the occupant's physique, the smaller the first seat-to-skeleton distance. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the imaging device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such a case, since the occupant sitting in the seat moves as the seat moves, the relationship between the occupant's physique and the first seat-to-skeleton distance, i.e., the larger the occupant's physique, the smaller the first seat-to-skeleton distance, remains unchanged. Note that "remains unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the occupant's physique from the distance between the point indicated by the first feature amount related to the seat outline and the point indicated by the second feature amount related to the occupant, a relationship which is maintained at a constant rate regardless of the position of the seat, thereby enabling a physique determination that is not affected by the position of the seat in which the occupant is sitting.

[0074] <Physique Determination Method Example (5)> For example, the physique determination conditions may define the vertical distance between a line (hereinafter referred to as a “skeleton line”) passing through a point indicated by a first feature amount and a point indicated by a second feature amount, and the physique corresponding to the vertical distance. The first feature amount calculation unit 12 calculates, as the first feature amount, the coordinates of a seat point that is a point on the contour of the seat in the captured image. The second feature amount calculation unit 13 calculates, as the second feature amount, the coordinates of one or more skeletal coordinate points among the multiple skeletal coordinate points detected by the skeletal detection unit 131. The physique determination unit 14 may determine the occupant's physique from the vertical distance between the seat point and the skeletal line (hereinafter referred to as a “second seat-skeleton distance”). Note that the vertical distance between the point indicated by the first feature amount and the skeletal line is, in other words, the distance of a perpendicular line from the point indicated by the first feature amount to the skeletal line.

[0075] In this case, the physique determination conditions include a definition of the second seat-to-seat skeletal distance between a skeletal line passing through a point indicated by the first feature amount and a point indicated by the second feature amount, and a physique corresponding to the second seat-to-seat skeletal distance. For example, the physique determination conditions include a correspondence between the second seat-to-seat skeletal distance and a physique, such as "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small female," "equivalent to a standard male," or "equivalent to a large male."

[0076] Furthermore, in the physical build determination conditions, the first feature amount calculation condition defines the following condition: "A representative seat point on the contour of the seat in the captured image (hereinafter referred to as the "fourth representative seat point") is selected, and the coordinates of the fourth representative seat point are calculated as the first feature amount. The seat point to be the fourth representative seat point is selected in accordance with the fourth representative seat point selection condition." In the first embodiment, the fourth representative seat point selection condition is a condition that determines which seat point is to be the fourth representative seat point. The fourth representative seat point selection condition is set in advance by an administrator or the like, and information indicating the fourth representative seat point selection condition (hereinafter referred to as the "fourth representative seat point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate seat point from among the seat points on the contour of the seat to be the fourth representative seat point. For example, the fourth representative seat point selection condition may be set to a condition that an arbitrary point on the contour of the seat is to be the fourth representative seat point. Furthermore, the fourth representative seat point selection condition may be updated as appropriate. The fourth representative seat point selection condition may be included in the second feature amount calculation condition. Here, as an example, it is assumed that the fourth representative seat point selection condition is set to a seat point corresponding to the right end of the upper side of the headrest as the fourth representative seat point.

[0077] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "One or more representative skeletal coordinate points (hereinafter referred to as "third representative skeletal coordinate points") are selected from the plurality of skeletal coordinate points, and the coordinates of the one or more third representative skeletal coordinate points are calculated as second feature amounts. The skeletal coordinate points to be used as the third representative skeletal coordinate points are selected in accordance with the third representative skeletal point selection conditions." In the first embodiment, the third representative skeletal point selection conditions are conditions that determine which skeletal coordinate points are to be used as the third representative skeletal coordinate points. The third representative skeletal point selection conditions are set in advance by an administrator or the like, and information indicating the third representative skeletal point selection conditions (hereinafter referred to as "third representative skeletal point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate skeletal coordinate point from the skeletal coordinate points detected by the skeleton detection unit 131 to be used as the third representative skeletal coordinate point. However, it is desirable that the third representative skeletal coordinate point be a skeletal coordinate point corresponding to a body part that moves less. For example, when the skeleton detection unit 131 detects only a certain skeleton coordinate point, the third representative skeleton point selection condition may be set to a condition that the skeleton coordinate point detected by the skeleton detection unit 131 is to be the third representative skeleton coordinate point. The third representative skeleton point selection condition may also be updated as appropriate. The third representative skeleton point selection condition may be included in the second feature amount calculation condition. Here, as an example, it is assumed that the third representative skeleton point selection condition is set to a condition that the skeleton coordinate point of the right shoulder is to be the third representative skeleton coordinate point.

[0078] 10 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (5) in the first embodiment. In Fig. 10, the same reference numerals are used for the points and the like that are described with reference numerals in Fig. 2, and redundant description will be omitted. In Fig. 10, "P21a" indicates the seat point at the right end of the upper edge of the headrest.

[0079] When the physique determination conditions are as described above, the first feature amount calculation unit 12 selects a fourth representative seat point from among the seat points on the seat contour. Here, the first feature amount calculation unit 12 selects the seat point corresponding to the right end of the upper edge of the headrest as the fourth representative seat point. The first feature amount calculation unit 12 then calculates the coordinates of the fourth representative seat point as the first feature amount. Note that the first feature amount calculation unit 12 may calculate the coordinates of the fourth representative seat point as the first feature amount by identifying the coordinates of a point corresponding to the fourth representative seat point on the detected seat contour. In FIG. 10 , the first feature amount calculation unit 12 sets the seat point "P21a" as the fourth representative seat point and calculates the coordinates of the seat point "P21a" as the coordinates of the fourth representative seat point, i.e., the first feature amount.

[0080] The second feature amount calculation unit 13 selects one or more third representative skeleton coordinate points from the multiple skeleton coordinate points detected by the skeleton detection unit 131. Here, the second feature amount calculation unit 13 selects the skeleton coordinate point of the right shoulder from the multiple skeleton coordinate points detected by the skeleton detection unit 131 as the third representative skeleton coordinate point. Then, the second feature amount calculation unit 13 calculates the coordinate of the right shoulder skeleton coordinate point as the second feature amount. Note that the second feature amount calculation unit 13 may specify the coordinate of the third representative skeleton coordinate point based on the skeleton coordinate point information, thereby calculating the coordinate of the third representative skeleton coordinate point as the second feature amount. In FIG. 10 , the second feature amount calculation unit 13 calculates the coordinate of the right shoulder skeleton coordinate point indicated by "P11a" as the coordinate of the third representative skeleton coordinate point, i.e., the second feature amount.

[0081] The physique determination unit 14 calculates the second seat-skeleton distance as the perpendicular distance (the distance indicated by "402a" in FIG. 10 ) between the point indicated by the coordinates of the fourth representative seat point calculated by the first feature amount calculation unit 12, i.e., the fourth representative seat point (the seat point indicated by "P21a" in FIG. 10 ), and the point indicated by the coordinates of the third representative skeleton coordinate point calculated by the second feature amount calculation unit 13, i.e., the skeleton line (the line indicated by "501a" in FIG. 10 ) passing through the third representative skeleton coordinate point (the skeleton coordinate point indicated by "P11a" in FIG. 10 ). The physique determination unit 14 then determines the physique of the occupant by comparing the calculated second seat-skeleton distance with the determination criterion conditions. For example, suppose the calculated second seat-skeleton distance is 100 pixels. For example, if the second seat-to-seat frame distance range of 88 to 110 is associated with a physique equivalent to a standard male in the determination criteria, the physique determination unit 14 determines that the occupant's physique is equivalent to a standard male.

[0082] The position of the fourth representative seat point on the captured image basically remains the same (if the seat is not moved), regardless of the occupant's physique. Note that "remains the same" here does not necessarily mean being exactly the same, but also includes being approximately the same. On the other hand, the position of the third representative skeleton coordinate point on the captured image changes depending on the occupant's physique. If the seat does not move, it is assumed that the position of the third representative skeleton coordinate point on the captured image approaches the contour of the seat as the occupant's physique increases. In other words, the second seat-to-skeleton distance decreases as the occupant's physique increases.

[0083] Here, FIGS. 11A, 11B, and 11C are diagrams for explaining the second seat-skeleton distance, which varies depending on the occupant's physique. FIG. 11A is the same as FIG. 10. Here, the occupant designated "C1" is assumed to be an adult male with a physique equivalent to a "standard male." FIG. 11B illustrates a seat having a contour designated "S" in FIG. 11A, in which an occupant designated "C2" is seated in place of the occupant designated "C1." The occupant designated "C2" in FIG. 11B is the same occupant designated "C2" as the occupant designated "C2" in FIG. 3B. In FIG. 11B, "402b" indicates the second seat-skeleton distance between the seat point corresponding to the right end of the upper edge of the headrest designated "P21a," here the fourth representative seat point, and the skeleton line (the line designated "501b" in FIG. 11B) passing through the skeleton coordinate point designated "P11b," here the third representative skeleton coordinate point. Figure 11C shows an occupant indicated by "C3" seated in place of the occupant indicated by "C1" in the seat having the contour indicated by "S" in Figure 11A. The occupant indicated by "C3" in Figure 11C is the same occupant as the occupant indicated by "C3" in Figure 3C. In Figure 11C, "402c" indicates the second seat-skeleton distance between the seat point corresponding to the right end of the upper edge of the headrest indicated by "P21a," here the fourth representative seat point, and the skeleton line (the line indicated by "501c" in Figure 11C) passing through the skeleton coordinate point indicated by "P11c," here the third representative skeleton coordinate point.

[0084] In the examples shown in Figures 11A, 11B, and 11C, the larger the occupant's physique, the higher the seat height, and the skeletal coordinate point of the right shoulder, i.e., the third representative skeletal coordinate point, is located closer to the upper part of the seat. As shown in Figures 11A, 11B, and 11C, the larger the occupant's physique, the smaller the second seat-to-skeleton distance. Specifically, the second seat-to-skeleton distance indicated by "402b" is smaller than the second seat-to-skeleton distance indicated by "402c," and the second seat-to-skeleton distance indicated by "402a" is smaller than the second seat-to-skeleton distance indicated by "402b." In other words, the larger the occupant's physique, the smaller the second seat-to-skeleton distance. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the imaging device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such a case, since the occupant sitting in the seat moves as the seat moves, the relationship between the occupant's physique and the second seat-to-skeleton distance, i.e., the larger the occupant's physique, the smaller the second seat-to-skeleton distance, remains unchanged. Note that "remains unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the occupant's physique from the perpendicular distance between a point indicated by the first feature amount related to the seat outline and a skeleton line passing through a point indicated by the second feature amount related to the occupant, a relationship which is maintained at a constant rate regardless of the position of the seat, thereby enabling a physique determination that is not affected by the position of the seat in which the occupant is sitting.

[0085] <Example of Physical Build Determination Method (6)> Furthermore, for example, the physical build determination conditions may define the vertical distance between a line passing through a point indicated by a first feature amount (hereinafter referred to as the “seat line”) and a point indicated by a second feature amount, and the physical build corresponding to the vertical distance, the first feature amount calculation unit 12 may calculate, as the first feature amount, the coordinates of one or more seat points that are points on the contour of the seat in the captured image, the second feature amount calculation unit 13 may calculate, as the second feature amount, the coordinates of a certain skeleton coordinate point among the plurality of skeleton coordinate points detected by the skeleton detection unit 131, and the physical build determination unit 14 may determine the physical build of the occupant from the vertical distance between the seat line and the certain skeleton coordinate point (hereinafter referred to as the “third seat-skeleton distance”).

[0086] In this case, the physique determination conditions include a definition of the third seat-skeleton distance between the seat line passing through the seat point indicated by the first feature amount and a certain skeletal coordinate point indicated by the second feature amount, and a physique corresponding to the third seat-skeleton distance. For example, the physique determination conditions include a correspondence between the third seat-skeleton distance and a physique, such as "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small female," "equivalent to a standard male," or "equivalent to a large male."

[0087] Furthermore, in the physical build determination conditions, the first feature amount calculation condition defines the following condition: "One or more representative seat points (hereinafter referred to as "fifth representative seat points") on the contour of the seat in the captured image are selected, and the coordinates of the fifth representative seat points are calculated as the first feature amount. The seat point to be the fifth representative seat point is selected in accordance with the fifth representative seat point selection condition." In the first embodiment, the fifth representative seat point selection condition is a condition that determines which seat point is to be the fifth representative seat point. The fifth representative seat point selection condition is set in advance by an administrator or the like, and information indicating the fifth representative seat point selection condition (hereinafter referred to as "fifth representative seat point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate seat point from among the seat points on the contour of the seat to be the fifth representative seat point. For example, the fifth representative seat point selection condition may be set to a condition that an arbitrary point on the contour of the seat is to be the fifth representative seat point. The fifth representative seat point selection condition may also be updated as appropriate. The fifth representative seat point selection condition may be included in the second feature amount calculation condition. Here, as an example, it is assumed that the fifth representative seat point selection condition is set to set the seat points corresponding to both ends of the upper side of the headrest as the fifth representative seat points.

[0088] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "A representative skeletal coordinate point (hereinafter referred to as the "fourth representative skeletal coordinate point") is selected from among the multiple skeletal coordinate points, and the coordinates of the fourth representative skeletal coordinate point are calculated as the second feature. The skeletal coordinate point to be used as the fourth representative skeletal coordinate point is selected in accordance with the fourth representative skeletal point selection conditions." In the first embodiment, the fourth representative skeletal point selection conditions are conditions that determine which skeletal coordinate point is to be used as the fourth representative skeletal coordinate point. The fourth representative skeletal point selection conditions are set in advance by an administrator or the like, and information indicating the fourth representative skeletal point selection conditions (hereinafter referred to as the "fourth representative skeletal point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate skeletal coordinate point from the skeletal coordinate points detected by the skeleton detection unit 131 to be used as the fourth representative skeletal coordinate point. However, it is desirable that the fourth representative skeletal coordinate point be a skeletal coordinate point corresponding to a body part with little movement. For example, when the skeleton detection unit 131 detects only a certain skeleton coordinate point, the fourth representative skeleton point selection condition may be set to set the skeleton coordinate point detected by the skeleton detection unit 131 as the fourth representative skeleton coordinate point. The fourth representative skeleton point selection condition may also be updated as appropriate. The fourth representative skeleton point selection condition may be included in the second feature amount calculation condition. Here, as an example, it is assumed that the fourth representative skeleton point selection condition is set to set the skeleton coordinate point of the neck as the fourth representative skeleton coordinate point.

[0089] 12 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (6) in the first embodiment. In FIG. 12, the same reference numerals are used for the points and the like that are described with reference numerals in FIG. 2, and redundant description will be omitted. In FIG. 12, "P21a" indicates the seat point at the right end of the upper edge of the headrest, and "P22a" indicates the seat point at the left end of the upper edge of the headrest. Also, in FIG. 12, "P15a" indicates the skeletal coordinate point of the occupant's neck.

[0090] When the physique determination conditions are as described above, the first feature amount calculation unit 12 selects one or more fifth representative seat points from among the seat points on the seat contour. Here, the first feature amount calculation unit 12 selects seat points corresponding to both ends of the upper edge of the headrest as the fifth representative seat points. The first feature amount calculation unit 12 then calculates the coordinates of the fifth representative seat points as the first feature amount. Note that the first feature amount calculation unit 12 may calculate the coordinates of the fifth representative seat points as the first feature amount by identifying the coordinates of points corresponding to the fifth representative seat points on the detected seat contour. In FIG. 12 , the first feature amount calculation unit 12 sets the seat points "P21a" and "P22a" as the fifth representative seat points, respectively, and calculates the coordinates of the seat points "P21a" and "P22a" as the coordinates of the fifth representative seat points, i.e., the first feature amount.

[0091] The second feature amount calculation unit 13 selects a fourth representative skeleton coordinate point from among the multiple skeleton coordinate points detected by the skeleton detection unit 131. Here, the second feature amount calculation unit 13 selects the neck skeleton coordinate point from among the multiple skeleton coordinate points detected by the skeleton detection unit 131 as the fourth representative skeleton coordinate point. The second feature amount calculation unit 13 then calculates the coordinate of the neck skeleton coordinate point as the second feature amount. Note that the second feature amount calculation unit 13 may specify the coordinate of the fourth representative skeleton coordinate point based on the skeleton coordinate point information, thereby calculating the coordinate of the fourth representative skeleton coordinate point as the second feature amount. In FIG. 12 , the second feature amount calculation unit 13 calculates the coordinate of the neck skeleton coordinate point indicated by "P15a" as the coordinate of the fourth representative skeleton coordinate point, i.e., the second feature amount.

[0092] The physique determination unit 14 calculates the third seat-to-skeleton distance as the vertical distance (the distance indicated by "403a" in FIG. 12 ) between the point indicated by the coordinates of the fifth representative seat point calculated by the first feature amount calculation unit 12, i.e., the seat line (the line indicated by "601a" in FIG. 12 ) passing through the fifth representative seat point (i.e., the seat points indicated by "P21a" and "P22a" in FIG. 12 ), and the point indicated by the coordinates of the fourth representative skeleton coordinate point calculated by the second feature amount calculation unit 13, i.e., the fourth representative skeleton coordinate point (i.e., the skeleton coordinate point indicated by "P15a" in FIG. 12 ). The physique determination unit 14 then compares the calculated third seat-to-skeleton distance with the determination criteria to determine the occupant's physique. For example, suppose the calculated third seat-to-skeleton distance is 60 pixels. For example, in the determination criteria, the range of the third seat-to-seat frame distance "50 to 65" is associated with a physique equivalent to a "standard male." In this case, the physique determination unit 14 determines that the occupant's physique is "standard male."

[0093] The position of the fifth representative seat point on the captured image basically remains the same (if the seat is not moved), regardless of the occupant's physique. Note that "remains the same" here does not necessarily mean being exactly the same, but also includes being approximately the same. On the other hand, the position of the fourth representative skeleton coordinate point on the captured image changes depending on the occupant's physique. If the seat does not move, it is assumed that the position of the fourth representative skeleton coordinate point on the captured image approaches the contour of the seat as the occupant's physique increases. In other words, the third seat-skeleton distance decreases as the occupant's physique increases.

[0094] Here, Figures 13A, 13B, and 13C are diagrams for explaining the third seat-skeleton distance, which varies depending on the occupant's physique. Figure 13A is the same as Figure 12. Here, the occupant designated "C1" is assumed to be an adult male with a physique equivalent to that of a "standard male." Figure 13B illustrates a seat having a contour designated "S" in Figure 13A, in which an occupant designated "C2" is seated in place of the occupant designated "C1." The occupant designated "C2" in Figure 13B is the same occupant designated "C2" in Figure 3B. In Figure 13B, "403b" indicates the third seat-skeleton distance between the seat points corresponding to both ends of the upper edge of the headrest designated "P21a" and "P22a," i.e., the seat line passing through the fifth representative seat point (the line designated "601a" in Figure 13B), and the skeleton coordinate point designated "P15b," i.e., the fourth representative skeleton coordinate point. Figure 13C shows an occupant indicated by "C3" seated in place of the occupant indicated by "C1" in a seat having a contour indicated by "S" in Figure 13A. The occupant indicated by "C3" in Figure 13C is the same occupant as the occupant indicated by "C3" in Figure 3C. In Figure 13C, "403c" indicates the third seat-skeleton distance between the seat line (the line indicated by "601a" in Figure 13C) passing through the seat points corresponding to both ends of the upper edge of the headrest indicated by "P21a" and "P22a," here the fifth representative seat point, and the skeleton coordinate point indicated by "P15c," here the fourth representative skeleton coordinate point.

[0095] In the examples shown in Figures 13A, 13B, and 13C, the larger the occupant's physique, the higher the seat height, and the skeletal coordinate point of the neck, i.e., the fourth representative skeletal coordinate point, is located closer to the upper part of the seat. As shown in Figures 13A, 13B, and 13C, the larger the occupant's physique, the smaller the third seat-to-skeleton distance. Specifically, the third seat-to-skeleton distance indicated by "403b" is smaller than the third seat-to-skeleton distance indicated by "403c," and the third seat-to-skeleton distance indicated by "403a" is smaller than the third seat-to-skeleton distance indicated by "403b." In other words, the larger the occupant's physique, the smaller the third seat-to-skeleton distance. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the imaging device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such a case, since the occupant sitting in the seat moves as the seat moves, the relationship between the occupant's physique and the third seat-to-skeleton distance, i.e., the larger the occupant's physique, the smaller the third seat-to-skeleton distance, remains unchanged. Note that "remains unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the occupant's physique from the perpendicular distance between the seat line passing through a point indicated by the first feature amount related to the seat contour and a point indicated by the second feature amount related to the occupant, a relationship which is maintained at a constant rate regardless of the position of the seat, thereby enabling physique determination that is not affected by the position of the seat in which the occupant is sitting.

[0096] <Physique Determination Method Example (7)> ​​For example, the physique determination conditions may define the horizontal distance between a seat line passing through a point indicated by a first feature amount and a skeleton line passing through a point indicated by a second feature amount, and the physique corresponding to the horizontal distance. The first feature amount calculation unit 12 calculates, as the first feature amount, the coordinates of one or more seat points that are points on the contour of the seat in the captured image. The second feature amount calculation unit 13 calculates, as the second feature amount, the coordinates of one or more skeleton coordinate points among the multiple skeleton coordinate points detected by the skeleton detection unit 131. The physique determination unit 14 may determine the occupant's physique from the horizontal distance between the seat line passing through the one or more seat points and a skeleton line that is horizontal to the seat line and passes through the one or more skeleton coordinate points (hereinafter referred to as a “fourth seat-skeleton distance”). In the description of the <Physique Determination Method Example (7)>, the skeleton line that is horizontal to the seat line and passes through one or more skeleton coordinate points is also referred to as a “horizontal skeleton line.”

[0097] In this case, the physique determination conditions include a definition of a fourth seat-skeleton distance between a seat line passing through the seat point indicated by the first feature amount and a horizontal skeleton line passing through one or more skeleton coordinate points indicated by the second feature amount, and a physique corresponding to the fourth seat-skeleton distance. For example, the physique determination conditions include a correspondence between the fourth seat-skeleton distance and a physique, such as "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small female," "equivalent to a standard male," or "equivalent to a large male."

[0098] Furthermore, in the physical build determination conditions, the first feature amount calculation condition defines the following condition: "One or more representative seat points (hereinafter referred to as "sixth representative seat points") on the contour of the seat in the captured image are selected, and the coordinates of the one or more sixth representative seat points are calculated as the first feature amount. The seat points to be used as the sixth representative seat points are selected in accordance with sixth representative seat point selection conditions." In the first embodiment, the sixth representative seat point selection conditions are conditions that determine which seat points are to be used as the sixth representative seat points. The sixth representative seat point selection conditions are set in advance by an administrator or the like, and information indicating the sixth representative seat point selection conditions (hereinafter referred to as "sixth representative seat point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate seat point from among the seat points on the contour of the seat to be used as the sixth representative seat point. For example, the sixth representative seat point selection conditions may be set to determine an arbitrary point on the contour of the seat as the sixth representative seat point. The sixth representative seat point selection condition may be updated as needed. The sixth representative seat point selection condition may be included in the second feature amount calculation condition. Here, as an example, it is assumed that the sixth representative seat point selection condition is set to a seat point corresponding to the right end of the upper edge of the headrest as the sixth representative seat point.

[0099] Furthermore, in the physique determination conditions, the second feature calculation conditions define the following condition: "One or more representative skeletal coordinate points (hereinafter referred to as "fifth representative skeletal coordinate points") are selected from the plurality of skeletal coordinate points, and the coordinates of the one or more fifth representative skeletal coordinate points are calculated as second feature amounts. The skeletal coordinate points to be used as the fifth representative skeletal coordinate points are selected in accordance with the fifth representative skeletal point selection conditions." In the first embodiment, the fifth representative skeletal point selection conditions are conditions that determine which skeletal coordinate points are to be used as the fifth representative skeletal coordinate points. The fifth representative skeletal point selection conditions are set in advance by an administrator or the like, and information indicating the fifth representative skeletal point selection conditions (hereinafter referred to as "fifth representative skeletal point selection condition information") is stored in a storage unit (not shown). The administrator or the like determines an appropriate skeletal coordinate point from the skeletal coordinate points detected by the skeleton detection unit 131 to be used as the fifth representative skeletal coordinate point. However, it is desirable that the fifth representative skeletal coordinate point be a skeletal coordinate point corresponding to a body part with little movement. For example, when the skeleton detection unit 131 detects only a certain skeleton coordinate point, the fifth representative skeleton point selection condition may be set to set the skeleton coordinate point detected by the skeleton detection unit 131 as the fifth representative skeleton coordinate point. The fifth representative skeleton point selection condition may be updated as appropriate. The fifth representative skeleton point selection condition may be included in the second feature value calculation condition. Here, as an example, it is assumed that the fifth representative skeleton point selection condition is set to set the skeleton coordinate points of both shoulders as the fifth representative skeleton coordinate points.

[0100] 14 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (7) in the first embodiment. In Fig. 14, the same reference numerals are used for the points and the like that are described with reference numerals in Fig. 2, and redundant description will be omitted. In Fig. 14, "P21a" indicates a seat point at the right end of the upper side of the headrest.

[0101] When the physique determination conditions are as described above, the first feature amount calculation unit 12 selects one or more sixth representative seat points from among the seat points on the seat contour. In this example, the first feature amount calculation unit 12 selects the seat point corresponding to the right end of the upper edge of the headrest as the sixth representative seat point. The first feature amount calculation unit 12 then calculates the coordinates of the sixth representative seat point as the first feature amount. Note that the first feature amount calculation unit 12 may calculate the coordinates of the sixth representative seat point as the first feature amount by identifying the coordinates of a point corresponding to the sixth representative seat point on the detected seat contour. In FIG. 14 , the first feature amount calculation unit 12 sets the seat point "P21a" as the sixth representative seat point and calculates the coordinates of the seat point "P21a" as the coordinates of the sixth representative seat point, i.e., the first feature amount.

[0102] The second feature amount calculation unit 13 selects one or more fifth representative skeleton coordinate points from the multiple skeleton coordinate points detected by the skeleton detection unit 131. Here, the second feature amount calculation unit 13 selects the skeleton coordinate points of both shoulders as fifth representative skeleton coordinate points from the multiple skeleton coordinate points detected by the skeleton detection unit 131. Then, the second feature amount calculation unit 13 calculates the coordinates of the skeleton coordinate points of both shoulders as second feature amounts. Note that the second feature amount calculation unit 13 may calculate the coordinates of the fifth representative skeleton coordinate points as second feature amounts by identifying the coordinates of the fifth representative skeleton coordinate points based on the skeleton coordinate point information. In FIG. 14 , the second feature amount calculation unit 13 calculates the coordinates of the skeleton coordinate point of the right shoulder indicated by "P11a" and the coordinates of the skeleton coordinate point of the left shoulder indicated by "P12a" as the coordinates of the fifth representative skeleton coordinate points, i.e., the second feature amounts.

[0103] The physique determination unit 14 calculates the fourth seat-skeleton distance (the distance indicated by "404a" in FIG. 14 ) between a seat line (the line indicated by "602a" in FIG. 14 ) passing through the point indicated by the coordinates of the sixth representative seat point calculated by the first feature amount calculation unit 12, i.e., the sixth representative seat point (the seat point indicated by "P21a" in FIG. 14 ), and a horizontal skeleton line (the line indicated by "501a" in FIG. 14 ) passing through the point indicated by the coordinates of the fifth representative skeleton coordinate point calculated by the second feature amount calculation unit 13, i.e., the fifth representative skeleton coordinate points (the skeleton coordinate points indicated by "P11a" and "P12a" in FIG. 14 ). The physique determination unit 14 then determines the occupant's physique by comparing the calculated fourth seat-skeleton distance with the determination criteria. For example, assume that the calculated fourth seat-skeleton distance is 80 pixels. For example, in the determination criteria, the fourth seat-to-frame distance range of 75 to 82 is associated with a physique equivalent to a standard male. In this case, the physique determination unit 14 determines that the occupant's physique is equivalent to a standard male.

[0104] The position of the sixth representative seat point on the captured image basically remains the same (if the seat is not moved), regardless of the occupant's physique. Note that "remains the same" here does not necessarily mean being exactly the same, but also includes being approximately the same. On the other hand, the position of the fifth representative skeleton coordinate point on the captured image changes depending on the occupant's physique. If the seat does not move, it is expected that the position of the fifth representative skeleton coordinate point on the captured image will approach the contour of the seat as the occupant's physique increases. In other words, the fourth seat-to-skeleton distance decreases as the occupant's physique increases.

[0105] Here, Figures 15A, 15B, and 15C are diagrams for explaining the fourth seat frame distance, which changes depending on the occupant's physique. Figure 15A is the same as Figure 14. Here, the occupant indicated by "C1" is assumed to be an adult male with a physique equivalent to that of a standard male. Figure 15B shows an occupant indicated by "C2" seated in place of the occupant indicated by "C1" in the seat having the contour indicated by "S" in Figure 15A. The occupant indicated by "C2" in Figure 15B is the same occupant as the occupant indicated by "C2" in Figure 3B. In Figure 15B, "404b" indicates the fourth seat-skeleton distance between the seat point corresponding to the right end of the upper edge of the headrest indicated by "P21a," here a seat line passing through the sixth representative seat point (the line indicated by "602a" in Figure 15B), and the horizontal skeleton line passing through the skeleton coordinate points indicated by "P11b" and "P12b," here a fifth representative skeleton coordinate point (the line indicated by "501b" in Figure 15B). Figure 15C shows an occupant indicated by "C3" seated in place of the occupant indicated by "C1" in the seat having the contour indicated by "S" in Figure 15A. The occupant indicated by "C3" in Figure 15C is the same occupant as the occupant indicated by "C3" in Figure 3C. In Figure 15C, "404c" indicates the fourth seat skeleton distance between the seat point corresponding to the right end of the upper edge of the headrest indicated by "P21a", in this case the seat straight line passing through the sixth representative seat point (in Figure 15C the straight line indicated by "602a"), and the skeleton coordinate points indicated by "P11c" and "P12c", in this case the horizontal skeleton straight line passing through the fifth representative skeleton coordinate point (in Figure 15C the straight line indicated by "501c").

[0106] In the examples shown in Figures 15A, 15B, and 15C, the larger the occupant's physique, the higher the seat height, and the skeletal coordinate points of both shoulders, i.e., the fifth representative skeletal coordinate point, are located higher on the seat. As shown in Figures 15A, 15B, and 15C, the fourth seat-to-skeleton distance becomes smaller as the occupant's physique becomes larger. Specifically, the fourth seat-to-skeleton distance indicated by "404b" is smaller than the fourth seat-to-skeleton distance indicated by "404c," and the fourth seat-to-skeleton distance indicated by "404a" is smaller than the fourth seat-to-skeleton distance indicated by "404b." In other words, the larger the occupant's physique, the smaller the fourth seat-to-skeleton distance. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the imaging device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such a case, since the occupant sitting in the seat moves as the seat moves, the relationship between the occupant's physique and the fourth seat-to-skeleton distance, i.e., the larger the occupant's physique, the smaller the fourth seat-to-skeleton distance, remains unchanged. Note that "remains unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the occupant's physique from the horizontal distance between the seat line passing through a point indicated by the first feature amount related to the seat contour and the horizontal skeleton line passing through a point indicated by the second feature amount related to the occupant, a relationship which is maintained at a constant rate regardless of the position of the seat, thereby enabling physique determination that is not affected by the position of the seat in which the occupant is sitting.

[0107] <Example of physique determination method (8)> For example, the conditions for physique determination define the ratio of the second feature to the first feature and the physique corresponding to the ratio, the first feature calculation unit 12 calculates the area of ​​the seat region surrounded by the outline of the seat in the captured image as the first feature, the second feature calculation unit 13 calculates the area of ​​the occupant's body region (hereinafter referred to as the "human body region") in the captured image as the second feature, and the physique determination unit 14 may determine the occupant's physique from the ratio of the area of ​​the human body region to the area of ​​the seat region.

[0108] In this case, the physique determination conditions define the "ratio of the second feature amount to the first feature amount, and the physique for that ratio" in the determination criteria conditions. For example, the physique determination criteria conditions correspond to a range of the ratio of the second feature amount to the first feature amount, and the physique, in this case, "equivalent to a 1-year-old child," "equivalent to a 3-year-old child," "equivalent to a 6-year-old child," "equivalent to a 10-year-old child," "equivalent to a small woman," "equivalent to a standard man," or "equivalent to a large man."

[0109] In addition, in the conditions for determining physical build, the first characteristic amount calculation condition defines the condition that "the area of ​​the seat region surrounded by the outline of the seat in the captured image is taken as the first characteristic amount."

[0110] In addition, in the physical build determination conditions, the second feature amount calculation condition defines the condition that "the area of ​​the human body region of the occupant's body in the captured image is taken as the second feature amount."

[0111] 16 is a diagram for explaining the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination by the physique determination unit 14 in the physique determination method example (8) in embodiment 1. In Fig. 16, the same reference numerals are used to denote the same points as those described with reference numerals in Fig. 2, and redundant description will be omitted.

[0112] When the physique determination conditions are as described above, the first feature amount calculation unit 12 calculates the area of ​​the seat area as the first feature amount. In Fig. 16, the first feature amount calculation unit 12 calculates the area of ​​the area indicated by "201a" as the area of ​​the seat area, i.e., the first feature amount. An example of calculating the area of ​​the seat area has already been described, so a duplicated description will be omitted.

[0113] The second feature amount calculation unit 13 calculates the area of ​​the human body region as the second feature amount. In FIG. 16 , the second feature amount calculation unit 13 calculates the area of ​​the region indicated by "701a" as the area of ​​the human body region, i.e., the second feature amount. The second feature amount calculation unit 13 may calculate the area of ​​the human body region using a known object detection method using machine learning, such as Mask R-CNN. For example, Mask R-CNN is capable of pixel-by-pixel recognition and can recognize object boundaries.

[0114] In the example physique determination method (8), as described above, the second feature amount calculation unit 13 does not use the coordinates of the skeleton coordinate points when calculating the second feature amount, in other words, the area of ​​the human body region. For example, when determining the physique of an occupant using a method such as the example physique determination method (8), the physique determination device 1 may be configured without including the skeleton detection unit 131.

[0115] The physique determination unit 14 calculates the ratio of the area of ​​the human body region (the region indicated by "701a" in FIG. 16) calculated by the second feature amount calculation unit 13 to the area of ​​the seat region (the region indicated by "201a" in FIG. 16) calculated by the first feature amount calculation unit 12. Then, the physique determination unit 14 determines the physique of the occupant by comparing the calculated ratio with the determination criterion conditions. For example, if the area of ​​the seat region calculated by the first feature amount calculation unit 12 is 1000 pixels, ２ , the area of ​​the human body region calculated by the second feature amount calculation unit 13 is 800 pixels ２ In this case, the physique determination unit 14 calculates the ratio of the area of ​​the human body region to the area of ​​the seat region as 800 / 1000 = 0.8. Then, the physique determination unit 14 compares the ratio "0.8" with the determination criterion conditions. For example, in the determination criterion conditions, the physique of "average male" is associated with a range of "0.7 to 0.85" for the ratio of the second feature amount to the first feature amount. In this case, the physique determination unit 14 determines that the occupant's physique is "average male."

[0116] The area of ​​the occupant's body region increases as the occupant's physique increases. On the other hand, the area of ​​the seat region does not change depending on the occupant's physique. Therefore, the larger the occupant's physique, the greater the ratio of the area of ​​the body region to the area of ​​the seat region.

[0117] Here, FIGS. 17A, 17B, and 17C are diagrams for explaining the ratio of the area of ​​the human body region to the area of ​​the seat region, which changes depending on the occupant's physique. FIG. 17A is the same as FIG. 16. Here, the occupant designated "C1" is assumed to be an adult male with a physique equivalent to a "standard male." FIG. 17B is a diagram showing an adult female occupant with a physique equivalent to a "small woman" seated in place of the occupant designated "C1" in a seat having a contour designated "S" in FIG. 17A. In FIG. 17B, the occupant is designated "C2." In addition, in FIG. 17B, "701b" indicates the human body region of the occupant designated "C2." FIG. 17C is a diagram showing an infant occupant with a physique equivalent to a "six-year-old child" seated in place of the occupant designated "C1" in a seat having a contour designated "S" in FIG. 17A. In Fig. 17C, the occupant is designated by "C3." Also, in Fig. 17C, "701c" indicates the human body region of the occupant designated by "C3."

[0118] As shown in Figures 17A, 17B, and 17C, the area of ​​the occupant's body region increases as the occupant's physique increases. Specifically, the area of ​​the body region indicated by "701b" is larger than the area of ​​the body region indicated by "701c," which in turn is larger than the area of ​​the body region indicated by "701a." Meanwhile, the size of the seat region indicated by "201a" does not change depending on the occupant's physique. That is, the larger the occupant's physique, the larger the ratio of the area of ​​the body region to the area of ​​the seat region. If the seat position is moved forward / backward or up / down, or if the seat reclining angle is changed, the positional relationship between the imaging device 2 and the seat changes, which may cause the seat to appear larger or smaller in the captured image. Even in such cases, the occupant seated in the seat also moves as the seat moves, so the ratio of the area of ​​the body region to the area of ​​the seat region remains unchanged. Note that "unchanged" here does not necessarily mean being exactly the same, but also includes being approximately the same. The physique determination unit 14 determines the physique of the occupant from the ratio between a first feature value related to the contour of the seat and a second feature value related to the occupant, the relationship of which is maintained at a constant rate regardless of the position of the seat, thereby making it possible to perform a physique determination that is not affected by the position of the seat in which the occupant is seated.

[0119] Regarding the above-described "Physical Size Determination Method Example (1)" to "Physical Size Determination Method Example (8)," for example, in "Physical Size Determination Method Example (1)" and "Physical Size Determination Method Example (3)," the second feature is calculated from the coordinates of skeletal coordinate points indicating the vertical and horizontal widths resulting from the occupant's physique. Therefore, it is assumed that the feature more accurately reflects the occupant's skeletal information. Therefore, the physique determination unit 14 can more accurately determine the occupant's physique. Furthermore, for example, in "Physical Size Determination Method Example (2)," "Physical Size Determination Method Example (4)," and "Physical Size Determination Method Example (8)," the second feature is calculated from a smaller number of skeletal coordinate points. Therefore, even if the occupant's body is obscured by luggage or the like, making it difficult to detect skeletal coordinate points in the captured image, the second feature calculation unit 13 can more accurately calculate the second feature. As a result, the physique determination unit 14 can more accurately determine the occupant's physique.

[0120] In the above description, the physique determination unit 14 has presented "Physique Determination Method Example (1)" to "Physique Determination Method Example (8)" as methods for determining an occupant's physique based on the first feature amount, the second feature amount, and the physique determination conditions, but these are merely examples. The physique determination unit 14 may determine the occupant's physique based on the first feature amount related to the seat contour and the second feature amount related to the occupant, which maintain a constant relationship regardless of the seat position. The physique determination device 1 can determine the occupant's physique based on various combinations of the second feature amount related to the occupant's skeleton (skeleton point area, distance between skeleton points, human body area, etc.) that can be detected from the captured image and the first feature amount related to the seat contour, thereby enhancing the information available for determining the occupant's physique.

[0121] After determining the physique of the occupant, the physique determination unit 14 outputs the physique determination result to the determination result output unit 15. The physique determination result includes information indicating the physique of the occupant. In the physique determination result, the information indicating the physique of the occupant may be associated with information that identifies the occupant.

[0122] The judgment result output unit 15 outputs the physique judgment result output from the physique judgment unit 14 to various devices. The function of the judgment result output unit 15 may be included in the physique judgment unit 14. In this case, the physique judgment device 1 may be configured without including the judgment result output unit 15.

[0123] The operation of the physique determination device 1 according to the first embodiment will be described. FIG. 18 is a flowchart for describing the operation of the physique determination device 1 according to the first embodiment. For example, when the power of the vehicle is turned on or when it is detected that an occupant has entered the vehicle, the physique determination device 1 starts the operation shown in the flowchart of FIG. 18 and repeats the operation shown in the flowchart of FIG. 18 until the power of the vehicle is turned off. Note that, for example, a control unit (not shown) of the physique determination device 1 detects that an occupant has entered the vehicle by obtaining sensor information detected by various sensors (not shown), such as a door sensor (not shown), provided in the vehicle, and outputs instructions to start operations to the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the physique determination unit 14, and the determination result output unit 15. Furthermore, for example, the physique determination device 1 may start the operation shown in the flowchart of FIG. 18 and then terminate the operation shown in the flowchart of FIG. 18 once it has determined the physique of the occupant.

[0124] The captured image acquisition unit 11 acquires a captured image of the occupant captured by the imaging device 2 (step ST1). The captured image acquisition unit 11 outputs the acquired captured image to the first feature amount calculation unit 12 and the second feature amount calculation unit 13.

[0125] The first feature amount calculation unit 12 calculates a first feature amount relating to the contour of the seat in the captured image based on the captured image acquired by the captured image acquisition unit 11 in step ST1 (step ST2). The first feature amount calculation unit 12 detects the contour of the seat in the captured image output from the captured image acquisition unit 11. If the first feature amount calculation unit 12 detects only a portion of the contour of the seat in the captured image, the first feature amount calculation unit 12 may estimate the undetected portion from the detected portion and interpolate the undetected portion. The first feature amount calculation unit 12 then calculates a first feature amount relating to the detected contour of the seat in the captured image. More specifically, the first feature amount calculation unit 12 calculates the first feature amount in accordance with the first feature amount calculation condition of the physique determination condition. The first feature amount calculation unit 12 outputs the calculated first feature amount to the physique determination unit 14.

[0126] The second feature amount calculation unit 13 calculates a second feature amount related to the occupant in the captured image based on the captured image acquired by the captured image acquisition unit 11 in step ST1 (step ST3). More specifically, the skeleton detection unit 131 of the second feature amount calculation unit 13 detects multiple skeleton coordinate points of the occupant that indicate body parts of the occupant in the captured image. Then, the second feature amount calculation unit 13 calculates the second feature amount based on the multiple skeleton coordinate points detected by the skeleton detection unit 131. More specifically, the second feature amount calculation unit 13 calculates the second feature amount in accordance with the second feature amount calculation condition of the physique determination condition. After calculating the second feature amount, the second feature amount calculation unit 13 outputs the calculated second feature amount to the physique determination unit 14.

[0127] The physique determination unit 14 determines the physique of the occupant based on the first feature calculated by the first feature calculation unit 12 in step ST2 and the second feature calculated by the second feature calculation unit 13 in step ST3 (step ST4). More specifically, the physique determination unit 14 determines the physique of the occupant based on the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination conditions. After determining the physique of the occupant, the physique determination unit 14 outputs the physique determination result to the determination result output unit 15, and the determination result output unit 15 outputs the physique determination result output from the physique determination unit 14 to various devices (step ST4).

[0128] In the operation of the physique determination device 1 shown in the flowchart of Fig. 18, the processing of step ST2 and the processing of step ST3 are performed in parallel, but this is merely an example. For example, the processing of step ST3 may be performed after the processing of step ST2, or vice versa. It is sufficient that the processing of step ST2 and the processing of step ST3 are performed before the processing of step ST4 is performed.

[0129] In this way, the physique determination device 1 calculates a first feature amount, which is a feature amount in the captured image relating to the contour of the seat, based on the captured image of the occupant captured by the imaging device 2. Furthermore, the physique determination device 1 calculates a second feature amount, which is a feature amount in the captured image relating to the occupant, based on the acquired captured image. Then, the physique determination device 1 determines the physique of the occupant based on the first feature amount and the second feature amount. This allows the physique determination device 1 to perform a physique determination that is not affected by the position of the seat in which the occupant is seated.

[0130] As described above, the physique determination result of the occupant's physique determined by the physique determination device 1 is used for various controls. One example of the various controls is the control of an airbag or a seat belt. In the control of the airbag or the seat belt, the inflation force of the airbag or the restraining force of the seat belt is adjusted according to the occupant's physique. Standards such as the Federal Motor Vehicle Safety Standards (FMVSS) and the European New Car Assessment Program (European New Car Assessment Program) define classes of occupant physiques that must be determined when controlling an airbag or a seat belt, as shown in FIG. 19 . As can be seen from FIG. 19 , there are significant differences in height or weight between different physique classes. Therefore, even a difference of one class is expected to significantly change the inflation force of the airbag or the restraining force of the seatbelt in airbag or seatbelt control. If the occupant's physique is incorrectly determined, necessary and sufficient control may not be performed, making it difficult to prevent unexpected events. Therefore, in control such as airbag or seatbelt control performed based on the occupant's physique, a technology that can accurately determine the occupant's physique is essential. In contrast, the physique determination device 1 according to the first embodiment, as described above, determines the physique using information about the contours of the seat, thereby enabling highly accurate determination of the occupant's physique that is not affected by the position of the seat in which the occupant is seated.

[0131] Additionally, in view of the need for vehicles to be widely used, cost considerations must also be taken into account so that end users can easily purchase them, etc. The physique determination device 1 according to the first embodiment can perform highly accurate physique determination, based solely on the captured image captured by the imaging device 2, without being affected by the position of the seat in which the occupant is seated.

[0132] In the first embodiment described above, the first feature amount calculation unit 12 in the physique determination device 1 may have a function of interpolating the undetected portion by estimating the undetected portion from the detected portion when only a portion of the seat contour is detected in the captured image. This is not limiting. For example, in the physique determination device 1, when the first feature amount calculation unit 12 cannot detect most of the seat contour in the captured image, the occupant's physique may be determined to be the largest physique that can be determined, i.e., "equivalent to a large male." When most of the seat contour is not detected in the captured image, it is assumed that the occupant's vertical size (e.g., sitting height) and horizontal size (e.g., shoulder width) are both large enough to obscure the seat contour. In other words, the physique determination device 1 can determine that the occupant's physique is sufficiently large. In this case, if the first feature amount calculation unit 12 cannot detect a seat contour equal to or greater than a predetermined threshold (hereinafter referred to as a "second contour detection threshold") from the captured image, the first feature amount calculation unit 12 outputs information indicating that the contour is undetectable (hereinafter referred to as "contour undetectable information") to the physique determination unit 14. Note that the second contour detection threshold is a value smaller than the above-mentioned first contour detection threshold. If the contour undetectable information is output from the first feature amount calculation unit 12, the physique determination unit 14 determines that the occupant's physique is equal to the set specified physique. In the first embodiment, the "set specified physique" refers to, for example, the largest physique that can be determined, in this case, "equivalent to a large male." The specified physique is determined in advance by an administrator or the like, and information indicating the "specified physique" (hereinafter referred to as "specified physique information") is stored in a location accessible to the physique determination unit 14, such as a storage unit (not shown). Here, the specified physique is set to "equivalent to a large male," but this is only an example, and the administrator or the like can set an appropriate physique as the specified physique, taking into consideration the installation location of the imaging device 2, etc.

[0133] In the first embodiment described above, the physique determination unit 14 in the physique determination device 1 determines the physique of the occupant based on the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and the physique determination conditions. This is merely an example, and the physique determination unit 14 may determine the physique of the occupant based on, for example, the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and a machine learning model (herein referred to as a “physique determination model”) that is generated in advance and stored in a location that the physique determination unit 14 can refer to, such as a storage unit (not shown). The physique determination model is generated in advance by an administrator or the like. The physique determination model is a machine learning model that receives the first feature and the second feature and outputs information indicating the physique. The physique determination unit 14 may be configured to determine the physique of the occupant based on a first feature quantity, which is a feature quantity in the captured image relating to the contour of the seat, and a second feature quantity, which is a feature quantity in the captured image relating to the occupant.

[0134] In the first embodiment described above, it is assumed that the occupant is properly seated in the seat within the vehicle cabin. However, it is possible that the occupant is not properly seated in the seat, for example, in a state of poor posture. The physique determination device 1 may perform a physique determination assuming that the occupant is in a state of poor posture.

[0135] For example, the physique determination device 1 may determine the posture of an occupant, and if it determines that the occupant is experiencing poor posture, it may not perform a physique determination on the occupant, or may reject the physique determination result and not output the physique determination result to various devices. For example, in the physique determination device 1, the second feature amount calculation unit 13 may first determine whether the occupant is experiencing poor posture. The second feature amount calculation unit 13 may determine whether the occupant is experiencing poor posture using a known posture determination method based on captured images. For example, the second feature amount calculation unit 13 may determine whether the occupant is experiencing poor posture based on the angle formed by the upper body (e.g., the neck and waist) (e.g., the angle formed by the line connecting the neck and waist and the perpendicular line from the neck). The second feature amount calculation unit 13 outputs information indicating that the occupant is experiencing poor posture (hereinafter referred to as "poor posture information") to the physique determination unit 14. The poor posture information includes information indicating that the occupant is experiencing poor posture. The posture error information may include information indicating the type of posture error the occupant is in, such as information indicating a forward-leaning state in the example described above. When the second feature amount calculation unit 13 outputs posture error information, the physique determination unit 14 may not perform a posture error determination or may reject the posture error determination result and not output the result to an external device. When the occupant is leaning forward, the occupant's body moves away from the seat back. Therefore, the second feature amount (such as the area of ​​the skeleton point area or the distance between skeleton points) that is a feature amount related to the occupant in the captured image may be larger than expected. For example, if the physique determination unit 14 determines the occupant's posture using the second feature amount, there is a possibility that the physique determination unit 14 may erroneously determine the occupant's posture. Therefore, when the second feature amount calculation unit 13 determines that the occupant is in a posture error, the physique determination unit 14 may not perform a posture error determination or may reject the posture error determination result and not output the result to an external device.

[0136] Furthermore, for example, the physique determination device 1 may determine the posture of an occupant, and if it determines that the occupant is experiencing poor posture, correct the skeletal coordinate points before determining the physique of the occupant. For example, in the physique determination device 1, the second feature amount calculation unit 13 first determines whether the occupant is experiencing poor posture. The second feature amount calculation unit 13 may determine whether the occupant is experiencing poor posture using a known posture determination method based on a captured image. Then, if it determines that the occupant is experiencing poor posture, the second feature amount calculation unit 13 corrects the skeletal coordinate points and calculates the second feature amount. For example, the second feature amount calculation unit 13 corrects the coordinates of the skeletal coordinate points on the captured image so that the occupant is sitting deep in the seat, in other words, so that the torso, which had slid diagonally forward, is raised and aligned with the seat back. For example, a transformation formula for determining how much to correct the coordinates of the skeletal coordinate points in the captured image depending on how much the torso is leaning in real space, as estimated from the skeletal coordinate points in the captured image, is generated in advance through learning and stored in a location accessible to the second feature amount calculation unit 13, such as a memory unit (not shown). The second feature amount calculation unit 13 can correct the coordinates of the skeletal coordinate points using the transformation formula generated in advance. The second feature amount calculation unit 13 then calculates the second feature amount using the corrected coordinates of the skeletal coordinate points or using the captured image in which the corrected coordinates of the skeletal coordinate points are reflected. If the occupant is sitting with their hips forward and shallow, the second feature amount in the vertical direction calculated from the skeletal coordinate points (such as the area of ​​the skeletal point region or the distance between skeletal points) may be smaller than expected. For example, if the physique determination unit 14 determines the physique of the occupant using the second feature amount calculated by the second feature amount calculation unit 13 without correcting the skeletal coordinate points, the physique determination unit 14 may erroneously determine the physique of the occupant. Therefore, the second feature calculation unit 13 may convert the coordinates of the skeletal coordinate points into coordinates of a position where the occupant is not losing their posture, and then calculate the second feature, thereby preventing the physique determination unit 14 from making an erroneous determination of the occupant's physique.

[0137] Among the posture imbalances, for example, a leaning-forward posture is estimated to be a posture that does not continue constantly. For example, even if the physique determination unit 14 does not perform a physique determination or rejects the physique determination result, this is temporary, and the physique determination unit 14 can determine the occupant's physique after the occupant is no longer leaning forward. On the other hand, for example, a shallow sitting posture is estimated to be a posture that continues constantly. In this case, if the physique determination unit 14 does not perform a physique determination or rejects the physique determination result, an event in which the occupant's physique determination is not performed may continue to occur. Therefore, in such a case, it is considered better to prevent the event in which the occupant's physique determination is not performed from continuing to occur by, for example, having the second feature amount calculation unit 13 correct the coordinates of the skeletal coordinate points. For example, the physique determination unit 14 may switch between not performing a physique determination or performing a physique determination after making a correction, depending on the state of the occupant's posture imbalance.

[0138] Furthermore, in the above-described first embodiment, the physique determination device 1 is an in-vehicle device mounted on a vehicle, and the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and a control unit (not shown) are provided in the physique determination device 1. However, without being limited to this, some of the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and the control unit (not shown) may be mounted in the in-vehicle device of the vehicle, and the rest may be provided in a server connected to the in-vehicle device via a network, and a physique determination system may be configured by the in-vehicle device and the server. In addition, the captured image acquisition unit 11, the first feature calculation unit 12, the second feature calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and a control unit (not shown) may all be provided on the server.

[0139] In the first embodiment described above, the subject is a vehicle occupant, and the physique determination device 1 determines the physique of the vehicle occupant. However, this is merely an example. For example, the subject may be a crew member of a ship or an airplane, and the physique determination device 1 may determine the physique of the occupant of a moving body other than a vehicle, such as a ship or an airplane. Furthermore, for example, the subject may be a person in a room, and the physique determination device 1 may determine the physique of the person in the room. The physique determination device 1 can determine the physique of various people seated in seats based on a first feature amount, which is a feature amount related to the contour of the seat in a captured image, and a second feature amount, which is a feature amount related to the subject in a captured image.

[0140] 20A and 20B are diagrams illustrating an example of the hardware configuration of a physique determination device 1 according to embodiment 1. In embodiment 1, the functions of the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and a control unit (not shown) are realized by a processing circuit 101. That is, the physique determination device 1 includes the processing circuit 101 for controlling the determination of the physique of a subject based on a captured image capturing at least a target area where the subject's upper body and the back of the seat on which the subject is seated should be located. The processing circuit 101 may be dedicated hardware as shown in FIG. 20A , or a processor 104 that executes a program stored in memory as shown in FIG. 20B .

[0141] When processing circuitry 101 is dedicated hardware, processing circuitry 101 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0142] When the processing circuit is the processor 104, the functions of the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and a control unit (not shown) are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 105. The processor 104 reads and executes the program stored in the memory 105 to execute the functions of the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and a control unit (not shown). In other words, the physique determination device 1 includes the memory 105 for storing a program that, when executed by the processor 104, results in the execution of steps ST1 to ST4 of FIG. 18 described above. In addition, the program stored in memory 105 can also be said to cause the computer to execute the processing procedures or methods of the image acquisition unit 11, the first feature calculation unit 12, the second feature calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and a control unit not shown. Here, the memory 105 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

[0143] Note that the functions of the captured image acquisition unit 11, the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and the control unit (not shown) may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the functions of the captured image acquisition unit 11 may be implemented by a processing circuit 101 as dedicated hardware, while the functions of the first feature amount calculation unit 12, the second feature amount calculation unit 13, the skeleton detection unit 131, the physique determination unit 14, the determination result output unit 15, and the control unit (not shown) may be implemented by a processor 104 reading and executing programs stored in a memory 105. The storage unit (not shown) may be, for example, the memory 105 or a HDD. The physique determination device 1 also includes an input interface device 102 and an output interface device 103 that communicate with devices such as the image capture device 2 via wired or wireless communication.

[0144] As described above, according to the first embodiment, the physique determination device 1 is configured to include: an image capture device 2 that captures an image of a target range in which the upper body of the subject and the back of the seat on which the subject is seated should be located; an image capture unit 11 that captures an image of the subject using the image capture device 2; a first feature amount calculation unit 12 that calculates a first feature amount, which is a feature amount in the image related to the contour of the seat, based on the image captured by the image capture unit 11; a second feature amount calculation unit 13 that calculates a second feature amount, which is a feature amount in the image related to the subject, based on the image captured by the image capture unit 11; and a physique determination unit 14 that determines the physique of the subject based on the first feature amount calculated by the first feature amount calculation unit 12 and the second feature amount calculated by the second feature amount calculation unit 13. Therefore, the physique determination device 1 can perform a physique determination that is not affected by the position of the seat in which the subject is seated.

[0145] More specifically, in the physique determination device 1, the physique determination unit 14 is configured to determine the physique of the subject based on the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and physique determination conditions that define the relationship between the first feature, the second feature, and the physique. This allows the physique determination device 1 to perform physique determination that is not affected by the position of the seat in which the subject is seated.

[0146] Furthermore, in the physique determination device 1, the physique determination unit 14 may determine the physique of the subject based on the first feature calculated by the first feature calculation unit 12, the second feature calculated by the second feature calculation unit 13, and a machine learning model that receives the first feature and the second feature and outputs information indicating the physique. This allows the physique determination device 1 to perform physique determination that is not affected by the position of the seat in which the subject is seated.

[0147] Furthermore, in the physique determination device 1, the first feature amount calculation unit 12 may detect the contour of the seat in the captured image, calculate the first feature amount based on the detected contour of the seat, and if the entire contour of the seat cannot be detected when the contour of the seat in the captured image is detected and the detected contour of the seat is equal to or greater than the first contour detection threshold, interpolate the undetected portion to detect the contour of the seat, and calculate the first feature amount based on the interpolated contour of the seat. This allows the physique determination device 1 to determine the physique of an occupant even if the entire contour of the seat cannot be detected in the captured image.

[0148] Furthermore, in the physique determination device 1, the first feature amount calculation unit 12 may detect the contour of the seat in the captured image and calculate the first feature amount based on the detected contour of the seat, and the physique determination unit 14 may determine that the physique of the subject is within a set standard physique if the first feature amount calculation unit 12 detects the contour of the seat in the captured image but is unable to detect a contour of the seat equal to or greater than the second contour detection threshold. In this way, even if the physique determination device 1 cannot detect most of the contour of the seat in the captured image, it can determine the physique of the subject by assuming, for example, that the subject has a physique large enough to obscure the contour of the seat.

[0149] In addition, in the present disclosure, any of the components of the embodiments may be modified or omitted.

[0150] The physique determination device according to the present disclosure can perform physique determination that is not affected by the position of the seat in which the subject is seated.

[0151] 1 physique determination device, 11 captured image acquisition unit, 12 first feature amount calculation unit, 13 second feature amount calculation unit, 131 skeleton detection unit, 14 physique determination unit, 15 determination result output unit, 101 processing circuit, 102 input interface device, 103 output interface device, 104 processor, 105 memory.

Claims

1. At a minimum, an imaging device that images the area in which the upper body of the subject and the backrest of the seat in which the subject is seated should be located, and an image acquisition unit that acquires an image of the subject captured by the imaging device, A first feature calculation unit calculates a first feature quantity, which is a feature quantity on the captured image relating to the contour of the seat, based on the captured image acquired by the image acquisition unit. A second feature calculation unit calculates a second feature quantity, which is a feature quantity on the captured image relating to the subject, based on the captured image acquired by the image acquisition unit. The system includes a body size determination unit that determines the body size of the subject based on the first feature calculated by the first feature calculation unit and the second feature calculated by the second feature calculation unit, The aforementioned body size determination unit, Based on the first feature calculated by the first feature calculation unit, the second feature calculated by the second feature calculation unit, and the body size determination conditions defined for the relationship between the first feature, the second feature, and body size, the body size of the subject is determined. In the aforementioned conditions for determining body size, the ratio of the second feature to the first feature and the body size corresponding to that ratio are defined. The first feature calculation unit calculates the area of ​​the seat region enclosed by the contour of the seat in the captured image as the first feature, The second feature calculation unit detects a plurality of skeletal coordinate points of the subject that indicate parts of the subject's body on the captured image, and calculates the area of ​​the skeletal point region formed by three or more of the detected plurality of skeletal coordinate points as the second feature quantity. The body size determination unit determines the body size of the subject based on the ratio of the area of ​​the skeletal point area to the area of ​​the sheet area. A body size determination device characterized by the following features.

2. The first feature calculation unit described above is: The contour of the seat on the captured image is detected, and the first feature quantity is calculated based on the detected contour of the seat. When detecting the contour of the seat in the captured image, if the entire contour of the seat cannot be detected, and the detected contour of the seat is equal to or greater than the first contour detection threshold, the undetected portion is interpolated to detect the contour of the seat, and the first feature quantity is calculated based on the interpolated contour of the seat. The body size determination device according to claim 1.

3. The first feature calculation unit described above is: The contour of the seat on the captured image is detected, and the first feature quantity is calculated based on the detected contour of the seat. The aforementioned body size determination unit, When the first feature calculation unit detects the contour of the seat in the captured image, if it is unable to detect a contour of the seat that is equal to or greater than the second contour detection threshold in the captured image, the unit determines the subject's physique to be a set standard physique. The body size determination device according to claim 1.

4. The aforementioned subject is a vehicle occupant. A body size determination device according to any one of claims 1 to 3, characterized in that

5. The image acquisition unit acquires an image of the subject using an imaging device that images the area in which at least the upper body of the subject and the backrest of the seat in which the subject is seated should be located, The first feature calculation unit calculates a first feature, which is a feature on the captured image relating to the contour of the seat, based on the captured image acquired by the image acquisition unit. The second feature calculation unit calculates a second feature, which is a feature on the captured image relating to the subject, based on the captured image acquired by the image acquisition unit. The body size determination unit includes the step of determining the body size of the subject based on the first feature calculated by the first feature calculation unit and the second feature calculated by the second feature calculation unit, The aforementioned body size determination unit, Based on the first feature calculated by the first feature calculation unit, the second feature calculated by the second feature calculation unit, and the body size determination conditions defined for the relationship between the first feature, the second feature, and body size, the body size of the subject is determined. In the aforementioned conditions for determining body size, the ratio of the second feature to the first feature and the body size corresponding to that ratio are defined. The first feature calculation unit calculates the area of ​​the seat region enclosed by the contour of the seat in the captured image as the first feature, The second feature calculation unit detects a plurality of skeletal coordinate points of the subject that indicate parts of the subject's body on the captured image, and calculates the area of ​​the skeletal point region formed by three or more of the detected plurality of skeletal coordinate points as the second feature quantity. The body size determination unit determines the body size of the subject based on the ratio of the area of ​​the skeletal point area to the area of ​​the sheet area. A method for determining body size characterized by the following features.