Body weight estimation device, body weight estimation method and body weight estimation program

Abstract

To provide a body weight estimation device, a body weight estimation method, and a body weight estimation program, capable of accurately estimating body weight of a crew member from a captured image capturing the crew member.SOLUTION: An image acquisition unit 20 acquires a captured image captured by a camera 14 and including a result obtained by measuring a distance in an image pixel unit, a person area acquisition unit 22 detects a crew member from the acquired captured image to acquire an image area including the crew member, a three-dimensional shape estimation unit 24 estimates a three-dimensional shape of the crew member from the image area of each crew member acquired by the person area acquisition unit 22, and a body weight estimation unit 26 determines a volume of the three-dimensional shape and determines body weight by multiplying the determined volume by density data prepared in advance.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a body weight estimation device, a body weight estimation method, and a body weight estimation program. [Background technology]

[0002] Patent document 1 proposes an information acquisition device that includes a skeleton estimation unit that estimates the skeleton of a living body, an information acquisition unit that acquires attribute data including the height of the living body, and a calculation unit that subtracts a specified value from the height to calculate a first length corresponding to a first skeleton that constitutes part of the height, and calculates the length of a target part of the living body using the ratio of skeletons that include the first skeleton in the estimated skeleton and the first length.

[0003] Patent document 2 proposes a weight estimation device for estimating the weight of an animal, which includes an information acquisition device that images an animal and acquires three-dimensional information at each of multiple points, a calculation device that calculates the geometric quantities of the animal's external shape based on the three-dimensional information acquired by the information acquisition device, and a weighing device that estimates the weight of the imaged animal based on the geometric quantities of the animal's external shape calculated by the calculation device.

[0004] Patent document 3 proposes a weight estimation device having an acquisition means for acquiring a first image of an object whose weight is to be estimated, a classification means for classifying the posture of the object in the first image acquired by the acquisition means into one of a plurality of predefined postures, and an estimation means for estimating the weight of the object from the first image using an estimation model corresponding to the posture classified by the classification means.

[0005] Patent document 4 proposes a vehicle occupant detection system to be mounted on a vehicle that uses a camera to detect the three-dimensional surface shape of a vehicle occupant from a single viewpoint, detects the distance between specific parts of the vehicle occupant and the positions of specific parts, and thereby detects the physique and posture of the vehicle occupant.

[0006] Patent Document 5 proposes a physique determination device that includes an image acquisition unit that acquires an image of an occupant seated in a vehicle seat captured by an imaging device, a position information calculation unit that calculates shoulder joint information that indicates the shoulder joint position of the occupant in the image, a twist determination unit that determines whether the occupant is in a twisted posture based on the shoulder joint information, and a physique determination unit that determines the physique of the occupant based on the shoulder joint information. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 7346701 [Patent Document 2] Patent No. 7284500 [Patent Document 3] Japanese Patent Application Publication No. 2023-129574 [Patent Document 4] Japanese Patent Application Laid-Open No. 2008-2838 [Patent Document 5] Patent No. 7287239 Summary of the Invention [Problem to be solved by the invention]

[0008] However, in the technology described in Patent Document 1, attribute data such as the height and weight of the occupant is registered in advance, so that the attribute data needs to be registered every time the occupant changes, which is not practical.

[0009] Furthermore, the technology described in Patent Document 2 uses a regression equation to predict an animal's weight from its height estimated from an image. However, when the subject is an animal and is applied to humans, although there is a certain degree of correlation between height and weight, there is room for improvement in order to accurately estimate weight.

[0010] Furthermore, the technology described in Patent Document 3 uses a regression equation classified by posture to predict body weight from partial length information such as the animal's torso length and hip width. However, when the subject is an animal and is applied to humans, there is room for improvement in order to estimate body weight as accurately as in Patent Document 2.

[0011] Furthermore, the technology described in Patent Document 4 is a technology that obtains the length of each body part by using a 3D point cloud of the body surface as a measurement value from an image captured by a 3D camera, but does not estimate weight.

[0012] Furthermore, the technology described in Patent Document 5 distinguishes between an adult's body type and a child's body type from a captured image of an occupant, but there is room for improvement in order to estimate weight accurately.

[0013] The present disclosure has been made in consideration of the above facts, and aims to provide a weight estimation device, a weight estimation method, and a weight estimation program that can accurately estimate the weight of an occupant from an image of the occupant. [Means for solving the problem]

[0014] In order to achieve the above object, the weight estimation device according to the first aspect includes an acquisition unit that detects occupants from a captured image of the interior of the vehicle and acquires an image area surrounding the occupant, an estimation unit that estimates the three-dimensional shape or shape parameters of each occupant in the vehicle interior from the image area of ​​the occupant, and a weight estimation unit that estimates the weight of each occupant from the estimation result of the estimation unit.

[0015] According to the first aspect, the acquisition unit detects occupants from a captured image of the vehicle interior and acquires an image area surrounding the occupant. The estimation unit estimates the three-dimensional shape or shape parameters of each occupant from the image area of ​​each occupant in the vehicle interior. The weight estimation unit estimates the weight of each occupant from the estimation result of the estimation unit. In this way, by estimating the weight of each occupant from the three-dimensional shape or shape parameters, the weight of the occupant can be accurately estimated.

[0016] A body weight estimation device according to a second aspect is the body weight estimation device according to the first aspect, wherein the captured image is an image captured by an imaging unit that captures an image of the interior of a vehicle and is capable of measuring distance.

[0017] According to the second aspect, it is possible to estimate a three-dimensional shape or shape parameters from a captured image.

[0018] The weight estimation device of the third aspect is the weight estimation device of the first aspect, wherein when the estimation unit estimates the three-dimensional shape of each occupant from the image area of ​​each occupant in the vehicle cabin, the estimation is performed using a shape model that can express the human body shape and posture as separate parameters.

[0019] According to the third aspect, it is possible to estimate the three-dimensional shape of an occupant by using a shape model that can express the human body shape and posture as separate parameters.

[0020] A weight estimation device according to a fourth aspect is the weight estimation device according to the third aspect, wherein when the estimation unit estimates the shape parameters, the estimation unit constructs a three-dimensional shape of a predetermined posture from the shape parameters, and the weight estimation unit estimates the weight of each occupant from density data of a human body prepared in advance and the volume of the three-dimensional shape constructed by the estimation unit.

[0021] According to the fourth aspect, by constructing a three-dimensional shape in a predetermined posture from shape parameters, it becomes possible to estimate the weight of the occupant from the density data of the human body and the volume of the three-dimensional shape.

[0022] A weight estimation device according to a fifth aspect is the weight estimation device according to the first aspect, wherein the estimation unit estimates the shape parameters, and the weight estimation unit estimates the weight of each occupant using a formula that is calculated in advance and that estimates weight from the shape parameters.

[0023] According to the fifth aspect, it is possible to estimate the weight of an occupant without preparing density data of the human body in advance.

[0024] In a weight estimation method according to a sixth aspect, a computer detects occupants from an image captured of the interior of a vehicle, acquires an image area surrounding the occupant, estimates the three-dimensional shape or shape parameters of each occupant from the image area of ​​the occupant in the vehicle, and performs a process of estimating the weight of each occupant from the estimated three-dimensional shape or shape parameters.

[0025] According to the sixth aspect, it is possible to provide a weight estimation method that can accurately estimate the weight of an occupant from a captured image of the occupant.

[0026] A weight estimation program according to a seventh aspect causes a computer to execute a process of detecting occupants from an image captured of the interior of a vehicle, acquiring an image area surrounding the occupant, estimating a three-dimensional shape or shape parameters of each occupant from the image area of ​​the occupant in the vehicle, and estimating the weight of each occupant from the estimated three-dimensional shape or shape parameters.

[0027] According to the seventh aspect, it is possible to provide a weight estimation program that can accurately estimate the weight of an occupant from a captured image of the occupant. [Effects of the Invention]

[0028] As described above, the present invention can provide a weight estimation device, a weight estimation method, and a weight estimation program that can accurately estimate the weight of an occupant from a captured image of the occupant. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a diagram showing a vehicle equipped with an occupant weight estimation device according to an embodiment of the present invention; [Figure 2] 1 is a block diagram showing a schematic configuration of an occupant weight estimation device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a functional block diagram showing the functional configuration of an occupant weight estimation device according to the first and second embodiments. [Figure 4] 5 is a flowchart showing an example of the flow of processing performed in the occupant weight estimation device according to the first and second embodiments. [Figure 5] FIG. 10 is a functional block diagram showing the functional configuration of an occupant weight estimation device according to a third embodiment and a fourth embodiment. [Figure 6] 10 is a flowchart showing an example of the flow of processing performed in an occupant weight estimation device according to a third embodiment and a fourth embodiment. [Figure 7] 10 is a flowchart showing an example of the flow of processing performed in an occupant weight estimation device according to a fifth embodiment. [Figure 8] 10 is a flowchart showing a part of a processing flow when a monocular camera is used as the camera. DETAILED DESCRIPTION OF THE INVENTION

[0030] An example of an embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a diagram showing a vehicle equipped with an occupant weight estimation device according to this embodiment, and Fig. 2 is a block diagram showing a schematic configuration of the occupant weight estimation device according to this embodiment.

[0031] The occupant weight estimation device 10 is mounted on a vehicle 12 and estimates the weight of an occupant riding in the passenger compartment of the vehicle 12. The occupant weight estimation device 10 according to this embodiment can estimate the occupant weight without providing a load sensor on the seat. The occupant weight estimated by the occupant weight estimation device 10 is used, for example, to distinguish between adults and children when controlling an airbag. For example, the occupant weight estimation device 10 is mounted on the vehicle 12 as an occupant state estimation ECU (Electronic Control Unit).

[0032] The vehicle 12 is provided with a camera 14, which photographs occupants in the vehicle cabin and outputs the photographed image to the occupant weight estimation device 10. The camera 14 is disposed in a position suitable for photographing occupants in the vehicle cabin. For example, it is disposed in approximately the center of the upper part of the windshield glass. Note that the camera 14 is preferably a wide-angle camera in order to photograph all occupants in the vehicle cabin. Furthermore, in order to function at night, an infrared camera equipped with a near-infrared light source (for example, an LED (Light Emitting Diode)) such as those used in general driver monitors is preferable. Note that the number of cameras 14 is not limited to one, and multiple cameras may be used. In the case of multiple cameras, they may be disposed in different positions.

[0033] In addition, in this embodiment, in order to acquire distance information about the occupant, a camera capable of measuring distance in units of image pixels, such as a parallel stereo camera or a TOF (Time of Flight) camera, is used. Note that for parallel stereo cameras, the technology described in "https: / / www.ieice-hbkb.org / files / ad_base / view_pdf.html?p= / files / 02 / 02gun_02hen_03.pdf#page=9" can be applied. Furthermore, as a technology capable of measuring distance in units of image pixels, well-known technologies such as "Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation," CVPR2022. and "A Learned Stereo Depth System for Robotic Manipulation in Homes," IEEE Robotics and Automation Letters 2022. can be applied.

[0034] As shown in FIG. 2, the occupant weight estimation device 10 according to this embodiment is configured with a general microcomputer including a CPU (Central Processing Unit) 10A, a ROM (Read Only Memory) 10B, a RAM (Random Access Memory) 10C, a storage 10D, an interface (I / F) 10E, and a bus 10F.

[0035] A camera 14 is connected to the I / F 10E, and an image captured by the camera 14 is input to the occupant weight estimation device 10.

[0036] (First embodiment) FIG. 3 is a functional block diagram showing the functional configuration of the occupant weight estimation device 10 according to this embodiment.

[0037] The occupant weight estimation device 10 of this embodiment functions as an image acquisition unit 20, a person area acquisition unit 22 as an example of an acquisition unit, a three-dimensional shape estimation unit 24 as an example of an estimation unit, and a weight estimation unit 26, by the CPU 10A expanding the occupant weight estimation program stored in the ROM 10B into the RAM 10C and executing it.

[0038] The image acquisition unit 20 acquires from the camera 14 an image captured by the camera 14. The acquired image includes a result of measuring distance in image pixel units.

[0039] The person area acquisition unit 22 detects the occupant from the acquired captured image and acquires the image area surrounding the occupant. For example, well-known techniques such as "YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors," CVPR2023, and "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," PAMI2017 can be applied.

[0040] The 3D shape estimation unit 24 estimates the 3D shapes of the occupants from the image areas of each occupant acquired by the person area acquisition unit 22. The 3D shape estimation unit 24 estimates the 3D shapes of the occupants by applying well-known techniques such as "DensePose: Dense Human Pose Estimation in the Wild," CVPR 2018, and "Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies," CVPR 2018.

[0041] In this case, the image coordinates of body feature points consisting of the positions of each part of the face and the positions of the body joints can be obtained by the method described in the aforementioned Patent Publication No. 7287239, and the 3D coordinates of these body feature points as seen from the camera can be obtained by the method of measuring distance in image pixel units described above. Using this 3D coordinate information enables more accurate estimation of the 3D human body shape.

[0042] The weight estimation unit 26 calculates the volume of the three-dimensional shape and calculates the weight by multiplying the calculated volume by density data prepared in advance. The volume of the three-dimensional shape can be calculated, for example, by "Method for calculating the volume of a 3D model": https: / / elosove.com / ?p=208. The "prepared human body density data" can be calculated, for example, by "Body Composition of Japanese Adult Men and Women by Density Method," Physical Fitness Science 1993, which states that the average density of the human body is approximately 1.02 to 1.07 g / cm. 3 Therefore, for example, the median value of 1.045 g / cm 3 Using this method, it is possible to estimate with an error of less than 3%.

[0043] Next, a specific process performed by the occupant weight estimation device 10 according to this embodiment configured as described above will be described. Fig. 4 is a flowchart showing an example of the flow of the process performed by the occupant weight estimation device 10 according to this embodiment. The process in Fig. 4 starts, for example, when an ignition switch (not shown) of the vehicle 12 is turned on.

[0044] In step 100, CPU 10A acquires an image of the interior of the vehicle, and the process proceeds to step 102. That is, image acquisition unit 20 acquires, from camera 14, an image captured by camera 14.

[0045] In step 102, CPU 10A detects an occupant and proceeds to step 104. That is, person area acquisition unit 22 detects an occupant from the captured image acquired by camera 14.

[0046] In step 104, CPU 10A determines whether an occupant has been detected. This determination is made by determining whether person area acquisition unit 22 has detected an occupant from the captured image. If the determination is negative, the process returns to step 100 to repeat the above-described process. If the determination is positive, the process proceeds to step 106.

[0047] In step 106, CPU 10A acquires the image area of ​​each occupant, and the process proceeds to step 108. That is, person area acquisition unit 22 acquires the image area surrounding the detected occupant.

[0048] In step 108, CPU 10A estimates the three-dimensional human body shape, and proceeds to step 110. That is, three-dimensional shape estimation unit 24 estimates the three-dimensional shape of each occupant from the image area of ​​each occupant acquired by person area acquisition unit 22.

[0049] In step 110, CPU 10A acquires general human body density data, and the process proceeds to step 112. The general human body density data is stored in advance in storage 10D or the like.

[0050] In step 112, CPU 10A estimates the weight of each occupant and ends the series of processes. That is, weight estimation unit 26 calculates the volume of the three-dimensional shape and calculates the weight by multiplying the calculated volume by density data prepared in advance.

[0051] By performing the processing in this manner, the occupant weight estimation device 10 according to this embodiment can accurately estimate the occupant weight without providing a load sensor on the seat.

[0052] (Second embodiment) This embodiment clarifies that the imaging unit is capable of distance measurement, which, as described above, makes it possible to estimate the three-dimensional human body shape more accurately by using three-dimensional coordinate information.

[0053] (Third embodiment) Next, an occupant weight estimation device according to a third embodiment will be described. The basic configuration is the same as that of the first embodiment, but the functional configuration is different from that of the first embodiment. Fig. 5 is a functional block diagram showing the functional configuration of an occupant weight estimation device 11 according to this embodiment. Note that functions common to the first embodiment are assigned the same reference numerals and will not be described again.

[0054] 5, the occupant weight estimation device 11 according to this embodiment has the functions of a shape parameter estimation unit 23 and a three-dimensional shape construction unit 25 instead of the three-dimensional shape estimation unit 24. The shape parameter estimation unit 23 and the three-dimensional shape construction unit 25 correspond to an example of an estimation unit.

[0055] The shape parameter estimation unit 23 estimates the shape parameters using a technology for estimating a "shape model capable of expressing the human body shape and posture as separate parameters." For example, the technology described in "SMPL: A Skinned Multi-Person Linear Model," SIGGRAPH ASIA 2015, can be applied. SMPL is a model that expresses the basic three-dimensional shape of the human body (height, body mass, etc.) using 10 shape parameters and expresses the degree of bending of the joint points of the arms and legs using 72 pose parameters.

[0056] The 3D shape construction unit 25 constructs a 3D shape of a predetermined pose from the shape parameters estimated by the shape parameter estimation unit 23. The 3D shape construction unit 25 can construct a 3D shape of a predetermined pose using, for example, the program "SMPL layer for PyTorch" (https: / / github.com / hongsukchoi / Pose2Mesh_RELEASE / blob / master / smplpytorch / demo.py). This program outputs an SMPL 3D human body mesh when given 10 shape parameters and 72 pose parameters. The 10 shape parameters are those estimated by the shape parameter estimation unit 23, and the 72 pose parameters are parameters that create a predetermined pose that has been calculated in advance. For example, a T-shaped pose can be used as the predetermined pose.

[0057] Next, specific processing performed by the occupant weight estimation device 11 according to this embodiment configured as described above will be described. Fig. 6 is a flowchart showing an example of the flow of processing performed by the occupant weight estimation device 11 according to this embodiment. The processing in Fig. 6 starts, for example, when an ignition switch (not shown) of the vehicle 12 is turned on. The same processing as in Fig. 4 will be described with the same reference numerals.

[0058] In step 100, CPU 10A acquires an image of the interior of the vehicle, and the process proceeds to step 102. That is, image acquisition unit 20 acquires, from camera 14, an image captured by camera 14.

[0059] In step 102, CPU 10A detects an occupant and proceeds to step 104. That is, person area acquisition unit 22 detects an occupant from the captured image acquired by camera 14.

[0060] In step 104, CPU 10A determines whether an occupant has been detected. This determination is made by determining whether person area acquisition unit 22 has detected an occupant from the captured image. If the determination is negative, the process returns to step 100 to repeat the above-described process. If the determination is positive, the process proceeds to step 106.

[0061] In step 106, CPU 10A acquires the image area of ​​each occupant, and the process proceeds to step 107. That is, person area acquisition unit 22 acquires the image area surrounding the detected occupant.

[0062] In step 107, CPU 10A estimates shape parameters and proceeds to step 109. That is, shape parameter estimation unit 23 estimates shape parameters using a technique for estimating a "shape model that can express the human body shape and posture as separate parameters."

[0063] In step 109, CPU 10A constructs a three-dimensional shape of a predetermined orientation from the shape parameters, and proceeds to step 110. That is, three-dimensional shape construction unit 25 constructs a three-dimensional shape of a predetermined orientation from the shape parameters estimated by shape parameter estimation unit 23.

[0064] In step 110, CPU 10A acquires general human body density data, and the process proceeds to step 112. The general human body density data is stored in advance in storage 10D or the like.

[0065] In step 112, CPU 10A estimates the weight of each occupant and ends the series of processes. That is, weight estimation unit 26 calculates the volume of the three-dimensional shape and calculates the weight by multiplying the calculated volume by density data prepared in advance.

[0066] Even when processing is performed in this manner, the occupant weight estimation device 11 according to this embodiment can accurately estimate the occupant weight without providing a load sensor on the seat.

[0067] (Fourth embodiment) This embodiment clarifies that in the weight estimation device of the third aspect, a three-dimensional shape of a predetermined posture is constructed by a three-dimensional shape construction unit 25 from shape parameters estimated by a shape parameter estimation unit 23, and a weight estimation unit 26 estimates the weight of each occupant from human body density data prepared in advance and the volume of the three-dimensional shape constructed by the three-dimensional shape construction unit 25.

[0068] (Fifth embodiment) Next, an occupant weight estimation device according to a fifth embodiment will be described. Note that this embodiment is a modified example of the fourth embodiment.

[0069] In the fourth embodiment, the weight of the occupant was estimated using density data of a general human body that was prepared in advance, but in this embodiment, the weight is estimated from the shape parameters using an estimation equation that is obtained in advance.

[0070] In this embodiment, the weight estimation unit 26 estimates the weight of the occupant from the shape parameters estimated by the shape parameter estimation unit 23 using an estimation formula that is obtained in advance and that estimates from the shape parameters.

[0071] The "predetermined relationship between shape parameters and body weight" can be derived by the following procedure. [Preparing training data] 1. Create a vector of shape parameters using random numbers and call it Sk. 2. Sk generates a 3D mesh shape, for example, in a T-shape, and performs the method described in the first embodiment.

[0072] The weight Wk is calculated using the process shown in the flowchart of Figure 4. 3. Repeat steps 1 and 2 to create a sufficient number of combinations of Sk and Wk. [Derivation of the relational expression] 1. Multiple regression analysis can be used to obtain an estimation formula for Wk from Sk.

[0073] Next, specific processing performed by the occupant weight estimation device according to this embodiment will be described. Fig. 7 is a flowchart showing an example of the flow of processing performed by the occupant weight estimation device according to this embodiment. The processing in Fig. 7 starts, for example, when an ignition switch (not shown) of the vehicle 12 is turned on. The same processing as in Fig. 6 will be described with the same reference numerals.

[0074] In step 100, CPU 10A acquires an image of the interior of the vehicle, and the process proceeds to step 102. That is, image acquisition unit 20 acquires, from camera 14, an image captured by camera 14.

[0075] In step 102, CPU 10A detects an occupant and proceeds to step 104. That is, person area acquisition unit 22 detects an occupant from the captured image acquired by camera 14.

[0076] In step 104, CPU 10A determines whether an occupant has been detected. This determination is made by determining whether person area acquisition unit 22 has detected an occupant from the captured image. If the determination is negative, the process returns to step 100 to repeat the above-described process. If the determination is positive, the process proceeds to step 106.

[0077] In step 106, CPU 10A acquires the image area of ​​each occupant, and the process proceeds to step 108. That is, person area acquisition unit 22 acquires the image area surrounding the detected occupant.

[0078] In step 107, CPU 10A estimates shape parameters and proceeds to step 111. That is, shape parameter estimation unit 23 estimates shape parameters using a technique for estimating a "shape model that can express the human body shape and posture as separate parameters."

[0079] In step 111, CPU 10A estimates the weight from the shape parameters using the estimation formula obtained in advance, and then ends the series of processes. That is, weight estimation unit 26 estimates the weight of the occupant from the shape parameters estimated by shape parameter estimation unit 23 using the estimation formula obtained in advance for estimation from shape parameters.

[0080] By performing the processing in this manner, the weight of the occupant can be estimated with high accuracy without preparing general human body density data in advance or providing a load sensor on the seat.

[0081] In the above embodiments, the camera 14 is capable of measuring distance, but a camera that cannot measure distance, such as a monocular camera, may also be used. In this case, the process shown in FIG. 8 can be performed.

[0082] Fig. 8 is a flowchart showing part of the processing flow when a monocular camera is used as camera 14. When applied to the first embodiment, the processing in Fig. 8 is performed instead of steps 100 to 108 in Fig. 4. When applied to the second embodiment, it is performed instead of steps 100 to 109 in Fig. 6. When applied to the third embodiment, it is performed instead of steps 100 to 108 in Fig. 7. The same processes as in Fig. 4 will be described with the same reference numerals.

[0083] In step 100, CPU 10A acquires an image of the interior of the vehicle, and the process proceeds to step 102. That is, image acquisition unit 20 acquires, from camera 14, an image captured by camera 14.

[0084] In step 102, CPU 10A detects an occupant and proceeds to step 104. That is, person area acquisition unit 22 detects an occupant from the captured image acquired by camera 14.

[0085] In step 104, CPU 10A determines whether an occupant has been detected. This determination is made by determining whether person area acquisition unit 22 has detected an occupant from the captured image. If the determination is negative, the process returns to step 100 to repeat the above-described process. If the determination is positive, the process proceeds to step 106.

[0086] In step 106, CPU 10A acquires the image area of ​​each occupant, and proceeds to step 114. That is, person area acquisition unit 22 acquires the image area surrounding the detected occupant.

[0087] In step 114, CPU 10A acquires a three-dimensional human body shape estimator or a shape parameter estimator that has been trained in advance using images with depth information, and the process proceeds to step .

[0088] In step 116, CPU 10A estimates the three-dimensional human body shape or shape parameters, returns the process, and performs the processes of the first to third embodiments.

[0089] In this way, by using an estimator that has been trained in advance using images with distance information, it is possible to estimate the weight of an occupant even with a camera that cannot measure distance.

[0090] In the above embodiment, the occupant weight estimation devices 10, 11 mounted on the vehicle 12 as an occupant state estimation ECU (Electronic Control Unit) have been described as an example, but the present invention is not limited to this and may be applied to a weight estimation device installed in another location, such as inside the vehicle.

[0091] Furthermore, although the processing performed by the occupant weight estimation device 10 in the above embodiment has been described as software processing performed by a computer executing a program, it may also be processing performed by hardware. Alternatively, it may be processing that combines both software and hardware. Furthermore, when processing is performed by software, the program may be stored in various storage media and distributed. Furthermore, although the program has been described as being stored in a ROM, this is not a limitation. For example, the program may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. Furthermore, the program may also be downloaded from an external device via a network.

[0092] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

[0093] The present disclosure may employ the following aspects. (1) an acquisition unit that detects an occupant from a captured image of the interior of the vehicle and acquires an image area surrounding the occupant; an estimation unit that estimates a three-dimensional shape or shape parameters of each occupant from the image area of ​​each occupant in the vehicle interior; a weight estimation unit that estimates the weight of each occupant from the estimation result of the estimation unit; A weight estimation device comprising:

[0094] (2) The weight estimation device according to (1), wherein the captured image is an image captured by a photographing unit that photographs the interior of a vehicle and is capable of measuring distance.

[0095] (3) The weight estimation device according to (1) or (2), wherein the estimation unit estimates the three-dimensional shape of each occupant from the image area of ​​the vehicle cabin using a shape model that can express the human body shape and posture as separate parameters.

[0096] (4) When estimating the shape parameters, the estimation unit constructs a three-dimensional shape of a predetermined posture from the shape parameters; The weight estimation device according to (3), wherein the weight estimation unit estimates the weight of each occupant from density data of a human body prepared in advance and the volume of the three-dimensional shape constructed by the estimation unit.

[0097] (5) the estimation unit estimates the shape parameters; The weight estimation device according to (1) or (2), wherein the weight estimation unit estimates the weight of each occupant using a formula for estimating weight from the shape parameters, which formula is obtained in advance.

[0098] (6) The computer Detecting an occupant from a captured image of the interior of the vehicle and acquiring an image area surrounding the occupant; estimating a three-dimensional shape or shape parameters of each occupant in the vehicle interior from the image region of the occupant; a weight estimation method for estimating the weight of each occupant from the estimation result of the three-dimensional shape or the shape parameters;

[0099] (7) On the computer, Detecting an occupant from a captured image of the interior of the vehicle and acquiring an image area surrounding the occupant; estimating a three-dimensional shape or shape parameters of each occupant in the vehicle interior from the image region of the occupant; a weight estimation program for executing a process of estimating the weight of each occupant from the estimation result of the three-dimensional shape or the shape parameters; [Explanation of symbols]

[0100] 10, 11 Occupant weight estimation device 14 Camera 20 Image acquisition unit 22 Person area acquisition part 23 Shape parameter estimation unit 24 3D shape estimation section 25 3D shape configuration part 26 Weight Estimation Section

Claims

1. an acquisition unit that detects an occupant from a captured image of the interior of the vehicle and acquires an image area surrounding the occupant; an estimation unit that estimates a three-dimensional shape or shape parameters of each occupant from the image area of ​​each occupant in the vehicle interior; a weight estimation unit that estimates the weight of each occupant from the estimation result of the estimation unit; A weight estimation device comprising:

2. 2. The weight estimation device according to claim 1, wherein the captured image is an image captured by a camera that captures an image of the interior of a vehicle and is capable of measuring distance.

3. 2. The weight estimation device according to claim 1, wherein the estimation unit estimates the three-dimensional shape of each occupant from the image area of ​​each occupant in the vehicle cabin using a shape model that can express human body shape and posture as separate parameters.

4. When estimating the shape parameters, the estimation unit constructs a three-dimensional shape of a predetermined posture from the shape parameters; 4. The weight estimation device according to claim 3, wherein the weight estimation unit estimates the weight of each occupant from density data of a human body prepared in advance and the volume of the three-dimensional shape constructed by the estimation unit.

5. the estimation unit estimates the shape parameters; The weight estimation device according to claim 1 , wherein the weight estimation unit estimates the weight of each of the occupants using a formula for estimating weight from the shape parameters, which formula is obtained in advance.

6. The computer Detecting an occupant from a captured image of the interior of the vehicle and acquiring an image area surrounding the occupant; estimating a three-dimensional shape or shape parameters of each occupant in the vehicle interior from the image region of the occupant; a weight estimation method for estimating the weight of each occupant from the estimation result of the three-dimensional shape or the shape parameters;

7. On the computer, Detecting an occupant from a captured image of the interior of the vehicle and acquiring an image area surrounding the occupant; estimating a three-dimensional shape or shape parameters of each occupant in the vehicle interior from the image region of the occupant; a weight estimation program for executing a process of estimating the weight of each occupant from the estimation result of the three-dimensional shape or the shape parameters;

Images