Physical information acquisition device

Abstract

To accurately acquire body information of a person.SOLUTION: A control device 10 as a body information acquisition device 20 includes a skeleton point detection unit 13 which detects a skeleton point SP of the person included in a photographed image Vd. The control device 10 includes a body information acquisition unit 15 which acquires the body information IB of the person on the basis of detection of the skeleton point SP. The control device 10 further includes an imaging state determination unit 27 which determines whether or not the imaging state VS of the person captured in the photographed image Vd is corresponding to the specific imaging state specified on the basis of a predetermined evaluation index value. The body information acquisition unit 15 does not execute acquisition of the body information IB on the basis of the detection of the skeleton point SP when the imaging state is corresponding to the specific imaging state.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a physical information acquisition device. [Background technology]

[0002] Conventionally, there are physical information acquisition devices that acquire physical information of a person captured in a photographed image by detecting the skeletal points of the person. For example, an occupant information determination device described in Patent Document 1 captures an image of a vehicle occupant and detects the head of the occupant, which is the skeletal point. Furthermore, this occupant information determination device detects the size of the head as a body dimension based on the skeletal point. As a result, the physique and posture of the occupant are determined as physical information of the captured image of the occupant.

[0003] In addition to detecting the skeleton points, this conventional example also detects the seat load of the occupant sitting in the vehicle seat. If the skeleton points cannot be detected from the captured image, for example, when the camera's field of view is blocked or the occupant's body is out of the field of view, the occupant's physical build and posture are determined based on the seat load. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-189101 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, even if an image can be captured, the accuracy of detecting skeleton points based on the captured image may be reduced, resulting in a problem that physical information of the target person cannot be obtained with high accuracy. [Means for solving the problem]

[0006] A physical information acquisition device that solves the above problem comprises a skeleton point detection unit that detects skeletal points of a person included in a captured image, a physical information acquisition unit that acquires physical information of the person based on the detection of the skeletal points, and an imaging state determination unit that determines whether the imaging state of the person shown in the captured image falls under a specific imaging state identified based on a predetermined evaluation index value, and if the imaging state falls under the specific imaging state, the physical information acquisition unit does not execute acquisition of the physical information based on the detection of the skeleton points.

[0007] According to the above configuration, a specific imaging state is set in a state where the accuracy of detecting skeleton points is reduced due to the imaging state of the person in the captured image, and in such a case, the detection of skeleton points by image analysis is not used to obtain physical information, thereby avoiding the occurrence of erroneous determination and enabling accurate acquisition of physical information, such as the physique and posture, of the person in the captured image.

[0008] A physical information acquisition device that solves the above problem preferably includes a body movement waveform generation unit that generates a body movement waveform of the person, and the imaging state determination unit determines that the specific imaging state exists when the magnitude of the body movement represented in the body movement waveform is equal to or greater than a predetermined body movement threshold value as the evaluation index value of the imaging state.

[0009] That is, when the "magnitude of body movement" of a person in a captured image is large, the accuracy of detecting skeleton points by image analysis may decrease. In this regard, with the above-mentioned configuration, a state in which the "magnitude of body movement" is large is identified from the waveform of the person's body movement. Then, by setting this state as a specific imaging state and not using the detection of skeleton points by image analysis to obtain physical information, the occurrence of misidentification can be suppressed.

[0010] Furthermore, by detecting larger body movements, such as leaning forward or twisting the upper body, which may cause the person to take an extreme posture and thus move out of the camera's field of view, as a sign of body movement, it is possible to prevent the associated deterioration in judgment accuracy and the occurrence of erroneous judgment. This makes it possible to obtain the person's physical information with greater accuracy.

[0011] In a physical information acquisition device that solves the above problem, it is preferable that the imaging state determination unit determines that the captured image corresponds to the specific imaging state when the luminance of the captured image is equal to or greater than a predetermined high luminance threshold value as the evaluation index value of the imaging state.

[0012] That is, when the "brightness" of a photographed image is high, the image may become brighter and the image of a person in the photographed image may become unclear, which may result in a decrease in the accuracy of detecting skeleton points by image analysis.

[0013] In this regard, according to the above configuration, the "brightness" of the captured image is used as an evaluation index value for the imaging state, and a state in which the "brightness" is too high is identified. Then, by setting such a state as a specific imaging state and not using the detection of skeleton points by image analysis to obtain body information, the occurrence of such misidentification can be suppressed.

[0014] In a physical information acquisition device that solves the above problem, it is preferable that the imaging state determination unit determines that the captured image corresponds to the specific imaging state when the luminance of the captured image is equal to or lower than a predetermined low luminance threshold value as the evaluation index value of the imaging state.

[0015] That is, if the "brightness" of a captured image is low, the captured image may become dark and the image of a person in the captured image may become unclear, which may result in a decrease in the accuracy of detecting skeleton points through image analysis.

[0016] In this regard, according to the above configuration, the "brightness" of the captured image is used as an evaluation index value for the imaging state, and a state in which the "brightness" is too low is identified. Then, by setting such a state as a specific imaging state and not using the detection of skeleton points by image analysis to obtain body information, the occurrence of such misidentification can be suppressed.

[0017] In the physical information acquisition device that solves the above problem, it is preferable that the imaging state determination unit performs determination of the specific imaging state based on the luminance of the captured image by cropping the imaging area of ​​the person appearing in the captured image.

[0018] According to the above configuration, it is possible to more accurately determine the imaging state of a person appearing in a captured image based on the "luminance" of the captured image. In the physical information acquisition device for solving the above problem, it is preferable that the physical information acquisition unit acquires, as the physical information of the person, physique information of an occupant seated in a vehicle seat.

[0019] That is, by determining the occupant's physical information based on the detection of skeleton points by image analysis, the occupant's physical constitution can be determined with high accuracy regardless of the seat position, such as the sliding position, reclining position, or lift position. Even if the occupant's posture changes, the occupant's physical constitution can be determined with high accuracy. Furthermore, in a specific imaging state in which the accuracy of skeleton point detection is reduced, the detection of skeleton points by image analysis is not used to obtain physical information, thereby avoiding misjudgment, and thereby enabling the occupant's physical information to be obtained with high accuracy. Effect of the Invention

[0020] According to the present invention, it is possible to acquire physical information of a person with high accuracy. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram of a vehicle. [Diagram 2]FIG. 1 is an explanatory diagram of an occupant seated in a vehicle seat and a camera that photographs the occupant. [Diagram 3] FIG. 2 is a block diagram showing a schematic configuration of a control device that functions as a physical information acquisition device. [Figure 4] FIG. 1 is an explanatory diagram showing human skeletal points. [Diagram 5] 10 is a flowchart showing a processing procedure for determining an imaging state based on a predetermined evaluation index value. [Figure 6] 6 is a flowchart showing an aspect of a physical information acquisition process executed in a normal imaging state. [Figure 7] 11 is a flowchart showing an aspect of a body information acquisition process executed when a skeleton point cannot be detected. [Figure 8] FIG. 11 is an explanatory diagram of a body movement waveform used for an evaluation index value of an imaging state. [Figure 9] FIG. 11 is an explanatory diagram of body movement determination based on the magnitude of body movement indicated by a body movement waveform. [Figure 10] 11 is a flowchart showing a procedure for determining body movement, and a physical information acquisition process executed when it is determined that a specific image capturing state exists based on the magnitude of body movement; [Figure 11] 4 is a diagram illustrating the distribution and average value of pixel luminance values ​​in a captured image. FIG. [Figure 12] 11 is an explanatory diagram of image capture state determination in which the luminance of a captured image, indicated by an average value of pixel luminance values, is used as an evaluation index value for the image capture state. [Figure 13] 11 is a flowchart showing a procedure for determining whiteout and blackout, and a physical information acquisition process executed when it is determined that a captured image falls under a specific image capturing state based on the luminance of the captured image. [Figure 14] 1A and 1B are explanatory diagrams of the imaging area of ​​a person appearing in a photographed image and its trimming. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, an embodiment of a physical information acquisition device will be described with reference to the drawings. As shown in Fig. 1 and Fig. 2, the vehicle 1 of this embodiment is provided with a camera 3 that captures images of the interior of the vehicle cabin 2. Specifically, the vehicle 1 of this embodiment is configured as a so-called four-door sedan with two rows of seats 4 arranged in the front and rear of the vehicle cabin 2. The camera 3 is, for example, an infrared camera. In the vehicle 1 of this embodiment, the camera 3 is installed at a position that captures images of passengers 5 seated in each seat 4 from the front side of the vehicle cabin 2 (the left side in each drawing), such as a center console 6 or an overhead console 7.

[0023] 3, in the vehicle 1 of this embodiment, a captured image Vd of the interior of the vehicle cabin 2 captured by the camera 3 is input to the control device 10. The control device 10 includes an image analysis unit 11 that analyzes the captured image Vd, and a person recognition unit 12 that cooperates with the image analysis unit 11 to recognize people in the vehicle cabin 2 captured in the captured image Vd, that is, occupants 5 of the vehicle 1, based on the results of the image analysis.

[0024] 3 and 4, the control device 10 of the present embodiment includes a skeleton point detection unit 13 that detects skeleton points SP of the person H included in the captured image Vd based on the result of the image analysis by cooperating with the image analysis unit 11 and the person recognition unit 12. That is, the skeleton points SP are unique points that characterize the body of the person H, such as joints and points on the body surface, and include, for example, the head, neck, shoulders, armpits, elbows, wrists, hands, waist, hip joints, buttocks, knees, and ankles. Furthermore, the control device 10 of the present embodiment includes a physical information acquisition unit 15 that acquires physical information IB about the person H appearing in the captured image Vd, that is, the occupant 5 of the vehicle 1, based on the detection of the skeleton points SP by the skeleton point detection unit 13. Thus, the control device 10 of the present embodiment has a function as a physical information acquisition device 20.

[0025] More specifically, in the person recognition unit 12 of this embodiment, an inference model generated by machine learning is used to execute a recognition process for the person H. Then, in the skeleton point detection unit 13, a detection process for the skeleton points SP is also executed by using an inference model generated by machine learning.

[0026] The physical information acquisition unit 15 of the present embodiment also includes a physique determination unit 21 that determines the physique of the person H, who is the occupant 5 of the vehicle 1, as the physical information of the person H captured in the captured image Vd. Specifically, the physique determination unit 21 determines whether the occupant 5 seated in the seat 4 of the vehicle 1 is an "adult", a "small adult" such as a woman, or a "child". Furthermore, the control device 10 of the present embodiment outputs the physical information IB of the occupant 5 acquired by the physical information acquisition unit 15 to a higher-level control device (not shown). In the vehicle 1 of the present embodiment, for example, airbag deployment control is executed based on the physical information Iph of the occupant 5 included in the physical information IB.

[0027] The physical information acquisition unit 15 of this embodiment includes a posture determination unit 22 that determines the posture of the person H, who is an occupant 5 of the vehicle 1, as the physical information IB of the person H captured in the captured image Vd. The control device 10 of this embodiment also outputs posture information Ipo of the occupant 5 acquired based on the result of the posture determination performed by the posture determination unit 22 to a higher-level control device (not shown) as the physical information IB of the occupant 5. The vehicle 1 of this embodiment is configured to execute, for example, airbag deployment control or permission determination for automatic driving control based on the posture information Ipo of the occupant 5.

[0028] 1 to 3, the vehicle 1 of this embodiment is equipped with a load sensor 25 provided below each seat 4. The load sensor 25 is disposed, for example, at four corners below the seat cushion or at two corners located on the inner side in the vehicle width direction. The control device 10 of this embodiment is equipped with a seat load detection unit 26 that detects the seat load WS of the occupant 5 seated in the seat 4 based on an output signal of the load sensor 25.

[0029] Furthermore, in the control device 10 of this embodiment, the seat load WS detected by the seat load detection unit 26 is also used for acquiring the physical information IB executed by the physical information acquisition unit 15. This enables the control device 10 of this embodiment to acquire the physical information IB of the occupant 5 with higher accuracy.

[0030] (Control of acquisition of physical information) Next, the control of acquiring the physical information IB executed by the control device 10 of this embodiment, specifically, the determination of the physical build of the occupant 5, will be described.

[0031] 3, the control device 10 of this embodiment includes an imaging state determination unit 27 that, when acquiring physical information IB of the person H based on detection of the skeleton points SP, determines an imaging state VS of the person H captured in the captured image Vd. Specifically, in the control device 10 of this embodiment, an evaluation index value for evaluating the imaging state VS of the captured image Vd of the person H from which physical information IB is acquired is set in advance. The imaging state determination unit 27 of this embodiment is configured to determine whether or not the imaging state VS of the occupant 5 captured in the captured image Vd corresponds to a specific imaging state specified based on the evaluation index value of the predetermined imaging state VS.

[0032] In the control device 10 of this embodiment, the determination result by the imaging state determination unit 27 is input to the physical information acquisition unit 15. The physical information acquisition unit 15 of this embodiment is configured to change the processing procedure for acquiring the physical information IB of the occupant 5 shown in the captured image Vd, more specifically, the processing procedure for the physique determination by the physique determination unit 21, based on the determination result of the imaging state VS.

[0033] That is, as shown in the flowchart of FIG. 5, when the control device 10 of this embodiment acquires a photographed image Vd of the interior of the vehicle compartment 2 (step 101), it executes image analysis to detect the skeleton points SP of the occupant 5 included in the photographed image Vd (step 102). Next, the control device 10 judges whether the detection process of the skeleton points SP in the above step 102 has been performed correctly (step 103). In the control device 10 of this embodiment, a situation in which the detection process of the skeleton points SP is not performed correctly corresponds to a situation in which the skeleton points SP of the occupant 5 cannot be detected from the photographed image Vd, such as when the field of view of the camera 3 is blocked or when the body of the occupant 5 is out of the angle of view of the camera 3. Furthermore, when the control device 10 judges that the skeleton points SP of the occupant have been detected (step 103: YES), it acquires an evaluation index value Vei for the imaging state VS of the occupant 5 shown in the photographed image Vd acquired in step 101 (step 104). Then, the control device 10 of this embodiment executes a determination process to determine whether or not the imaging state VS of the occupant 5 shown in the captured image Vd corresponds to a specific imaging state VS1 based on the evaluation index value Vei acquired in step 104 (step 105).

[0034] Next, when the result of the imaging state determination in the above step 105 indicates that the imaging state does not fall under the specific imaging state VS1 (step 106: NO), the control device 10 of this embodiment executes a process of acquiring physical information IB corresponding to the normal imaging state VS0 (step 107). Then, when the result of the imaging state determination in the above step 105 indicates that the imaging state falls under the specific imaging state VS1 (step 106: YES), the control device 10 executes a process of acquiring physical information IB corresponding to the specific imaging state VS1 (step 108).

[0035] Furthermore, when the control device 10 of this embodiment determines in the above step 103 that the skeleton point SP of the occupant has not been detected (step 103: NO), it does not execute the processes of the above steps 104 to 108. In this case, it executes the process of acquiring the physical information IB corresponding to the case where the skeleton point SP is in the non-detection state VS2 (step 109).

[0036] Specifically, control device 10 of this embodiment executes a process of acquiring physical information IB based on the detection of skeleton points SP executed in step 102 as the normal state physical information acquisition process in step 107. Furthermore, control device 10 executes a process of acquiring physical information IB not depending on the detection of skeleton points SP executed in step 102 as the specific state physical information acquisition process in step 108. In other words, if it is determined in step 106 that the specific imaging state VS1 exists (step 106: YES), control device 10 of this embodiment does not execute a process of acquiring physical information IB based on the detection of skeleton points SP. Then, control device 10 of this embodiment also executes a process of acquiring physical information IB not depending on the detection of skeleton points SP in the non-detection state physical information acquisition process in step 109.

[0037] More specifically, as shown in the flowcharts of FIG. 3 and FIG. 6, in the control device 10 of this embodiment, when the control device 10 is in the normal imaging state VS0 (step 201: YES), the physical information acquisition unit 15 acquires the skeleton point SP of the occupant 5 from the skeleton point detection unit 13 (step 202). In addition, the physical information acquisition unit 15 acquires the seat load WS of the occupant 5 captured in the captured image Vd from the seat load detection unit 26 (step 203). Furthermore, the physical information acquisition unit 15 calculates the feature amount FV of the occupant 5 to acquire the physical information IB based on the skeleton point SP acquired in the above step 201 and the seat load WS of the occupant 5 acquired in the above step 201 (step 204). Then, the physical information acquisition unit 15 of this embodiment executes acquisition processing of the physical information IB based on the feature amount FV of the occupant 5 calculated in this step 203 (step 205).

[0038] Specifically, the physical information acquisition unit 15 of this embodiment calculates the feature amount FV based on the positions of skeleton points SP, such as the head and shoulders of the occupant 5, detected from the captured image Vd. The physical information acquisition unit 15 also calculates the feature amount FV based on the body dimensions indicated by the skeleton points SP, such as the shoulder width of the occupant 5. Furthermore, the physical information acquisition unit 15 of this embodiment calculates the feature amount FV based on the seat load WS of the occupant 5. Then, in the physical information acquisition unit 15 of this embodiment, these feature amounts FV are input to an inference model generated by machine learning, whereby the physical type determination unit 21 performs a physical type determination of the occupant 5.

[0039] More specifically, the control device 10 of this embodiment periodically executes the processes of steps 101 to 109 shown in Fig. 5. That is, in the control device 10 of this embodiment, detection of skeleton points SP by the skeleton point detection unit 13, acquisition of physical information IB by the physical information acquisition unit 15, and determination of the imaging state VS by the imaging state determination unit 27 are periodically executed.

[0040] 3, in the control device 10 of this embodiment, the physical information acquisition unit 15 includes a previous value storage unit 28 that stores the physical information IB of the person H acquired in the previous period as a previous value IBb of the physical information IB. The physical information acquisition unit 15 of this embodiment is configured to set the previous value IBb of the physical information IB as the physical information IB acquired in the current period as the non-detection time physical information acquisition process (see FIG. 5, step 109).

[0041] More specifically, as shown in the flowchart of Fig. 7, when the skeleton point SP is in a non-detection state VS2 (step 301: YES), the physical information acquisition unit 15 of this embodiment first judges whether the non-detection state VS2 has continued for a predetermined time or more (step 302). If the physical information acquisition unit 15 judges that the non-detection state VS2 of the skeleton point SP has not continued for a predetermined time or more (step 302: NO), it reads out the previous value IBb of the physical information IB from the previous value storage unit 28 (step 303). Then, the physical information acquisition unit 15 of this embodiment acquires this previous value IBb as the physical information IB of the occupant 5 in the current cycle (step 304).

[0042] Specifically, in the physical information acquisition unit 15 of this embodiment, when the skeleton point SP is in the non-detection state VS2, the physique determination unit 21 acquires the result of the physique determination performed in the previous cycle, that is, the previous value Iphb of the physique information Iph, from the previous value holding unit 28. Then, the physique determination unit 21 of this embodiment is configured to use this previous value Iphb as the physique information Iph of the occupant 5 acquired in the current cycle, thereby carrying over the result of the physique determination performed in the previous cycle to the physique determination in the current cycle.

[0043] If the physical information acquisition unit 15 of this embodiment determines in step 302 that the non-detection state VS2 of the skeleton point SP has continued for a predetermined time or longer (step 302: YES), it does not execute the processes of steps 303 and 304. Then, it is assumed that there is no occupant 5 on the seat 4 shown in the captured image Vd, from whom physical information IB is to be acquired, that is, that the seat 4 is vacant (step 305).

[0044] More specifically, in the control device 10 of this embodiment, the "magnitude of body movement" of the occupant 5 captured in the captured image Vd of the vehicle interior 2 is set as the evaluation index value Vei of the image capture state VS. The image capture state determination unit 27 of this embodiment is configured to execute a determination process of the image capture state VS based on the "magnitude of body movement" of the occupant 5 (see FIG. 5, steps 104 and 105).

[0045] 3, the imaging state determination unit 27 of this embodiment is provided with a body movement waveform generation unit 31 that generates a body movement waveform α of the person H appearing in the captured image Vd. The imaging state determination unit 27 of this embodiment is further provided with a body movement determination unit 32 that uses the "magnitude of body movement" appearing in the body movement waveform α as an evaluation index value Vei for the imaging state VS and executes a determination as to whether or not the imaging state VS corresponds to a specific imaging state VS1.

[0046] 3 and 8, the body movement waveform generator 31 of this embodiment holds, in its memory area 31m, time series data DtSP of the skeleton points SP periodically detected by the skeleton point detector 13. The body movement waveform generator 31 also holds, in its memory area 31m, time series data DtWS of the seat load WS periodically detected by the seat load detector 26. The body movement waveform generator 31 of this embodiment then generates a body movement waveform α of the occupant 5 shown in the captured image Vd of the vehicle interior 2, based on these time series data DtSP and DtWS.

[0047] As shown in Fig. 8, in the body movement waveform generator 31 of this embodiment, the time series data DtSP of the skeleton point SP is represented, for example, by the X coordinate SPx and the Y coordinate SPy in the captured image Vd in which the occupant 5 appears. Furthermore, the body movement waveform generator 31 of this embodiment performs frequency analysis of the time series data DtSP of the skeleton point SP. Specifically, the body movement frequency components αx, αy contained in the time series data DtSP of the skeleton point SP are extracted by executing a filter process. Furthermore, the body movement waveform generator 31 also executes a similar filter process on the time series data DtWS of the seat load WS. The body movement waveform generator 31 of this embodiment is configured to generate a body movement waveform α of the occupant 5 in this manner.

[0048] 3 and 9, the body movement determining unit 32 of this embodiment acquires the amplitude A of the body movement waveform α output by the body movement waveform generating unit 31. Specifically, the body movement determining unit 32 of this embodiment acquires the amplitude A by digitally processing the body movement waveform α of the occupant 5. Furthermore, the body movement determining unit 32 compares this amplitude A with a predetermined body movement threshold Ath. Then, when the amplitude A of the body movement waveform α is equal to or greater than the body movement threshold Ath, the body movement determining unit 32 of this embodiment is configured to determine that the imaging state VS of the occupant 5 captured in the captured image Vd is a body movement occurrence state VS1a corresponding to the specific imaging state VS1.

[0049] 10, the image capture state determination unit 27 of this embodiment reads out each of the time series data DtSP, DtWS of the skeleton point SP and the seat load WS of the occupant 5 (steps 401 and 402). Then, the image capture state determination unit 27 generates a body movement waveform α of the occupant 5 based on each of the time series data DtSP, DtWS (step 403).

[0050] In the control device 10 of this embodiment, the body movement waveform α of the occupant 5 based on the time series data DtSP of the skeleton point SP is given priority. The body movement waveform α of the occupant 5 based on the time series data DtWS of the seat load WS is positioned as a complement to the body movement waveform α of the occupant 5 based on the time series data DtWS of the seat load WS.

[0051] Furthermore, the imaging state determination unit 27 determines whether or not the amplitude A of the body movement waveform α generated in the above step 403 is equal to or greater than the body movement threshold Ath (step 404). If the amplitude A of the body movement waveform α is equal to or greater than the body movement threshold Ath (A≧Ath, step 404: YES), the imaging state determination unit 27 determines that the imaging state VS of the occupant 5 is a body movement occurrence state VS1a in which a large body movement has occurred in the occupant 5 (step 405).

[0052] That is, the amplitude A of the body movement waveform α directly indicates the "magnitude of body movement" of the occupant 5 captured in the captured image Vd. When the "magnitude of body movement" indicated by this body movement waveform α is large, that is, in the body movement occurrence state VS1a, the detection accuracy of the skeleton points SP by the image analysis may decrease.

[0053] Considering this, in the control device 10 of this embodiment, the body movement waveform generating unit 31 and the body movement determining unit 32 provided in the imaging state determining unit 27 detect that the imaging state VS of the occupant 5 shown in the captured image Vd is in such a body movement occurrence state VS1a. Then, the control device 10 of this embodiment is configured to execute a specific time body information acquisition process (see FIG. 5, step 108) that does not depend on detection of the skeleton point SP, based on the result of the imaging state determination by the imaging state determining unit 27.

[0054] More specifically, in the control device 10 of this embodiment, when the control device 10 is in such a body movement occurrence state VS1a, the body information acquisition unit 15 reads out the body information IB of the person H acquired in the previous cycle, that is, the previous value IBb of the body information IB (step 406). Furthermore, the body information acquisition unit 15 of this embodiment sets this previous value IBb as the body information IB of the occupant 5 in the current cycle (step 407). Thus, the control device 10 of this embodiment is configured to carry over the body information IB of the occupant 5 acquired in the previous cycle to the current cycle, thereby avoiding erroneous determination caused by the body movement of the occupant 5.

[0055] Specifically, in the physical information acquisition unit 15 of this embodiment, when in a body movement occurrence state VS1a, the physique determination unit 21 acquires the result of the physique determination performed in the previous cycle, that is, the previous value Iphb of the physique information Iph, from the previous value holding unit 28. Then, the physique determination unit 21 of this embodiment is configured to take over the determination result in the previous cycle for the current cycle by using this previous value Iphb as the physique information Iph of the occupant 5 acquired in the current cycle.

[0056] Still, in the imaging state determination unit 27 of the present embodiment, when it is determined in step 404 that the amplitude A of the body movement waveform α is smaller than the body movement threshold value Ath (A < Ath, step 404: NO), the processes of steps 405 to 407 are not executed. Further, in this case, the imaging state determination unit 27 determines that there is "no large body movement" in the occupant 5 reflected in the captured image Vd (no body movement, step 408). And the control device 10 of the present embodiment is configured such that, thereby, when it is determined that the imaging state VS of the occupant 5 reflected in the captured image Vd is in the normal imaging state VS0, the normal body information acquisition process is executed (see FIG. 5, step 107).

[0057] Also, in the control device 10 of the present embodiment, together with the "magnitude of body movement" of the person H as described above, the "luminance" of the captured image Vd is acquired as the evaluation index value Vei of the imaging state VS. And the imaging state determination unit 27 of the present embodiment is configured to execute the determination process of the imaging state VS based on the "luminance" of the captured image Vd (see FIG. 5, steps 104 and 105).

[0058] That is, when the "luminance" of the captured image Vd is high, as the captured image Vd becomes brighter, the image of the person H reflected in the captured image Vd may become unclear. In particular, when an infrared camera is used or the like, when the captured image Vd is a monochrome image, the image of the person H included in the captured image Vd is displayed entirely in white, that is, in a so-called "whiteout" state, and this tendency is more likely to appear prominently. And thereby, there is a possibility that the detection accuracy of the skeleton point SP by the image analysis may decrease.

[0059] Also, when the "luminance" of the captured image Vd is low, as the captured image Vd becomes darker, the image of the person H reflected in the captured image Vd may become unclear. Further, also in this case, the image of the person H included in the monochrome captured image Vd is in a so-called "blackening" state where it is displayed entirely in black. And thereby, there is a possibility that the detection accuracy of the skeleton point SP by the image analysis may decrease.

[0060] In consideration of this, the imaging state determination unit 27 of this embodiment executes a process of determining the imaging state VS based on the "luminance" of the captured image Vd. If the "luminance" of the captured image Vd is "too high" or "too low", the acquisition of the physical information IB based on the detection of the skeleton points SP is not performed, thereby avoiding a decrease in accuracy due to misjudgment.

[0061] 3, the imaging state determination unit 27 of this embodiment is provided with a pixel luminance value detection unit 33 that detects a pixel luminance value L for each pixel constituting the captured image Vd. Furthermore, in the control device 10 of this embodiment, the "luminance" of the captured image Vd is expressed using this pixel luminance value L. The imaging state determination unit 27 of this embodiment is provided with a blown-out highlight determination unit 34 and a dark image determination unit 35 that determine whether or not the "luminance" of the captured image Vd represented by this pixel luminance value L is the evaluation index value Vei of the imaging state VS and that judges whether or not the imaging state VS corresponds to the specific imaging state VS1.

[0062] 11 and 12, the pixel luminance value detection unit 33 of this embodiment calculates an average value Lav of the pixel luminance values ​​L as a value indicating the "luminance" of the captured image Vd. Note that FIG. 11 is a graph showing the distribution of the pixel luminance values ​​L in the captured image Vd, with the horizontal axis representing the pixel luminance values ​​L and the vertical axis representing the number of pixels n. In addition, in the imaging state determination unit 27 of this embodiment, the blown-out highlight determination unit 34 compares the average value Lav of the pixel luminance values ​​L output by the pixel luminance value detection unit 33 with a predetermined high luminance threshold value LthH. Then, when the average value Lav of the pixel luminance values ​​L is equal to or greater than the high luminance threshold value LthH, the blown-out highlight determination unit 34 determines that the imaging state VS of the occupant 5 shown in the captured image Vd is a blown-out highlight state VS1b corresponding to the specific imaging state VS1.

[0063] The black appearance determination unit 35 also compares the average value Lav of the pixel luminance values ​​L with a predetermined low luminance threshold value LthL. If the average value Lav of the pixel luminance values ​​L is equal to or lower than the low luminance threshold value LthL, the black appearance determination unit 35 determines that the imaging state VS of the occupant 5 shown in the captured image Vd is a black appearance state VS1c that corresponds to the specific imaging state VS1.

[0064] More specifically, as shown in the flowcharts of Figures 3 and 13, when the imaging state determination unit 27 of this embodiment acquires a captured image Vd of the inside of the vehicle compartment 2 (step 501), it trims the imaging area γ of the person H, i.e., the occupant 5, appearing in the captured image Vd (step 502).

[0065] As shown in FIG. 14, in the imaging state determination unit 27 of this embodiment, the trimming process for cutting out the imaging region γ of the occupant 5 is performed in a form including the upper body of the occupant 5, at least the head and torso.

[0066] 3 and 13, in the imaging state determination unit 27 of this embodiment, the pixel luminance value detection unit 33 detects the pixel luminance value L in the imaging region γ of the occupant 5 and calculates the average value Lav. The imaging state determination unit 27 of this embodiment thereby obtains the "luminance" of the captured image Vd in which the occupant 5 appears (step 503).

[0067] Next, in the imaging state determination unit 27 of this embodiment, the blown-out highlight determination unit 34 compares the average value Lav of the pixel brightness values ​​L calculated as the "brightness" of the captured image Vd in step 503 with the high brightness threshold value LthH (step 504). If the average value Lav of the pixel brightness values ​​L is equal to or greater than the high brightness threshold value LthH (Lav≧LthH, step 504: YES), the blown-out highlight determination unit 34 determines the imaging state VS of the occupant 5 shown in the captured image Vd to be a blown-out highlight state VS1b (step 505).

[0068] Furthermore, in step 504, when it is determined that the value is smaller than the high brightness threshold LthH (Lav < LthH, step 504: NO), the black smear determination unit 35 compares the average value Lav of the pixel brightness value L with the low brightness threshold LthL (step 506). Then, when the average value Lav of the pixel brightness value L is less than or equal to the low brightness threshold LthL (Lav ≤ LthL, step 506: YES), the black smear determination unit 35 determines that the imaging state VS of the occupant 5 reflected in the captured image Vd is the black smear state VS1b (step 507).

[0069] Also, in the control device 10 of the present embodiment, when it is determined that the specific imaging state VS1 is applicable because such a white-out state VS1b or black smear state VS1c is reached, the body information acquisition unit 15 acquires the seat load WS of the occupant 5 (step 508). And the body information acquisition unit 15 of the present embodiment is configured to execute the acquisition of the body information IB for the occupant 5 reflected in the captured image Vd based on this seat load WS (step 509).

[0070] Specifically, in the body information acquisition unit 15 of the present embodiment, in this case, the physique determination unit 21 executes the physique determination of the occupant 5 by comparing the seat load WS with a predetermined threshold. And the body information acquisition unit 15 of the present embodiment is configured to acquire the physique information Iph of the occupant 5 thereby.

[0071] In addition, when the average value Lav of the pixel brightness value L is between the low brightness threshold LthL and the high brightness threshold LthH (LthL < Lav < LthH, step 506: NO), the imaging state determination unit 27 of the present embodiment determines that it is in the normal imaging state VS0 (step 510). And in the control device 10 of the present embodiment, thereby, in the body information acquisition unit 15, the normal-time body information acquisition process, that is, the acquisition of the body information IB based on the detection of the skeleton points SP is executed (see FIG. 5, step 107).

[0072] Next, the operation of the present embodiment will be described. In the control device 10 having the function of the physical information acquisition device 20, it is determined whether or not the image capture state VS of the occupant 5 captured in the captured image Vd of the vehicle interior 2 corresponds to a specific image capture state VS1 specified based on a predetermined evaluation index value Vei. If the image capture state corresponds to such a specific image capture state VS1, acquisition of the physical information IB of the occupant 5 based on the detection of the skeleton point SP is not executed.

[0073] Next, the effects of this embodiment will be described. (1) The control device 10 as the physical information acquisition device 20 includes a skeleton point detection unit 13 that detects skeleton points SP of person H included in the captured image Vd. The control device 10 also includes a physical information acquisition unit 15 that acquires physical information IB of person H based on the detection of the skeleton points SP. The control device 10 further includes an imaging state determination unit 27 that determines whether or not the imaging state VS of person H captured in the captured image Vd corresponds to a specific imaging state VS1 specified based on a predetermined evaluation index value Vei. If the imaging state VS corresponds to the specific imaging state VS1, the physical information acquisition unit 15 does not execute acquisition of physical information IB based on the detection of the skeleton points SP.

[0074] According to the above configuration, a state in which the detection accuracy of the skeleton points SP is reduced due to the imaging state VS of the person H in the captured image Vd is set as the specific imaging state VS1, and in such a case, the detection of the skeleton points SP by the image analysis is not used for acquiring the physical information IB. This makes it possible to avoid the occurrence of erroneous determination and to acquire the physical information IB of the person H with high accuracy.

[0075] (2) The imaging state determination unit 27 is provided with a body movement waveform generation unit 31 that generates a body movement waveform α of the person H appearing in the captured image Vd. Furthermore, the imaging state determination unit 27 includes a body movement determination unit 32 that determines whether the imaging state VS of the person H corresponds to a specific imaging state VS1 or not, using the "magnitude of body movement" appearing in the body movement waveform α as an evaluation index value Vei of the imaging state VS. Then, the body movement determination unit 32 determines that the imaging state VS of the person H corresponds to a specific imaging state VS1 when the amplitude A of the body movement waveform α indicating the "magnitude of body movement" is equal to or greater than the body movement threshold value Ath (A≧Ath).

[0076] That is, when the "magnitude of body movement" of person H captured in captured image Vd is large, the accuracy of detecting skeleton points SP by image analysis may decrease. In this regard, according to the above configuration, a body movement occurrence state VS1a in which the "magnitude of body movement" is large is specified from the body movement waveform α of person H. Then, this body movement occurrence state VS1a is set as the specific imaging state VS1, and the detection of skeleton points SP by image analysis is not used to acquire physical information IB, thereby suppressing the occurrence of erroneous determination.

[0077] Furthermore, the body movement occurrence state VS1a can be taken as a sign of larger "body movement" such as when the person H takes an extreme posture, such as leaning forward or twisting the upper body, which causes the person H to move out of the angle of view of the camera 3, thereby avoiding the associated decrease in judgment accuracy and occurrence of erroneous judgment. This makes it possible to acquire the physical information IB of the person H with higher accuracy.

[0078] (3) The imaging state determination unit 27 is provided with a pixel luminance value detection unit 33 that detects a pixel luminance value L for each pixel constituting the captured image Vd. The imaging state determination unit 27 also includes a blown-out highlight determination unit 34 that determines whether or not the imaging state VS of the person H corresponds to the specific imaging state VS1 by using the "luminance" of the captured image Vd represented by the average value Lav of the pixel luminance values ​​L as an evaluation index value Vei of the imaging state VS. Then, the blown-out highlight determination unit 34 determines that the imaging state VS of the person H corresponds to the specific imaging state VS1 when the average value Lav of the pixel luminance values ​​L is equal to or greater than a predetermined high luminance threshold value LthH (Lav≧LthH).

[0079] That is, when the "brightness" of the captured image Vd is high, the captured image Vd becomes bright, and the image of the person H reflected in the captured image Vd may become unclear. This may result in a decrease in the accuracy of detecting the skeleton points SP by image analysis.

[0080] In this regard, according to the above configuration, the "luminance" of the captured image Vd represented by the average pixel luminance value Lav is set as the evaluation index value Vei of the imaging state VS, and a blown-out highlight state VS1b in which the "luminance" is too high is identified. Then, this blown-out highlight state VS1b is set as the specific imaging state VS1, and the detection of the skeleton points SP by image analysis is not used to acquire the physical information IB, thereby suppressing the occurrence of such misidentification.

[0081] (4) The imaging state determination unit 27 includes a black appearance determination unit 35 that determines whether or not the imaging state VS corresponds to a specific imaging state VS1 by using the “luminance” of the captured image Vd represented by the average value Lav of the pixel luminance values ​​L as an evaluation index value Vei for the imaging state VS. When the average value Lav of the pixel luminance values ​​L is equal to or lower than a predetermined low luminance threshold value LthL (Lav≦LthL), the black appearance determination unit 35 determines that the imaging state VS of the person H corresponds to the specific imaging state VS1.

[0082] That is, even if the "brightness" of the photographed image Vd is low, the photographed image Vd may become dark, and the image of the person H reflected in the photographed image Vd may become unclear. This may result in a decrease in the accuracy of detecting the skeleton points SP by image analysis.

[0083] In this regard, according to the above configuration, the "luminance" of the captured image Vd represented by the average pixel luminance value Lav is set as the evaluation index value Vei of the imaging state VS, and the black appearance state VS1c in which the "luminance" is too low is specified. Then, this black appearance state VS1c is set as the specific imaging state VS1, and the detection of the skeleton points SP by the image analysis is not used to acquire the physical information IB, thereby suppressing the occurrence of misidentification.

[0084] (5) The imaging state determination unit 27 performs determination of the specific imaging state VS1 based on the "luminance" of the captured image Vd by trimming the imaging region γ of the person H appearing in the captured image Vd. This makes it possible to perform imaging state determination based on the "luminance" of the captured image Vd with higher accuracy for the imaging state VS of the person H appearing in the captured image Vd.

[0085] (6) The physical information acquisition unit 15 acquires physical information Iph of the occupant 5 seated in the seat 4 of the vehicle 1 as the physical information IB of the person H appearing in the captured image Vd. That is, by determining the physical information Iph of the occupant 5 based on the detection of the skeleton points SP by image analysis, the physical information Iph of the occupant 5 can be determined with high accuracy regardless of the seat position, such as the sliding position, reclining position, or lift position. Even if the posture of the occupant 5 changes, the physical information can be determined with high accuracy. Furthermore, when the vehicle is in a specific imaging state VS1 in which the detection accuracy of the skeleton points SP decreases, the detection of the skeleton points SP by image analysis is not used to obtain the physical information IB, thereby avoiding misjudgment, and thereby enabling the physical information Iph of the occupant 5 to be obtained with high accuracy.

[0086] (7) The body movement waveform generating unit 31 generates a body movement waveform α of the person H based on the time series data DtSP of the skeleton points SP. According to the above configuration, in addition to detecting skeleton points SP by image analysis, a body movement waveform α of the person H captured in the captured image Vd can be generated. This makes it possible to generate the body movement waveform α of the person H without adding any new configuration.

[0087] (8) The control device 10 includes a seat load detection unit 26 that detects the seat load WS of the occupant 5. The body movement waveform generation unit 31 generates a body movement waveform of the occupant 5 based on the time-series data DtWS of the seat load WS.

[0088] According to the above configuration, it is possible to generate a body movement waveform α of the person H seated in the seat 4, that is, the occupant 5, by utilizing the load sensor 25 of the seat 4 that is adopted in many vehicles 1. This makes it possible to generate the body movement waveform α of the occupant 5 seated in the seat 4 without adding any new components.

[0089] (9) Control device 10 periodically executes detection of skeleton points SP by skeleton point detection unit 13, acquisition of physical information IB by physical information acquisition unit 15, and determination of imaging state VS by imaging state determination unit 27. Physical information acquisition unit 15 also includes a previous value holding unit 28 that holds the physical information IB acquired in the previous cycle as a previous value IBb of the physical information IB. When physical information acquisition unit 15 is in a body movement occurrence state VS1a that is determined to correspond to specific imaging state VS1 based on the "magnitude of body movement" appearing in the body movement waveform α, physical information acquisition unit 15 sets the previous value IBb of the physical information IB as the physical information IB acquired in the current cycle.

[0090] That is, by using the judgment result before the accuracy of detecting the skeleton points SP by image analysis decreases, it is possible to avoid misjudgment. And by storing the previous value IBb of the physical information IB, which is the previous "judgment result", it is possible to avoid a situation where the current value and the previous value of the state value used in the judgment are mixed.

[0091] (10) When the captured image Vd is in a blown-out highlight state VS1b or a darkened highlight state VS1c that is determined to correspond to a specific imaging state based on the "luminance" of the captured image Vd, the physical information acquisition unit 15 acquires the physical information IB of the occupant 5 based on the seat load WS. This makes it possible to acquire the physical information IB of the occupant 5 with high accuracy even in a state in which the detection accuracy of the skeleton points SP by image analysis is reduced.

[0092] (11) The physical information acquisition unit 15 inputs the feature values ​​FV calculated based on the detection of the skeleton points SP to an inference model generated by machine learning, thereby acquiring the physical information IB of the person H. This makes it possible to accurately acquire the physical information IB of the person H based on the detection of the skeleton points SP by the image analysis.

[0093] (12) The physical information acquisition unit 15 uses the seat load WS as the feature value FV. This makes it possible to more accurately acquire the physical information IB of the person H based on the inference model generated by the machine learning.

[0094] The above embodiment can be modified as follows: The above embodiment and the following modifications can be combined with each other to the extent that no technical contradiction occurs.

[0095] In the above embodiment, the feature values ​​FV of the person H are calculated based on the positions of the skeleton points SP included in the captured image Vd, the body dimensions indicated by the skeleton points SP, and the seat load WS of the occupant 5. The feature values ​​FV are then input to an inference model generated by machine learning to execute physical information IB of the person H appearing in the captured image Vd, more specifically, a physical build determination. However, without being limited to this, the present invention may be applied to a configuration in which physical information IB is acquired based on detection of the skeleton points SP by image analysis using a statistical method, map calculation, or the like.

[0096] Furthermore, the present invention may be embodied in a configuration that does not include the load sensor 25 for detecting the seat load WS. In such a configuration, when the specific imaging state VS1 is reached and the acquisition of the physical information IB based on the detection of the skeleton point SP is not executed, for example, the previous value IBb of the physical information IB acquired in the previous period may be set as the physical information IB acquired in the current period. The body movement waveform α may also be generated based on the time series data DtSP of the skeleton point SP.

[0097] The type of the camera 3 may be changed as desired, for example, to use a visible light camera, etc. The installation position may also be changed as desired. In addition, the skeleton points SP and body dimensions used to acquire the body information IB may be set arbitrarily.

[0098] In the above embodiment, the camera 3 installed in the vehicle 1 captures an image of the occupant 5 in the passenger compartment 2, more specifically, the occupant 5 seated in the seat 4. Then, for the occupant 5 as a person H captured in the captured image Vd, physical information IB is acquired based on detection of skeleton points SP by image analysis. However, the present invention is not limited to this, and may be configured to acquire physical information IB of an occupant 5 of the vehicle 1 who is in a standing position, such as a shared vehicle. Then, for example, the present invention may be applied to a configuration in which physical information IB of a person H captured in the captured image Vd on the road or inside a building is acquired.

[0099] In the above embodiment, the body movement occurrence state VS1a based on the "magnitude of body movement" appearing in the body movement waveform α, the blown-out highlight state VS1b and the blackened highlight state VS1c based on the "luminance" of the captured image Vd are specified as the imaging state VS corresponding to the specific imaging state VS1. However, the present invention is not limited to this, and the imaging state may be determined based on either the "magnitude of body movement" or the "luminance" of the captured image Vd. Also, the imaging state determination based on the "luminance" of the captured image Vd may be determined to be applicable only to either the blown-out highlight state VS1b or the blackened highlight state VS1c. Furthermore, the determination of the application of either the blown-out highlight state VS1b or the blackened highlight state VS1c may be combined with the determination of the body movement occurrence state VS1a. The determination of the application of the body movement occurrence state VS1a, the blown-out highlight state VS1b and the blackened highlight state VS1c may be combined with the determination of the application of the specific imaging state VS1 based on the evaluation index value Vei of the other imaging state VS.

[0100] In the above embodiment, the trimming process for cutting out the imaging region γ of the occupant 5 is performed in a form that includes the upper body of the occupant 5, at least the head and torso, but the setting range of the imaging region γ may be changed arbitrarily. For example, the cut-out shape of the imaging region γ to be trimmed may be changed, such as making the imaging region γ an ellipse. Also, a configuration may be adopted in which a plurality of imaging regions γ are cut out. And, a configuration may be adopted in which the imaging state of the imaging region γ of the occupant 5 is determined based on the overall "brightness" of the captured image Vd, without trimming the imaging region γ of the occupant 5.

[0101] In the above embodiment, the average pixel luminance value Lav is used as the value representing the “luminance” of the captured image Vd. However, a configuration may be adopted in which the median pixel luminance value or the peak value of the distribution is used.

[0102] Next, the technical ideas that can be understood from the above embodiment and modified examples will be described. (i) The body movement waveform generating unit generates the body movement waveform based on time-series data of the skeletal points.

[0103] According to the above configuration, in addition to detecting skeleton points by image analysis, it is possible to generate a body movement waveform of a person appearing in the captured image. This makes it possible to generate a body movement waveform of a person without adding any new configuration.

[0104] (b) The vehicle seat vehicle control device further includes a seat load detection unit that detects the seat load of the occupant, and the body movement waveform generation unit generates the body movement waveform based on time-series data of the seat load.

[0105] According to the above configuration, it is possible to generate a body movement waveform of a person sitting in the seat, that is, an occupant, by utilizing a load sensor of the seat that is adopted in many vehicles. This makes it possible to generate a body movement waveform of an occupant sitting in the seat without adding any new components.

[0106] (c) A pixel brightness value detection unit detects a pixel brightness value for each pixel constituting the captured image, and the image capture state determination unit determines the average, median, or peak value of the pixel brightness values ​​as the brightness of the captured image. This makes it possible to quantitatively determine whether the captured image is in a specific image capture state using the "brightness" of the captured image as an evaluation index value for the image capture state.

[0107] (ii) The detection of skeletal points, the acquisition of the physical information, and the determination of the imaging state are performed periodically, and the physical information acquisition unit includes a previous value holding unit which holds the physical information acquired in the previous cycle as a previous value of the physical information, and when the specific imaging state is met, the previous value of the physical information acquired in the previous cycle is set as the physical information acquired in the current cycle.

[0108] In other words, by using the judgment result before the accuracy of detecting skeleton points by image analysis decreases, it is possible to avoid misjudgment. And by storing the previous value of the body information, which is the previous "judgment result", it is possible to avoid a situation where the current value and the previous value of the state value used in the judgment are mixed.

[0109] (e) A seat load detection unit detects the seat load of the occupant, and the physical information acquisition unit acquires the physical information of the person based on the seat load when the specific image capture state is met. This makes it possible to acquire the physical information of the occupant with high accuracy even when the accuracy of detecting skeleton points by image analysis is reduced.

[0110] (e) When it is determined that the specific imaging state exists based on the luminance of the captured image, the physical information acquisition unit acquires physical information of the person based on the load on the seat, and when it is determined that the specific imaging state exists based on the magnitude of the body movement, the physical information acquisition unit sets the previous value of the physical information acquired in the previous cycle as the physical information acquired in the current cycle.

[0111] That is, when it is determined that a specific imaging state exists based on the magnitude of body movement, the accuracy of acquiring the person's physical information based on the load on the seat may also decrease. Therefore, in such a case, by using the determination result before the accuracy of detecting the skeleton points by image analysis decreases, it is possible to avoid misjudgment and acquire the physical information with high accuracy.

[0112] (g) The physical information acquisition unit acquires the person's physical information by inputting the feature amount calculated based on the detection of the skeleton points to an inference model generated by machine learning, thereby making it possible to acquire the person's physical information with high accuracy based on the detection of the skeleton points by the image analysis.

[0113] (h) The physical information acquisition unit uses the seat load as the feature amount, thereby making it possible to acquire the person's physical information more accurately based on the inference model generated by the machine learning. [Explanation of symbols]

[0114] 10...Control device 13...Bone point detection section 15…Physical information acquisition department 20…Physical information acquisition device 27... Image capture state determination unit Vd…Photo image H…person SP: skeleton point IB…Physical information Vei…Evaluation index value VS: Image capture state VS1: Specific imaging condition

Claims

1. a skeleton point detection unit for detecting skeleton points of a person included in a photographed image; a physical information acquisition unit that acquires physical information of the person based on the detection of the skeleton points; an imaging state determination unit that determines whether or not an imaging state of the person appearing in the captured image corresponds to a specific imaging state that is specified based on a predetermined evaluation index value, the physical information acquisition unit does not execute acquisition of the physical information based on detection of the skeleton points when the specific imaging state is met, The imaging state determination unit determines that the imaging state corresponds to the specific imaging state when the luminance of the captured image is equal to or greater than a predetermined high luminance threshold value, using a luminance of the captured image as the evaluation index value of the imaging state, The imaging state determination unit is configured to determine the specific imaging state based on the luminance of the captured image by trimming an imaging area of ​​the person appearing in the captured image.

2. A skeleton point detection unit for detecting skeleton points of a person included in a photographed image; a physical information acquisition unit that acquires physical information of the person based on the detection of the skeleton points; an imaging state determination unit that determines whether or not an imaging state of the person appearing in the captured image corresponds to a specific imaging state that is specified based on a predetermined evaluation index value, the physical information acquisition unit does not execute acquisition of the physical information based on detection of the skeleton points when the specific imaging state is met, The image capturing state determination unit determines that the image capturing state corresponds to the specific image capturing state when the luminance of the captured image is equal to or lower than a predetermined low luminance threshold value, using a luminance of the captured image as the evaluation index value of the image capturing state, The imaging state determination unit is configured to determine the specific imaging state based on the luminance of the captured image by trimming an imaging area of ​​the person appearing in the captured image.

3. 3. The physical information acquisition device according to claim 1, a body movement waveform generating unit for generating a body movement waveform of the person, the imaging state determination unit determines that the imaging state corresponds to the specific imaging state when the magnitude of the body movement represented in the body movement waveform is equal to or greater than a predetermined body movement threshold value as the evaluation index value of the imaging state.

4. The physical information acquisition device according to any one of claims 1 to 3, The physical information acquisition unit acquires physical information of an occupant seated in a vehicle seat as the person's physical information.

Images