Occupant physical build detection device and occupant physical build detection method

Abstract

An occupant physique detection device (2) is configured so as to comprise a photographed image acquisition unit (11) that acquires a photographed image in which an occupant of a vehicle is captured from a camera (1) that photographs the occupant; and a skeletal point detection unit (12) that, from the photographed imaged acquired by the photographed image acquisition unit (11), detects three or more skeletal points that have no obstacles between said points and the camera (1) and that can be used to estimate the physique of the occupant among five or more predetermined skeletal points, including skeletal points on both shoulders and skeletal points on both hips of the occupant, and that outputs respective position coordinates on the photographed image for the three or more skeletal points. The occupant physique detection device (2) also comprises: an area calculation unit (13) that calculates the area of a polygon having the skeletal points as apexes, using the position coordinates of the skeletal points outputted from the skeletal point detection unit (12); and a physique estimation unit (14) that estimates the physique of the occupant from the area of the polygon calculated by the area calculation unit (13).

Description

[Technical field]

[0001] The present disclosure relates to an occupant physical size detection device and an occupant physical size detection method. [Background technology]

[0002] 2. Description of the Related Art There is an occupant physical build detection device that detects the physical build of a vehicle occupant based on a captured image in which the vehicle occupant is photographed. Patent Document 1 discloses such an occupant physical build detection device that includes an acquisition unit, a calculation unit, and a discrimination unit. The acquisition unit acquires a captured image of the vehicle occupant from a camera that captures the vehicle occupant. The calculation unit detects each of the skeletal points at both shoulders and both waists of the occupant from the captured image acquired by the acquisition unit, and calculates the area of ​​the occupant's trunk from the position coordinates of each of the skeletal points on the captured image. The trunk area is the area of ​​the occupant's torso. The discrimination unit discriminates the occupant's physique from the trunk area calculated by the calculation unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-96946 A Summary of the Invention [Problem to be solved by the invention]

[0004] A passenger in a vehicle is usually seated in a seat. Therefore, the passenger's posture is often a seated posture. When the passenger is in a seated posture, depending on the installation position of the camera, occlusion may occur in which either the shoulder skeleton point or the waist skeleton point is blocked by, for example, the passenger's forearm, the passenger's hand, the passenger's thigh, or luggage. In the occupant physique detection device disclosed in Patent Document 1, unless all four skeletal points, namely the skeletal points of both shoulders and both waists, are in a detectable state, the calculation unit cannot detect any of the four skeletal points, thereby causing a problem in that the discrimination unit cannot discriminate the occupant's physique.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to obtain an occupant physique detection device that can increase the number of occupant conditions from which the occupant's physique can be estimated compared to conventional methods. [Means for solving the problem]

[0006] The occupant physique detection device according to the present disclosure includes an image acquisition unit that acquires an image in which an occupant is captured from a camera that captures an image of the vehicle occupant, and a skeleton point detection unit that detects, from the image acquired by the image acquisition unit, three or more skeleton points that are not bound to the camera by obstacles and can be used to estimate the occupant's physique, among five or more predetermined skeleton points including skeleton points of both shoulders and both waists of the occupant, and outputs position coordinates of each of the three or more skeleton points on the captured image.The occupant physique detection device also includes an area calculation unit that calculates the area of ​​a polygon having each skeleton point as a vertex, using the position coordinates of each skeleton point output from the skeleton point detection unit, and a physique estimation unit that estimates the occupant's physique from the area of ​​the polygon calculated by the area calculation unit. The skeleton point detection unit includes a skeleton point search unit that searches for three or more skeleton points that are free from obstacles between the camera and can be used to estimate the physique of the occupant, among five or more predetermined skeleton points including skeleton points of both shoulders and both waists of the occupant, from the photographed image acquired by the photographed image acquisition unit, and a skeleton point selection unit that selects three skeleton points from the three or more skeleton points searched for by the skeleton point search unit, and outputs position coordinates of each of the selected three skeleton points on the photographed image to the area calculation unit. is. Effect of the Invention

[0007] According to the present disclosure, the number of occupant conditions for which the occupant's physical build can be estimated can be increased compared to conventional cases. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram showing an occupant physique detection device 2 according to a first embodiment. [Diagram 2] 1 is a hardware configuration diagram showing hardware of an occupant physique detection device 2 according to a first embodiment. [Diagram 3] FIG. 1 is a hardware configuration diagram of a computer in the case where the occupant physical build detection device 2 is realized by software, firmware, or the like. [Figure 4] 4 is a flowchart showing an occupant physique detection method which is a processing procedure of the occupant physique detection device 2. [Diagram 5] FIG. 2 is an explanatory diagram showing an example of a captured image in which an occupant is captured. [Figure 6] 1 is an explanatory diagram showing the correspondence between the area S of a triangle and physique P. [Figure 7] 4 is an explanatory diagram showing an example of control of an airbag or the like corresponding to a physique P of an occupant. FIG. [Figure 8] FIG. 11 is a configuration diagram showing an occupant physique detection device 2 according to a second embodiment. [Figure 9] FIG. 11 is a hardware configuration diagram showing hardware of an occupant physique detection device 2 according to a second embodiment. [Figure 10] FIG. 13 is an explanatory diagram showing a state in which an occupant has his / her arms lowered. [Figure 11] FIG. 13 is an explanatory diagram showing a state in which an occupant raises his / her arms in the vehicle width direction. [Figure 12] FIG. 13 is an explanatory diagram showing a state in which an occupant raises his / her arms in the vehicle width direction. [Figure 13] FIG. 11 is a configuration diagram showing an occupant physique detection device 2 according to a third embodiment. [Figure 14] FIG. 11 is a hardware configuration diagram showing hardware of an occupant physique detection device 2 according to a third embodiment. [Figure 15] FIG. 13 is an explanatory diagram showing a state in which an occupant has his / her arms lowered. [Figure 16] FIG. 13 is an explanatory diagram showing a state in which an occupant raises his / her arms in the direction of travel. [Figure 17] FIG. 11 is a configuration diagram showing an occupant physique detection device 2 according to a fourth embodiment. [Figure 18] FIG. 11 is a hardware configuration diagram showing hardware of an occupant physique detection device 2 according to a fourth embodiment. [Figure 19] FIG. 13 is an explanatory diagram showing a state in which an occupant has his / her arms lowered. [Figure 20] FIG. 11 is a configuration diagram showing an occupant physique detection device 2 according to a fifth embodiment. [Figure 21] FIG. 13 is a hardware configuration diagram showing hardware of an occupant physique detection device 2 according to a fifth embodiment. [Figure 22] FIG. 11 is an explanatory diagram showing a state in which an occupant is sitting closer to the rear window of the vehicle than the appropriate position for calculating the area S of the triangle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In order to describe the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1 FIG. 1 is a configuration diagram showing an occupant physical build detection device 2 according to the first embodiment. FIG. 2 is a hardware configuration diagram showing the hardware of the occupant physical build detection device 2 according to the first embodiment. In FIG. 1, a camera 1 is realized by, for example, a video camera, an infrared camera, a visible light camera, or an ultraviolet camera. The camera 1 is installed, for example, near the center of the dashboard in the vehicle width direction, or near the center of the ceiling of the vehicle in the vehicle width direction. The camera 1 captures an image of a vehicle occupant and outputs image data representing the captured image in which the occupant appears to an occupant physical build detection device 2. The installation position of camera 1 is not limited to near the center of the dashboard, but may be, for example, a position on the dashboard directly facing the driver's seat or directly facing the passenger seat.

[0011] The occupant physique detection device 2 includes a photographed image acquisition unit 11, a skeleton point detection unit 12, an area calculation unit 13, and a physique estimation unit . The occupant physical build detection device 2 is a device that estimates the occupant's physical build based on the captured image represented by the image data.

[0012] The photographed image acquisition unit 11 is realized by, for example, a photographed image acquisition circuit 21 shown in FIG. The captured image acquisition unit 11 acquires, from the camera 1, image data representing a captured image in which an occupant is captured. The photographed image acquisition unit 11 outputs the image data to the skeleton point detection unit 12 .

[0013] The skeleton point detection unit 12 is realized by, for example, a skeleton point detection circuit 22 shown in FIG. The skeleton point detection unit 12 includes a skeleton point search unit 12a and a skeleton point selection unit 12b. The skeleton point detection unit 12 acquires image data from the photographed image acquisition unit 11 . The skeleton point detection unit 12 detects, from the captured image represented by the image data, three or more skeleton points out of five or more predetermined skeleton points including the skeleton points of both shoulders and both waists of the occupant that are free from obstacles between them and the camera 1 and that can be used to estimate the occupant's physique.

[0014] A skeleton point with no obstacle between it and the camera 1 is a skeleton point with no occlusion occurring. For example, for a skeleton point in the waist, the occupant's forearm, hand, thigh, luggage, etc. may be an obstacle. For example, for a skeleton point in the shoulder, the occupant's hand, luggage, etc. may be an obstacle. For example, for a skeleton point in the elbow, the occupant's torso, luggage, etc. may be an obstacle. Examples of the five or more predetermined skeletal points include a skeletal point of the left shoulder, a skeletal point of the right shoulder, a skeletal point of the left waist, a skeletal point of the right waist, a skeletal point of the left elbow, a skeletal point of the right elbow, a midpoint between the left clavicle and the right clavicle (hereinafter referred to as the "first midpoint"), or a midpoint between the skeletal point of the left shoulder and the skeletal point of the right shoulder (hereinafter referred to as the "second midpoint"). The first midpoint is a point on the line segment connecting the right end of the left clavicle and the left end of the right clavicle, and is a position that is equidistant from the right end of the left clavicle and from the left end of the right clavicle. However, the first midpoint is not limited to a position that is strictly equidistant, and may be a position that is deviated from the position that is equidistant within a range that does not cause practical problems. The second intermediate point is a point on the line segment connecting the skeleton point of the left shoulder and the skeleton point of the right shoulder, and is a position that is approximately equal in distance from the skeleton point of the left shoulder and the skeleton point of the right shoulder. However, the second intermediate point is not limited to a position that is exactly equal in distance, and may be a position that is shifted from the position that is equal in distance within a range that does not cause practical problems.

[0015] The skeleton point detection section 12 outputs to the area calculation section 13 the position coordinates of each of the three or more skeleton points on the captured image. 1, for convenience of explanation, the skeleton point detection unit 12 detects three skeleton points. However, this is merely an example, and the skeleton point detection unit 12 may detect four or more skeleton points and output the position coordinates of each of the four or more skeleton points on the captured image to the area calculation unit 13.

[0016] The skeleton point search unit 12a searches for three or more skeleton points from among five or more predetermined skeleton points, including the skeleton points of both shoulders and both waists of the occupant, from the captured image represented by the image data, which are skeleton points that have no obstacles between them and the camera 1 and can be used to estimate the occupant's physique. The skeleton point selection unit 12b selects three skeleton points from the three or more skeleton points searched for by the skeleton point search unit 12a, and outputs the position coordinates of each of the selected three skeleton points on the captured image to the area calculation unit 13.

[0017] The area calculation unit 13 is realized by, for example, an area calculation circuit 23 shown in FIG. The area calculation unit 13 acquires the position coordinates of each of the three or more skeleton points from the skeleton point detection unit 12. The area calculation unit 13 uses the position coordinates of each skeleton point to calculate the area of ​​a polygon having each skeleton point as a vertex. The area calculation unit 13 outputs the area calculation result to the physique estimation unit 14. 1, for ease of explanation, it is assumed that the skeleton point detection unit 12 detects three skeleton points that can be used to estimate the occupant's physical build, and outputs the position coordinates of each of the three skeleton points to the area calculation unit 13. In this case, the area calculation unit 13 uses the position coordinates of each skeleton point to calculate the area of ​​a triangle as the area of ​​a polygon having each skeleton point as a vertex. If skeleton point detection unit 12 outputs, for example, the position coordinates of each of four skeleton points to area calculation unit 13, area calculation unit 13 calculates the area of ​​a quadrangle having the four skeleton points as vertices. If skeleton point detection unit 12 outputs, for example, the position coordinates of each of five skeleton points to area calculation unit 13, area calculation unit 13 calculates the area of ​​a pentagon having the five skeleton points as vertices.

[0018] The physique estimation unit 14 is realized by, for example, a physique estimation circuit 24 shown in FIG. The physique estimation unit 14 acquires the area calculation result from the area calculation unit 13. The physique estimation unit 14 estimates the physique of the occupant from the area indicated by the calculation result.

[0019] In Fig. 1, it is assumed that each of the components of the occupant physique detection device 2, that is, the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 13, and the physique estimation unit 14, is realized by dedicated hardware as shown in Fig. 2. In other words, it is assumed that the occupant physique detection device 2 is realized by a photographed image acquisition circuit 21, a skeleton point detection circuit 22, an area calculation circuit 23, and a physique estimation circuit 24. Each of the photographed image acquisition circuit 21, the skeletal point detection circuit 22, the area calculation circuit 23 and the body size estimation circuit 24 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.

[0020] The components of the occupant physical size detection device 2 are not limited to those realized by dedicated hardware, and the occupant physical size detection device 2 may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the memory of a computer. The computer means hardware that executes the program, and includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0021] FIG. 3 is a hardware configuration diagram of a computer in the case where the occupant physical build detection device 2 is realized by software, firmware, or the like. When the occupant physique detection device 2 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 13, and the physique estimation unit 14 is stored in the memory 31. Then, a processor 32 of the computer executes the program stored in the memory 31.

[0022] 2 shows an example in which each of the components of the occupant physical size detection device 2 is realized by dedicated hardware, and Fig. 3 shows an example in which the occupant physical size detection device 2 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the occupant physical size detection device 2 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0023] Next, the operation of the occupant physical build detection device 2 shown in FIG. 1 will be described. Physical build generally refers to the external appearance of the body. For example, body measurements such as height, weight, or chest circumference indicate the size of the body as the external appearance of the body. Therefore, these body measurements can be indicators of physical build. The area of ​​the occupant's torso also represents the size of the body, and therefore the area of ​​the torso can be an index of the physique. It can be said that the larger the area of ​​the torso, the larger the physique. The area of ​​the occupant's torso is roughly the area of ​​a rectangle enclosed by the skeletal points of the left shoulder, right shoulder, left hip, and right hip of the occupant. Therefore, each of the skeletal points of the left shoulder, right shoulder, left hip, and right hip is a skeletal point that can be used to estimate the physique of the occupant.

[0024] The length of the upper arm is generally proportional to height, and the taller a person is, the longer the upper arm tends to be. Therefore, the length of the upper arm can be an indicator of physical constitution, similar to height. The length of the upper arm is the distance between the skeletal point of the shoulder and the skeletal point of the elbow. Since the length of the upper arm can be an index of physique, an area proportional to the length of the upper arm, specifically, the area of ​​a rectangle enclosed by the left shoulder skeletal point, the right shoulder skeletal point, the left elbow skeletal point, and the right elbow skeletal point, can also be an index of physique. The larger the area of ​​the rectangle, the larger the physique. Therefore, each of the left shoulder skeletal point, the right shoulder skeletal point, the left elbow skeletal point, and the right elbow skeletal point is a skeletal point that can be used to estimate the occupant's physique. For the same reason, the area of ​​a triangle enclosed by the left shoulder skeleton point, the right shoulder skeleton point, and the left arm elbow skeleton point or the right arm elbow skeleton point can also be an index of physique. The area of ​​the triangle is approximately half the area of ​​the above-mentioned rectangle, and it can be said that the larger the area of ​​the triangle, the larger the physique. Therefore, the left shoulder skeleton point, the right shoulder skeleton point, the left arm elbow skeleton point or the right arm elbow skeleton point are skeleton points that can be used to estimate the occupant's physique.

[0025] For the same reason, the area of ​​a triangle enclosed by the skeletal point of the occupant's left shoulder or right shoulder, the skeletal point of the elbow of the left arm or the skeletal point of the elbow of the right arm, and the first intermediate point can also be an index of the physique. The area of ​​the triangle is approximately half the area of ​​the above-mentioned rectangle, and it can be said that the larger the area of ​​the triangle, the larger the physique. Therefore, the skeletal point of the occupant's left shoulder or right shoulder, the skeletal point of the elbow of the left arm or the skeletal point of the elbow of the right arm, and the first intermediate point are skeletal points that can be used to estimate the occupant's physique. It should be noted that if the right end of the left clavicle is searched for as a skeleton point, and the left end of the right clavicle is searched for as a skeleton point, then the first midpoint can be detected.

[0026] For the same reason, the area of ​​a triangle enclosed by the skeletal point of the left shoulder or the right shoulder, the skeletal point of the left elbow or the skeletal point of the right elbow, and the second intermediate point of the occupant can also be an index of the physique. The area of ​​the triangle is approximately half the area of ​​the above-mentioned rectangle, and it can be said that the larger the area of ​​the triangle, the larger the physique. The skeletal point of the left shoulder or the right shoulder, the skeletal point of the left elbow or the skeletal point of the right elbow, and the second intermediate point are skeletal points that can be used to estimate the physique of the occupant. It should be noted that if the skeleton points of the left shoulder and the right shoulder are found, the second intermediate point can be detected. When only one of the left shoulder skeleton point or the right shoulder skeleton point is detected, by calculating the vertical distance between the detected shoulder skeleton point and the first midpoint, it is possible to detect a position vertically lower than the first midpoint by that distance as the second midpoint.

[0027] FIG. 4 is a flowchart showing an occupant physique detection method which is a processing procedure of the occupant physique detection device 2. As shown in FIG. Camera 1 photographs the vehicle occupants. The camera 1 outputs image data representing a captured image such as that shown in FIG. FIG. 5 is an explanatory diagram showing an example of a captured image in which an occupant is captured. The upper body and part of the lower body of the occupant are captured in the captured image shown in Fig. 5. Specifically, the shoulders, elbows, collarbone, and chest of the occupant are captured in the captured image shown in Fig. 5. In the example of Fig. 5, the occupant's thighs cover the occupant's waist, so it may be difficult to detect the skeleton points of the waist. In this case, the occupant's thighs are an obstacle between the camera 1 and the waist. However, if an occupant sitting in the passenger seat, for example, has reclined the seat significantly, the occupant's waist is unlikely to be blocked by the thighs, so the waist skeleton points may still be detected.

[0028] The captured image acquisition unit 11 of the occupant physical build detection device 2 acquires image data representing a captured image from the camera 1 (step ST1 in FIG. 4). The photographed image acquisition unit 11 outputs the image data to the skeleton point detection unit 12 .

[0029] The skeleton point detection unit 12 acquires image data from the photographed image acquisition unit 11 . The skeleton point detection unit 12 detects three skeleton points from among five or more predetermined skeleton points, including the skeleton points of both shoulders and both waists of the occupant, that are free from obstacles between them and the camera 1 and that can be used to estimate the occupant's physique (step ST2 in Figure 4). The skeleton point detection section 12 outputs the position coordinates of each of the three skeleton points on the captured image to the area calculation section 13 (step ST3 in FIG. 4).

[0030] The skeleton point detection process performed by the skeleton point detection unit 12 will now be described in detail. The skeleton point search unit 12a searches for three or more skeleton points that can be used to estimate the occupant's physique from among five or more predetermined skeleton points, including the skeleton points of both shoulders and both waists of the occupant, from the captured image represented by the image data. The skeleton point search process itself is a known technique, and therefore a detailed description thereof will be omitted. One known technique is a skeleton estimation technique called "Open Pose." In the example of Figure 5, three or more skeleton points that can be used to estimate the occupant's physique are searched for, including a left shoulder skeleton point, a right shoulder skeleton point, a left arm elbow skeleton point, a right arm elbow skeleton point, a first intermediate point, and a second intermediate point.

[0031] The skeleton point selection unit 12b selects three skeleton points that can be used to estimate the physique of an occupant from among the three or more skeleton points searched for by the skeleton point search unit 12a. Specifically, the skeleton point selection unit 12b selects, as a first skeleton point, one of the skeleton point on the left shoulder and the skeleton point on the right shoulder from among the three or more skeleton points. For example, if the distance from camera 1 to the left shoulder is shorter than the distance from camera 1 to the right shoulder, the skeleton point selection unit 12b selects the skeleton point of the left shoulder as the first skeleton point. For example, if the distance from camera 1 to the left shoulder is longer than the distance from camera 1 to the right shoulder, skeleton point selection unit 12b selects the skeleton point on the right shoulder as the first skeleton point. For example, if the vehicle is a right-hand drive vehicle, the occupant is a driver, and camera 1 is installed near the center of the dashboard in the width direction of the vehicle, the distance from camera 1 to the left shoulder will be shorter than the distance from camera 1 to the right shoulder. For example, if the vehicle is a right-hand drive vehicle, the occupant is sitting in the passenger seat, and camera 1 is installed near the center of the dashboard in the width direction of the vehicle, the distance from camera 1 to the left shoulder will be longer than the distance from camera 1 to the right shoulder.

[0032] Next, the skeleton point selection unit 12b selects, as a second skeleton point, one of the skeleton point of the elbow of the left arm and the skeleton point of the elbow of the right arm from among the three or more skeleton points. For example, if the skeleton point of the left shoulder is selected as the first skeleton point, the skeleton point selection unit 12b selects the skeleton point of the elbow of the left arm as the second skeleton point. For example, if the skeleton point of the right shoulder is selected as the first skeleton point, the skeleton point selection unit 12b selects the skeleton point of the elbow of the right arm as the second skeleton point. Here, if the skeleton point of the left shoulder is selected as the first skeleton point, the skeleton point selection unit 12b selects the skeleton point of the elbow of the left arm as the second skeleton point. However, this is merely an example, and the skeleton point selection unit 12b may select the skeleton point of the elbow of the right arm as the second skeleton point. For example, if the skeleton point of the elbow of the left arm is hidden by luggage, the skeleton point selection unit 12b selects the skeleton point of the elbow of the right arm as the second skeleton point. Here, if the skeleton point of the right shoulder is selected as the first skeleton point, the skeleton point selection unit 12b selects the skeleton point of the elbow of the right arm as the second skeleton point. However, this is merely an example, and the skeleton point selection unit 12b may select the skeleton point of the elbow of the left arm as the second skeleton point. For example, if the skeleton point of the elbow of the right arm is hidden by luggage, the skeleton point selection unit 12b selects the skeleton point of the elbow of the left arm as the second skeleton point.

[0033] Next, from among the three or more skeleton points, the skeleton point selection unit 12b selects, as a third skeleton point, one of the first intermediate point, the second intermediate point, or the left shoulder skeleton point and the right shoulder skeleton point, which has not been selected as the first skeleton point (hereinafter referred to as an "unselected shoulder skeleton point"). The skeleton point selected as the third skeleton point may be any one of the first midpoint, the second midpoint, and the unselected shoulder skeleton point. If priorities are set for the first intermediate point, the second intermediate point, and the unselected shoulder skeleton points, the skeleton point selection unit 12b can use a selection method that gives priority to selecting skeleton points with high priorities.

[0034] For example, assume that the priority of an unselected shoulder skeleton point is higher than the priority of a first intermediate point, which in turn is higher than the priority of a second intermediate point. In this case, if an unselected shoulder skeleton point has been found by the skeleton point search section 12a, the skeleton point selection section 12b selects the unselected shoulder skeleton point as the third skeleton point. When the skeleton point search unit 12a has not searched for an unselected shoulder skeleton point and the first intermediate point has been searched for, the skeleton point selection unit 12b selects the first intermediate point as the third skeleton point. If the unselected shoulder skeleton point is blocked by luggage or the like, the skeleton point search unit 12a will not search for the unselected shoulder skeleton point. When neither the unselected shoulder skeleton point nor the first intermediate point has been searched for by the skeleton point search unit 12a, if the second intermediate point has been searched for, the skeleton point selection unit 12b selects the second intermediate point as the third skeleton point. If the right end of the left clavicle or the left end of the right clavicle is blocked by luggage or the like, the skeleton point search unit 12a will not search for the first intermediate point.

[0035] For example, assume that the priority of a first intermediate point is higher than the priority of a second intermediate point, which in turn is higher than the priority of an unselected shoulder skeleton point. In this case, if the first intermediate point has been found by the skeleton point search section 12a, the skeleton point selection section 12b selects the first intermediate point as the third skeleton point. When the skeleton point searching section 12a has not found the first intermediate point but has found the second intermediate point, the skeleton point selecting section 12b selects the second intermediate point as the third skeleton point. When neither the first nor the second intermediate point has been searched for by the skeleton point search unit 12a, if an unselected shoulder skeleton point has been searched for, the skeleton point selection unit 12b selects the unselected shoulder skeleton point as the third skeleton point. If either the left or right shoulder skeleton point has not been searched for, and the first intermediate point has not been searched for, the skeleton point search unit 12a does not search for the second intermediate point.

[0036] Finally, the skeleton point selection unit 12b outputs the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point to the area calculation unit 13. 5, the skeleton point of the left shoulder, the skeleton point of the right shoulder, the skeleton point of the left elbow, the skeleton point of the right elbow, the first intermediate point, and the second intermediate point are searched for. However, this is merely an example, and for example, the skeleton point of the left waist or the skeleton point of the right waist may be searched for. When a skeleton point of the left waist or a skeleton point of the right waist is searched for, the skeleton point selection unit 12b may, for example, select the skeleton point of the left waist or the skeleton point of the right waist as the second skeleton point.

[0037] The area calculation unit 13 acquires from the skeleton point detection unit 12 the position coordinates of each of the first skeleton point, the second skeleton point, and the third skeleton point. The area calculation unit 13 calculates the area S of a triangle having the first skeleton point, the second skeleton point, and the third skeleton point as vertices, using the respective position coordinates (step ST4 in FIG. 4). The area calculation unit 13 outputs the calculation result of the area S to the physique estimation unit 14.

[0038] The physique estimation unit 14 acquires the calculation result of the area S from the area calculation unit 13. The physique estimation unit 14 estimates the physique P of the occupant from the area S of the triangle indicated by the calculation result (step ST5 in FIG. 4). As described above, the area S of the triangle can be an index of the physique. Therefore, the physique estimation unit 14 may output the area S itself as an index of the physique, but here, in order to qualitatively distinguish the physique P of the occupant, the physique estimation unit 14 classifies the occupant's physique P into N stages based on the area S of the triangle. N is an integer of 2 or more. The physique estimation unit 14 outputs the estimation result of the physique P to, for example, a vehicle control device (not shown). The vehicle control device is, for example, a device that adjusts the tensile strength of a seat belt of the vehicle or the output strength of an airbag of the vehicle.

[0039] The physique estimation process performed by the physique estimation unit 14 will now be described in detail. When the physique estimation unit 14 classifies the occupant's physique P into N stages, the physique estimation unit 14 sets (N-1) threshold values ​​Th n n=1, , N-1. The threshold Th n is a threshold value for the area S of a triangle, and is stored, for example, in the internal memory of the physique estimation unit 14. n may be provided from outside the occupant physical build detection device 2. Th1 <Th2<····<Th N-2 <Th N-1

[0040] The physique estimation unit 14 calculates the area S of the triangle and (N-1) threshold values ​​Th n Compare with (n=1, ,N-1). The physique estimation unit 14 calculates the area S and the threshold value Th as follows: n Based on the comparison result, the occupant's physical size P is estimated. However, for example, the area S of a triangle when the first skeletal point is the skeletal point of the right shoulder, the second skeletal point is the skeletal point of the elbow of the right arm, and the third skeletal point is the skeletal point of the left shoulder is different from the area S of a triangle when the first skeletal point is the skeletal point of the right shoulder, the second skeletal point is the skeletal point of the elbow of the right arm, and the third skeletal point is the first intermediate point. That is, even for the same occupant, the area S of the triangle is different for each possible combination of the first skeleton point, the second skeleton point, and the third skeleton point. Therefore, the correspondence between the area S and the physique P is different for each possible combination, and therefore, (N-1) threshold values ​​Th n is stored in the internal memory of the physique estimation unit 14.

[0041] Comparison results physique S <Th1→ P1 Th1≦S <Th2→ P2 Th2≦S <Th3→ P3 Th3≦S <Th4→ P4 : Th n-2 ≦S <Th n-1 → P N-1 Th n-1≦S → P N P1 <P2<····<P N-1 <P N

[0042] FIG. 6 is an explanatory diagram showing the correspondence relationship between the area S of a triangle and the physique P. FIG. 6 shows that the area S and the physique P are correlated, with the horizontal axis representing the area S of the triangle and the vertical axis representing the physique P of the occupant. In the example of FIG. 6, the physique P of the occupant is classified into one of P1, P2, P3, and P4 based on the area S.

[0043] In the passenger physique detection device 2 shown in FIG. 1, the physique estimation unit 14 calculates the area S and the threshold value Th n The physique P of the occupant is estimated based on the comparison result. Even if the area S calculated by the area calculation unit 13 is the area of ​​a polygon other than a triangle, there is a correspondence relationship between the area of ​​the polygon and the physique P. Therefore, the physique estimation unit 14 estimates the physique P of the occupant based on the comparison result between the area S of the polygon other than a triangle and the threshold value Th n By comparing this with the passenger's physical size P, the passenger's physical size P can be estimated. In the passenger physique detection device 2 shown in FIG. 1, the physique estimation unit 14 calculates the area S and the threshold value Th n The physique P of the occupant is estimated based on the comparison result with. However, this is merely an example, and for example, the physique estimation unit 14 may calculate the physique P from the area S as shown in the following formula (1). P = α × S (1) In equation (1), α is a proportionality constant greater than 1, for example.

[0044] A control device of the vehicle (not shown) obtains the estimation result of the physique P from the physique estimation unit 14. The vehicle control device controls, for example, the size of an airbag to be inflated when a vehicle collision occurs, as well as the tensile strength of a seat belt when a collision occurs. The control device controls the size of the airbag and the tensile strength of the seat belt in accordance with the estimated result of the physique P, thereby making it possible to reduce injury to the occupant in the event of a collision. FIG. 7 is an explanatory diagram showing an example of control of an airbag or the like corresponding to the physique P of an occupant. The example of FIG. 7 shows that the larger the occupant's physique P is, the larger the size of the airbag to be inflated will be, and the stronger the tensile strength of the seat belt will be.

[0045] In the above-described first embodiment, the occupant physique detection device 2 is configured to include a photographed image acquisition unit 11 that acquires a photographed image in which an occupant is captured from a camera 1 that photographs the vehicle occupant, and a skeleton point detection unit 12 that detects, from the photographed image acquired by the photographed image acquisition unit 11, three or more skeleton points that are skeleton points that have no obstacle between them and the camera 1 and can be used to estimate the occupant's physique, among five or more predetermined skeleton points including skeleton points of both shoulders and skeleton points of both waists of the occupant, and outputs position coordinates of each of the three or more skeleton points on the photographed image. The occupant physique detection device 2 also includes an area calculation unit 13 that calculates the area of ​​a polygon having each skeleton point as a vertex, using the position coordinates of each skeleton point output from the skeleton point detection unit 12, and a physique estimation unit 14 that estimates the occupant's physique from the area of ​​the polygon calculated by the area calculation unit 13. Thus, the occupant physique detection device 2 can increase the number of occupant states for which the occupant's physique can be estimated, compared to the conventional case.

[0046] Embodiment 2 In the second embodiment, the angle θ between the first straight line L1 and the second straight line L2 a We will explain an occupant body size detection device 2 that has an area calculation unit 15 that calculates the area S of a triangle using the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point only when the angle is within the allowable angle range. The first straight line L1 is a straight line connecting the first skeleton point and the second skeleton point, and the second straight line L2 is a straight line connecting the first skeleton point and the third skeleton point. In the occupant physique detection device 2 according to the second embodiment, if the first skeleton point is a skeleton point of the left shoulder of the occupant, the second skeleton point is a skeleton point of the elbow of the left arm of the occupant. If the first skeleton point is a skeleton point of the right shoulder of the occupant, the second skeleton point is a skeleton point of the elbow of the right arm of the occupant.

[0047] Fig. 8 is a configuration diagram showing an occupant physical build detection device 2 according to embodiment 2. In Fig. 8, the same reference numerals as in Fig. 1 denote the same or corresponding parts, and therefore description thereof will be omitted. Fig. 9 is a hardware configuration diagram showing the hardware of an occupant physical build detection device 2 according to embodiment 2. In Fig. 9, the same reference numerals as in Fig. 2 denote the same or corresponding parts, and therefore description thereof will be omitted.

[0048] The area calculation unit 15 is realized by, for example, an area calculation circuit 25 shown in FIG. The area calculation unit 15 acquires the position coordinates of each of the three skeleton points from the skeleton point detection unit 12. The area calculation unit 15 uses the position coordinates of each skeleton point to calculate the area of ​​a triangle having each skeleton point as a vertex. However, the area calculation unit 15 calculates the angle θ a is within the allowable angle range, the area S of the triangle is calculated using the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point. a If is outside the allowable angle range, the area calculation unit 15 does not perform the process of calculating the area S of the triangle. The area calculation unit 15 outputs the calculation result of the area S to the physique estimation unit 14.

[0049] In Fig. 8, it is assumed that the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 15 and the physique estimation unit 14, which are the components of the occupant physique detection device 2, are each realized by dedicated hardware as shown in Fig. 9. In other words, it is assumed that the occupant physique detection device 2 is realized by a photographed image acquisition circuit 21, a skeleton point detection circuit 22, an area calculation circuit 25 and a physique estimation circuit 24. Each of the photographed image acquisition circuit 21, the skeleton point detection circuit 22, the area calculation circuit 25 and the body size estimation circuit 24 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of these.

[0050] The components of the occupant physical size detection device 2 are not limited to those realized by dedicated hardware, and the occupant physical size detection device 2 may be realized by software, firmware, or a combination of software and firmware. When the occupant physique detection device 2 is realized by software, firmware, or the like, programs for causing a computer to execute the respective processing procedures in the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 15, and the physique estimation unit 14 are stored in a memory 31 shown in Fig. 3. Then, a processor 32 shown in Fig. 3 executes the programs stored in the memory 31.

[0051] 9 shows an example in which each of the components of the occupant physical size detection device 2 is realized by dedicated hardware, and FIG 3 shows an example in which the occupant physical size detection device 2 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the occupant physical size detection device 2 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0052] Next, the operation of the occupant physique detection device 2 shown in Fig. 8 will be described. Since the components other than the area calculation unit 15 are the same as those of the occupant physique detection device 2 shown in Fig. 1, only the operation of the area calculation unit 15 will be described here. When the occupant has his / her arms lowered as shown in FIG. 10, the angle θ between the first straight line L1 and the second straight line L2 is a is an angle close to 90 degrees. a is the angle between the first straight line L1 and the second straight line L2 in a two-dimensional plane defined by the vehicle width direction and the vehicle vertical direction (hereinafter referred to as "in the first plane"). FIG. 10 is an explanatory diagram showing a state in which an occupant has his / her arms lowered. FIG. 10 shows an example in which the third skeleton point is the skeleton point of the left shoulder.

[0053] When the passenger raises his / her arm in the width direction of the vehicle as shown in FIG. 11, the angle θ between the first straight line L1 and the second straight line L2 is a is an angle greater than 90 degrees in the first plane. The area S of the triangle in this case may be smaller than the area S when the arm is lowered. In addition, when the passenger raises his / her arm in the opposite direction to the direction shown in FIG. 11 as shown in FIG. 12, the angle θ a is an angle smaller than 90 degrees in the first plane. The area S of the triangle in this case may be smaller than the area S when the arm is lowered. When the occupant has his / her arms raised, the area S of the triangle may become smaller and may not accurately represent the occupant's body size P. Each of Fig. 11 and Fig. 12 is an explanatory diagram showing a state in which an occupant raises his / her arms in the vehicle width direction. 11 and 12 show an example in which the third skeleton point is the skeleton point of the left shoulder.

[0054] The area calculation unit 15 acquires from the skeleton point detection unit 12 the position coordinates of each of the first skeleton point, the second skeleton point, and the third skeleton point. The area calculation unit 15 has information indicating the allowable angle range. The allowable angle range is θ L ~θ H θ L is an angle smaller than 90 degrees, and θ H is an angle greater than 90 degrees. The area calculation unit 15 identifies a first straight line L1 connecting the first skeleton point and the second skeleton point from the position coordinates of each skeleton point, and identifies a second straight line L2 connecting the first skeleton point and the third skeleton point. Then, the area calculation unit 15 calculates the angle θ between the first straight line L1 and the second straight line L2. a Find the angle θ ais within the allowable angle range.

[0055] The area calculation unit 15 calculates the angle θ a If the angle is within the allowable angle range, the area S of the triangle is calculated using the coordinates of each position, and the calculation result of the area S is output to the physique estimation unit 14. The area calculation unit 15 calculates the angle θ a However, if it is outside the allowable angle range, the process of calculating the triangle area S is not performed. In this case, the physique estimation unit 14 does not perform the process of estimating the physique P.

[0056] In the above-described second embodiment, the occupant physique detection device 2 shown in FIG. 8 is configured so that, when the first skeletal point selected by the skeleton point selection unit 12b is the skeletal point of the left shoulder of the occupant and the second skeletal point is the skeletal point of the elbow of the left arm of the occupant, or when the first skeletal point selected by the skeleton point selection unit 12b is the skeletal point of the right shoulder of the occupant and the second skeletal point is the skeletal point of the elbow of the right arm of the occupant, the area calculation unit 15 calculates the area of ​​a triangle having the first skeletal point, the second skeletal point and the third skeletal point as vertices, respectively, using the position coordinates of the first skeletal point, the second skeletal point and the third skeletal point, only when an angle formed by a first straight line connecting the first skeletal point and the second skeletal point and a second straight line connecting the first skeletal point and the third skeletal point is within an allowable angle range. Therefore, the occupant physical size detection device 2 shown in Figure 8, like the occupant physical size detection device 2 shown in Figure 1, can increase the number of occupant states in which the occupant's physical size can be estimated compared to conventional methods, and can avoid physical size estimation processing in states where the estimation accuracy is low.

[0057] Embodiment 3 In the third embodiment, the angle θ between the first straight line L1 and the third straight line L3 b We will describe an occupant body size detection device 2 that has an area calculation unit 16 that calculates the area S of a triangle using the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point only when is equal to or greater than a first threshold value Th1. The third straight line L3 is a straight line connecting the wrist of the arm on which the skeleton point selected as the second skeleton point is located and the second skeleton point. In the occupant physique detection device 2 according to the third embodiment, if the first skeleton point is a skeleton point of the left shoulder of the occupant, the second skeleton point is a skeleton point of the elbow of the left arm of the occupant. If the first skeleton point is a skeleton point of the right shoulder of the occupant, the second skeleton point is a skeleton point of the elbow of the right arm of the occupant.

[0058] Fig. 13 is a configuration diagram showing an occupant physical build detection device 2 according to embodiment 3. In Fig. 13, the same reference numerals as in Fig. 1 denote the same or corresponding parts, and therefore description thereof will be omitted. Fig. 14 is a hardware configuration diagram showing the hardware of an occupant physical build detection device 2 according to embodiment 3. In Fig. 14, the same reference numerals as in Fig. 2 denote the same or corresponding parts, and therefore description thereof will be omitted.

[0059] The area calculation unit 16 is realized by, for example, an area calculation circuit 26 shown in FIG. The area calculation unit 16 acquires the position coordinates of each of the three skeleton points from the skeleton point detection unit 12. The area calculation unit 16 uses the position coordinates of each skeleton point to calculate the area S of a triangle having each skeleton point as a vertex. However, the area calculation unit 16 calculates the angle θ b is equal to or greater than the first threshold Th1, the area S of the triangle is calculated using the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point. b is smaller than the first threshold value Th1, the area calculation unit 16 does not perform the process of calculating the area S of the triangle. The area calculation unit 13 outputs the calculation result of the area S to the physique estimation unit 14.

[0060] 13, it is assumed that the captured image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 16 and the physique estimation unit 14, which are the components of the occupant physique detection device 2, are each realized by dedicated hardware as shown in Fig. 14. In other words, it is assumed that the occupant physique detection device 2 is realized by a captured image acquisition circuit 21, a skeleton point detection circuit 22, an area calculation circuit 26 and a physique estimation circuit 24. Each of the photographed image acquisition circuit 21, the skeleton point detection circuit 22, the area calculation circuit 26 and the body size estimation circuit 24 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of these.

[0061] The components of the occupant physical size detection device 2 are not limited to those realized by dedicated hardware, and the occupant physical size detection device 2 may be realized by software, firmware, or a combination of software and firmware. When the occupant physique detection device 2 is realized by software, firmware, or the like, programs for causing a computer to execute the respective processing procedures in the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 16, and the physique estimation unit 14 are stored in a memory 31 shown in Fig. 3. Then, a processor 32 shown in Fig. 3 executes the programs stored in the memory 31.

[0062] 14 shows an example in which each of the components of the occupant physical size detection device 2 is realized by dedicated hardware, while Fig. 3 shows an example in which the occupant physical size detection device 2 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the occupant physical size detection device 2 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0063] Next, the operation of the occupant physique detection device 2 shown in Fig. 13 will be described. Since the components other than the area calculation unit 16 are the same as those of the occupant physique detection device 2 shown in Fig. 1, only the operation of the area calculation unit 16 will be described here. When the occupant has his / her arms lowered as shown in FIG. 15, the angle θ between the first line L1 and the third line L3 is b is an angle close to 180 degrees. b is the angle between the first straight line L1 and the third straight line L3 in a two-dimensional plane defined by the traveling direction of the vehicle and the vertical direction of the vehicle (hereinafter referred to as "in the second plane"). FIG. 15 is an explanatory diagram showing a state in which an occupant has his / her arms lowered.

[0064] When the passenger raises his / her arm in the direction of travel of the vehicle as shown in FIG. 16, the angle θ b may have an angle smaller than 180 degrees in the second plane. The area S of the triangle may be smaller than the area S of the triangle when the arm is lowered. When the occupant raises his / her arms in the direction of travel, the area S of the triangle may become smaller and may not accurately represent the physique P of the occupant. FIG. 16 is an explanatory diagram showing a state in which a passenger raises his / her arms in the traveling direction.

[0065] The area calculation unit 16 acquires from the skeleton point detection unit 12 the position coordinates of each of the first skeleton point, the second skeleton point, and the third skeleton point. The area calculation unit 16 has information indicating the first threshold value Th1. The area calculation unit 16 specifies a first straight line L1 connecting the first skeletal point and the second skeletal point from the position coordinates of each skeletal point, and specifies a third straight line L3 connecting the second skeletal point and the wrist of the arm where the skeletal point selected as the second skeletal point is located. Then, the area calculation unit 16 calculates the angle θ between the first straight line L1 and the third straight line L3. b Find the angle θ b is equal to or greater than a first threshold value Th1. If the camera 1 is installed near the center of the dashboard in the vehicle width direction, the camera 1 captures the passenger at an angle to the traveling direction of the vehicle. b It is possible to obtain

[0066] The area calculation unit 16 calculates the angle θ b is equal to or greater than the first threshold value Th1, the area S of the triangle is calculated using the respective position coordinates, and the calculation result of the area S is output to the physique estimation unit . The area calculation unit 16 calculates the angle θ b is smaller than the first threshold value Th1, the process of calculating the triangle area S is not performed. In this case, the physique estimation unit 14 does not perform the process of estimating the physique P.

[0067] In the above-described third embodiment, the occupant physique detection device 2 shown in FIG. 13 is configured so that, when the first skeletal point selected by the skeleton point selection unit 12b is the skeletal point of the left shoulder of the occupant and the second skeletal point is the skeletal point of the elbow of the left arm of the occupant, or when the first skeletal point selected by the skeleton point selection unit 12b is the skeletal point of the right shoulder of the occupant and the second skeletal point is the skeletal point of the elbow of the right arm of the occupant, the area calculation unit 16 calculates the area of ​​a triangle having the first skeletal point, the second skeletal point, and the third skeletal point as vertices, using the position coordinates of the first skeletal point, the second skeletal point, and the third skeletal point, respectively, only when an angle formed by a first straight line connecting the first skeletal point and the second skeletal point and a third straight line connecting the wrist of the arm on which the skeleton point selected as the second skeletal point is located and the second skeletal point is equal to or greater than a first threshold value. Therefore, the occupant physical size detection device 2 shown in Figure 13, like the occupant physical size detection device 2 shown in Figure 1, can increase the number of occupant states in which the occupant's physical size can be estimated compared to conventional methods, and can avoid physical size estimation processing in states where the estimation accuracy is low.

[0068] Embodiment 4 In embodiment 4, an occupant body size detection device 2 is described that is equipped with an area calculation unit 17 that calculates an area S of a triangle using the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point only when the ratio of the distance D2 between the first skeleton point and the second skeleton point on the captured image to the distance D3 between the first skeleton point and the third skeleton point on the captured image is greater than or equal to a second threshold value Th2. In the occupant physique detection device 2 according to the fourth embodiment, if the first skeleton point is a skeleton point of the left shoulder of the occupant, the second skeleton point is a skeleton point of the elbow of the left arm of the occupant. If the first skeleton point is a skeleton point of the right shoulder of the occupant, the second skeleton point is a skeleton point of the elbow of the right arm of the occupant.

[0069] Fig. 17 is a configuration diagram showing an occupant physical build detection device 2 according to embodiment 4. In Fig. 17, the same reference numerals as in Fig. 1 denote the same or corresponding parts, and therefore description thereof will be omitted. Fig. 18 is a hardware configuration diagram showing the hardware of an occupant physical build detection device 2 according to embodiment 4. In Fig. 18, the same reference numerals as in Fig. 2 denote the same or corresponding parts, and therefore description thereof will be omitted.

[0070] The area calculation unit 17 is realized by, for example, an area calculation circuit 27 shown in FIG. The area calculation unit 17 acquires the position coordinates of each of the three skeleton points from the skeleton point detection unit 12. The area calculation unit 17 uses the position coordinates of each skeleton point to calculate the area S of a triangle having each skeleton point as a vertex. However, only when a ratio of a distance D2 between the first and second skeleton points on the captured image to a distance D3 between the first and third skeleton points on the captured image is equal to or greater than a second threshold value Th2, the area calculation unit 17 calculates the area S of the triangle using the position coordinates of each of the first, second, and third skeleton points. Therefore, if the ratio of the distance D2 to the distance D3 is smaller than the second threshold value Th2, the area calculation unit 17 does not perform the process of calculating the area S of the triangle. The area calculation unit 17 outputs the calculation result of the area S to the physique estimation unit 14.

[0071] 17, it is assumed that the captured image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 17 and the physique estimation unit 14, which are the components of the occupant physique detection device 2, are each realized by dedicated hardware as shown in Fig. 18. In other words, it is assumed that the occupant physique detection device 2 is realized by a captured image acquisition circuit 21, a skeleton point detection circuit 22, an area calculation circuit 27 and a physique estimation circuit 24. Each of the photographed image acquisition circuit 21, the skeleton point detection circuit 22, the area calculation circuit 27 and the body size estimation circuit 24 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of these.

[0072] The components of the occupant physical size detection device 2 are not limited to those realized by dedicated hardware, and the occupant physical size detection device 2 may be realized by software, firmware, or a combination of software and firmware. When the occupant physique detection device 2 is realized by software, firmware, or the like, programs for causing a computer to execute the respective processing procedures of the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 17, and the physique estimation unit 14 are stored in a memory 31 shown in Fig. 3. Then, a processor 32 shown in Fig. 3 executes the programs stored in the memory 31.

[0073] 18 shows an example in which each of the components of the occupant physical size detection device 2 is realized by dedicated hardware, while Fig. 3 shows an example in which the occupant physical size detection device 2 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the occupant physical size detection device 2 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0074] Next, the operation of the occupant physique detection device 2 shown in Fig. 17 will be described. Since the components other than the area calculation unit 17 are the same as those of the occupant physique detection device 2 shown in Fig. 1, only the operation of the area calculation unit 17 will be described here. When the occupant has his / her arms lowered as shown in Fig. 19, if the skeleton point selected as the third skeleton point is an unselected shoulder skeleton point, the ratio of distance D2 to distance D3 is approximately 2:1. As described above, the unselected shoulder skeleton point is either the left shoulder skeleton point or the right shoulder skeleton point, which has not been selected as the first skeleton point. If the skeleton point selected as the third skeleton point is the first midpoint or the second midpoint, the ratio of distance D2 to distance D3 is approximately 1:1. FIG. 19 is an explanatory diagram showing a state in which an occupant has his / her arms lowered.

[0075] When the occupant raises his / her arm within the second plane as shown in Fig. 16, the distance D2 on the captured image is shorter than when the arm is lowered downward. The distance D3 on the captured image does not change. Therefore, when the occupant raises his / her arm in the second plane, the ratio of distance D2 to distance D3 is smaller than when the occupant lowers his / her arm, and the area S of the triangle in this case may be smaller than the area S when the occupant lowers his / her arm. When the occupant raises his / her arms in the direction of travel, the area S of the triangle may become smaller and may not accurately represent the physique P of the occupant.

[0076] The area calculation unit 17 acquires from the skeleton point detection unit 12 the position coordinates of each of the first skeleton point, the second skeleton point, and the third skeleton point. The area calculation unit 17 has information indicating the second threshold value Th2. The second threshold value Th2 is a value smaller than 1 / 2 if the skeleton point selected as the third skeleton point is an unselected shoulder skeleton point. The second threshold value Th2 is a value smaller than 1 if the skeleton point selected as the third skeleton point is the first midpoint or the second midpoint. The area calculation unit 17 determines a distance D3 between the first skeleton point and the third skeleton point on the captured image from the position coordinates of each skeleton point, and determines a distance D2 between the first skeleton point and the second skeleton point on the captured image. Then, the area calculation unit 17 obtains a ratio D2 / D3 of the distance D2 to the distance D3, and determines whether or not the ratio D2 / D3 is equal to or greater than a second threshold value Th2.

[0077] If the ratio D2 / D3 is equal to or greater than the second threshold value Th2, the area calculation unit 17 calculates the area S of the triangle using the respective position coordinates, and outputs the calculation result of the area S to the physique estimation unit . If the ratio D2 / D3 is smaller than the second threshold value Th2, the area calculation unit 17 does not perform the process of calculating the triangle area S. In this case, the physique estimation unit 14 does not perform the process of estimating the physique P.

[0078] In the above-described fourth embodiment, the occupant physique detection device 2 shown in FIG. 17 is configured so that, when the first skeletal point selected by the skeleton point selection unit 12b is the skeleton point of the left shoulder of the occupant and the second skeletal point is the skeleton point of the elbow of the left arm of the occupant, or when the first skeletal point selected by the skeleton point selection unit 12b is the skeleton point of the right shoulder of the occupant and the second skeletal point is the skeleton point of the elbow of the right arm of the occupant, the area calculation unit 17 calculates the area of ​​a triangle having the first skeletal point, the second skeletal point, and the third skeletal point as vertices, using the position coordinates of the first skeletal point, the second skeletal point, and the third skeletal point, respectively, only when the ratio of the distance between the first skeletal point and the second skeletal point on the photographed image to the distance between the first skeletal point and the third skeletal point on the photographed image is equal to or greater than a second threshold value. Therefore, the occupant physical size detection device 2 shown in Figure 17, like the occupant physical size detection device 2 shown in Figure 1, can increase the number of occupant states in which the occupant's physical size can be estimated compared to conventional methods, and can avoid physical size estimation processing in states where the estimation accuracy is low.

[0079] Embodiment 5. In the fifth embodiment, an occupant physical build detection device 2 including an area correction unit 18 that corrects the area of ​​the polygon calculated by the area calculation unit 13 according to the distance from the camera 1 to the occupant will be described.

[0080] Fig. 20 is a configuration diagram showing an occupant physical build detection device 2 according to embodiment 5. In Fig. 20, the same reference numerals as in Fig. 1 denote the same or corresponding parts, and therefore description thereof will be omitted. Fig. 21 is a hardware configuration diagram showing the hardware of an occupant physical build detection device 2 according to embodiment 5. In Fig. 21, the same reference numerals as in Fig. 2 denote the same or corresponding parts, and therefore description thereof will be omitted.

[0081] The area correction unit 18 is realized by, for example, an area correction circuit 28 shown in FIG. The area correction unit 18 corrects the area S of the polygon calculated by the area calculation unit 13 in accordance with the distance from the camera 1 to the occupant. The area correcting unit 18 outputs the corrected area S′ to the physique estimating unit 14.

[0082] In the occupant physique detection device 2 shown in Fig. 20, the area correction unit 18 is applied to the occupant physique detection device 2 shown in Fig. 1. However, this is merely an example, and the area correction unit 18 may be applied to the occupant physique detection device 2 shown in Fig. 8, the occupant physique detection device 2 shown in Fig. 13, or the occupant physique detection device 2 shown in Fig. 17.

[0083] 20, it is assumed that each of the components of the occupant physique detection device 2, that is, the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 13, the physique estimation unit 14, and the area correction unit 18, is realized by dedicated hardware as shown in Fig. 21. That is, it is assumed that the occupant physique detection device 2 is realized by the photographed image acquisition circuit 21, the skeleton point detection circuit 22, the area calculation circuit 23, the physique estimation circuit 24, and the area correction circuit 28. Each of the photographed image acquisition circuit 21, the skeleton point detection circuit 22, the area calculation circuit 23, the body size estimation circuit 24 and the area correction circuit 28 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of these.

[0084] The components of the occupant physical size detection device 2 are not limited to those realized by dedicated hardware, and the occupant physical size detection device 2 may be realized by software, firmware, or a combination of software and firmware. When the occupant physique detection device 2 is realized by software, firmware, or the like, programs for causing a computer to execute the respective processing procedures in the photographed image acquisition unit 11, the skeleton point detection unit 12, the area calculation unit 13, the physique estimation unit 14, and the area correction unit 18 are stored in a memory 31 shown in Fig. 3. Then, a processor 32 shown in Fig. 3 executes the programs stored in the memory 31.

[0085] 21 shows an example in which each of the components of the occupant physical size detection device 2 is realized by dedicated hardware, and Fig. 3 shows an example in which the occupant physical size detection device 2 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the occupant physical size detection device 2 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0086] Next, the operation of the occupant physique detection device 2 shown in Fig. 20 will be described. Since the occupant physique detection device 2 shown in Fig. 1 is the same as the occupant physique detection device 2 shown in Fig. 1 except for the area correction unit 18, only the operation of the area correction unit 18 will be mainly described here. For ease of explanation, in the occupant physical build detection device 2 shown in FIG. 20, it is assumed that the area S of the polygon calculated by the area calculation unit 13 is the area of ​​a triangle having three skeleton points as vertices. The area correction unit 18 corrects the area S of the triangle calculated by the area calculation unit 13 in accordance with the distance from the camera 1 to the occupant. The area correcting unit 18 outputs the corrected area S′ to the physique estimating unit 14.

[0087] The correction process of the area S by the area correction unit 18 will be specifically described below. For example, when the camera 1 is installed near the center of the dashboard in the vehicle width direction, if the occupant is sitting closer to the windshield of the vehicle than the appropriate position for calculating the area S of the triangle, the center line of the occupant will move outward in the captured image as shown in Fig. 22. In Fig. 22, of the two occupants, the occupant on the right side moves to the right in the figure with the center line of the occupant moving outward. The center line of the occupant is a line that indicates the center part of the occupant in the vehicle width direction. At this time, the area in which the occupant exists will expand in the captured image. On the other hand, if the occupant is sitting closer to the rear window of the vehicle than the appropriate position for calculating the area S of the triangle, the center line of the occupant will move inward in the captured image. For a right-side occupant, the inward direction is the left direction in the figure. In this case, the area in which the occupant is present will shrink in the captured image. FIG. 22 is an explanatory diagram showing a state in which an occupant is sitting closer to the rear window of the vehicle than the appropriate position for calculating the area S of the triangle.

[0088] The area correction unit 18 calculates the distance in the vehicle width direction between the reference center line and the center line of the captured image. x The reference center line is the center line of the occupant when the occupant is sitting in a position that is appropriate for calculating the area S of the triangle. x is the distance on the captured image. The area correction unit 18 acquires image data from the photographed image acquisition unit 11 . The area correction unit 18 identifies the center line of the occupant captured in the captured image represented by the image data. The process of identifying the center line itself is a known technique, and therefore a detailed description thereof will be omitted. The area correction unit 18 detects the distance in the vehicle width direction between the center line of the specified occupant and the center point of the captured image. xCalculate the distance in the vehicle width direction. x is the distance on the captured image.

[0089] The area correction unit 18 detects the distance in the vehicle width direction. x is the distance base x If the distance is shorter than the distance detection x and distance base x A correction is made to enlarge the area S of the triangle so that it is roughly proportional to the difference between The area correction unit 18 detects the distance in the vehicle width direction. x is the distance base x If the distance is longer than , the position where the occupant is sitting is closer to the windshield of the vehicle than the appropriate position for calculating the area S of the triangle. x and distance base x A correction is made to reduce the area S of the triangle so that it is approximately inversely proportional to the difference between The area correction unit 18 detects the distance in the vehicle width direction. x is the distance base x If it is the same, the process of correcting the area S is not performed. If the area S has been corrected, the area correction unit 18 outputs the corrected area S' to the physique estimation unit 14, and if the area S has not been corrected, the area correction unit 18 outputs the area S calculated by the area calculation unit 13 as the corrected area S' to the physique estimation unit 14 as is.

[0090] The physique estimation unit 14 obtains the corrected area S′ from the area correction unit 18. The physique estimation unit 14 estimates the physique P of the occupant from the corrected area S'. The physique estimation unit 14 outputs the estimation result of the physique P to, for example, a control device of the vehicle (not shown).

[0091] In the above-described fifth embodiment, the occupant physique detection device 2 shown in Fig. 20 is configured to include an area correction unit 18 that corrects the area of ​​the polygon calculated by the area calculation unit 13 in accordance with the distance from the camera 1 to the occupant, and the physique estimation unit 14 estimates the occupant's physique from the area corrected by the area correction unit 18. Therefore, like the occupant physique detection device 2 shown in Fig. 1, the occupant physique detection device 2 shown in Fig. 20 can increase the number of occupant states for which the occupant's physique can be estimated compared to the conventional cases, and can also improve the accuracy of physique estimation compared to the occupant physique detection device 2 shown in Fig. 1.

[0092] In the passenger physique detection device 2 shown in FIG. 20, the area correction unit 18 detects the distance x However, this is merely an example, and for example, the area correction unit 18 may include a radar and calculate the distance from the camera 1 to the occupant using the radar. In this case, if the calculated distance is longer than the reference distance, the area correction unit 18 performs a correction to enlarge the area S of the triangle so that it is approximately proportional to the difference between the calculated distance and the reference distance. If the calculated distance is shorter than the reference distance, the area correction unit 18 performs a correction to reduce the area S of the triangle so that the area S is approximately inversely proportional to the difference between the calculated distance and the reference distance.

[0093] In addition, the present disclosure allows free combination of the respective embodiments, modification of any of the components of each embodiment, or omission of any of the components of each embodiment. [Industrial Applicability]

[0094] The present disclosure is suitable for an occupant body size detection device and an occupant body size detection method. [Explanation of symbols]

[0095] 1 camera, 2 occupant body size detection device, 11 photographed image acquisition section, 12 skeleton point detection section, 12a skeleton point search section, 12b skeleton point selection section, 13 area calculation section, 14 body size estimation section, 15 area calculation section, 16 area calculation section, 17 area calculation section, 18 area correction section, 21 photographed image acquisition circuit, 22 skeleton point detection circuit, 23 area calculation circuit, 24 body size estimation circuit, 25 area calculation circuit, 26 area calculation circuit, 27 area calculation circuit, 28 area correction circuit, 31 memory, 32 processor.

Claims

1. an image acquisition unit that acquires an image of a vehicle occupant from a camera that captures the vehicle occupant; a skeleton point detection unit that detects, from the photographed image acquired by the photographed image acquisition unit, three or more skeleton points that are not separated from the camera by an obstacle and that can be used to estimate the physique of the occupant, among five or more predetermined skeleton points including both shoulder skeleton points and both waist skeleton points of the occupant, and outputs position coordinates of each of the three or more skeleton points on the photographed image; an area calculation unit that calculates an area of ​​a polygon having each of the skeleton points as vertices, using the position coordinates of each of the skeleton points output from the skeleton point detection unit; a physical build estimation unit that estimates the physical build of the occupant from the area of ​​the polygon calculated by the area calculation unit; Equipped with The skeleton point detection unit includes a skeleton point searching unit that searches for three or more skeleton points that are not bound to the camera and can be used to estimate the physique of the occupant, among five or more predetermined skeleton points including skeleton points of both shoulders and both waists of the occupant, from the photographed image acquired by the photographed image acquisition unit, and a skeleton point selection unit that selects three skeleton points from the three or more skeleton points searched for by the skeleton point searching unit, and outputs position coordinates of each of the selected three skeleton points on the photographed image to the area calculation unit. An occupant physique detection device comprising:

2. The area calculation unit is calculating an area of ​​a triangle having each skeleton point as a vertex by using the position coordinates of each skeleton point output from the skeleton point selection unit; The physique estimation unit is 2. The occupant physical build detection device according to claim 1, wherein the occupant's physical build is estimated from the area of ​​the triangle calculated by the area calculation unit.

3. The skeleton point selection unit selecting, as a first skeleton point, one of a skeleton point of a left shoulder and a skeleton point of a right shoulder of the occupant, as a second skeleton point, one of a skeleton point of an elbow of a left arm and a skeleton point of an elbow of a right arm of the occupant, and selecting, as a third skeleton point, a midpoint between a left clavicle and a right clavicle of the occupant, a midpoint between the skeleton points of the left shoulder and the right shoulder, or one of the skeleton points of the left shoulder and the right shoulder, which has not been selected as the first skeleton point, 2. The occupant physical size detection device according to claim 1, wherein position coordinates of the first skeleton point, the second skeleton point and the third skeleton point on the photographed image are output to the area calculation unit.

4. when the first skeleton point selected by the skeleton point selection unit is a skeleton point of a left shoulder of the occupant and the second skeleton point is a skeleton point of an elbow of a left arm of the occupant, or when the first skeleton point selected by the skeleton point selection unit is a skeleton point of a right shoulder of the occupant and the second skeleton point is a skeleton point of an elbow of a right arm of the occupant, The area calculation unit is 4. The occupant physical size detection device according to claim 3, characterized in that only when an angle formed by a first straight line connecting the first skeletal point and the second skeletal point and a second straight line connecting the first skeletal point and the third skeletal point is within an allowable angle range, an area of ​​a triangle having vertices at the first skeletal point, the second skeletal point, and the third skeletal point is calculated using the position coordinates of the first skeletal point, the second skeletal point, and the third skeletal point, respectively.

5. when the first skeleton point selected by the skeleton point selection unit is a skeleton point of a left shoulder of the occupant and the second skeleton point is a skeleton point of an elbow of a left arm of the occupant, or when the first skeleton point selected by the skeleton point selection unit is a skeleton point of a right shoulder of the occupant and the second skeleton point is a skeleton point of an elbow of a right arm of the occupant, The area calculation unit is a first straight line connecting the first skeleton point and the second skeleton point; 4. The occupant physical size detection device according to claim 3, characterized in that only when an angle formed by a wrist of an arm on which the skeleton point selected as the second skeleton point is located and a third line connecting the second skeleton point is equal to or greater than a first threshold value, an area of ​​a triangle having vertices at the first skeleton point, the second skeleton point, and the third skeleton point is calculated using position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point, respectively.

6. when the first skeleton point selected by the skeleton point selection unit is a skeleton point of a left shoulder of the occupant and the second skeleton point is a skeleton point of an elbow of a left arm of the occupant, or when the first skeleton point selected by the skeleton point selection unit is a skeleton point of a right shoulder of the occupant and the second skeleton point is a skeleton point of an elbow of a right arm of the occupant, The area calculation unit is 4. The occupant body size detection device according to claim 3, characterized in that only when a ratio of a distance between the first skeleton point and the second skeleton point on the captured image to a distance between the first skeleton point and the third skeleton point on the captured image is equal to or greater than a second threshold value, an area of ​​a triangle having vertices at the first skeleton point, the second skeleton point, and the third skeleton point is calculated using the position coordinates of the first skeleton point, the second skeleton point, and the third skeleton point, respectively.

7. an area correction unit that corrects the area of ​​the polygon calculated by the area calculation unit in accordance with a distance from the camera to the occupant; The physique estimation unit is 2. The occupant physical size detection device according to claim 1, wherein the physical size of the occupant is estimated from the area corrected by the area correction unit.

8. A method for detecting an occupant's physique, the method being performed by an apparatus including an image acquisition unit, a skeleton point detection unit, an area calculation unit, and a physique estimation unit, The photographed image acquisition unit acquires a photographed image in which the occupant is captured from a camera that photographs the occupant of the vehicle, the skeleton point detection unit searches for three or more skeleton points that are free from obstacles between the camera and can be used to estimate the physique of the occupant, among a predetermined five or more skeleton points including skeleton points of both shoulders and both waists of the occupant, from the photographed image acquired by the photographed image acquisition unit, selects three skeleton points from the three or more skeleton points searched, and outputs position coordinates of each of the three skeleton points on the photographed image; the area calculation unit calculates an area of ​​a polygon having each of the skeleton points as vertices, using the position coordinates of each of the skeleton points output from the skeleton point detection unit; The physique estimation unit estimates the physique of the occupant from the area of ​​the polygon calculated by the area calculation unit. A method for detecting occupant physique.

Images