Position detection device, body size detection system, and position detection method

The position detection device enhances three-dimensional positioning accuracy by using a camera and distance sensor to calculate feature point angles and distances, addressing inaccuracies in conventional methods due to individual differences and obstructions.

JP7592159B2Active Publication Date: 2024-11-29MITSUBISHI ELECTRIC CORP
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
JP2023520572
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-05-10
Publication Date
2024-11-29
Estimated Expiration
2041-05-10

AI Technical Summary

Technical Problem

Conventional methods for estimating the three-dimensional position of a person using distance between reference feature points are prone to inaccuracies due to individual differences and environmental obstructions, leading to incorrect positioning.

Method used

A position detection device that utilizes a camera and a distance measurement sensor to detect feature points, calculate the angle of the line connecting the camera and the feature point, and determine the three-dimensional position by combining this angle with the detected distance, thereby enhancing accuracy.

Benefits of technology

The device can accurately specify the three-dimensional position of a person by minimizing processing requirements and accounting for individual variations and obstructions, improving positioning precision compared to conventional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007592159000002
    Figure 0007592159000002
  • Figure 0007592159000003
    Figure 0007592159000003
  • Figure 0007592159000004
    Figure 0007592159000004
Patent Text Reader

Abstract

This present invention comprises: a feature point detection unit (11) that detects, on the basis of a captured image captured by a camera (2), a target feature point in the captured image corresponding to a part of the human body; an angle calculation unit (12) that calculates, on the basis of a position of the target feature point on the captured image detected by the feature point detection unit (11), a feature point position angle representing the angle of a straight line connecting the camera (2) to the target feature point with respect to the imaging axis of the camera (2); a distance detection unit (13) that detects a first distance from a distance measurement sensor (3) to the target feature point on the basis of sensor information acquired by the distance measurement sensor (3); and a position identification unit (14, 14a, 14b) that identifies the three-dimensional position of the target feature point on the basis of the feature point position angle calculated by the angle calculation unit (12) and the first distance detected by the distance detection unit (13).
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a position detection device, a body size detection system, and a position detection method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for detecting feature points in an image in which an area including a person is captured, and for identifying the three-dimensional position of the person based on the detected feature points. For example, Patent Document 1 discloses a method for extracting feature points in an area corresponding to the face of a vehicle occupant in an image captured of the area, and estimating the three-dimensional position of the head of the occupant based on the feature point distance based on the extracted feature points and the reference feature point distance. In this method, the value of the reference feature point distance is a value set based on a statistical value of actual measurements of multiple people, a value calculated by guiding the head position of the occupant to a predetermined position when the occupant gets on the vehicle, or a value calculated at a timing when there is a high probability that the head position of the occupant is in the predetermined position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 163124 Summary of the Invention [Problem to be solved by the invention]

[0004] In a conventional method for estimating a three-dimensional position of a person as disclosed in Patent Document 1, a distance between reference feature points is essential for estimating the three-dimensional position, and there was a problem in that if the distance between the reference feature points is not appropriately set, the three-dimensional position of the person may be erroneously estimated.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a position detection device that can identify the three-dimensional position of a person with higher accuracy than conventional techniques that estimate the three-dimensional position of a person using the distance between reference feature points as a prerequisite. [Means for solving the problem]

[0006] The position detection device disclosed herein includes a feature point detection unit that detects target feature points corresponding to parts of a human body in an captured image based on an image captured by a camera; an angle calculation unit that calculates a feature point position angle indicating an angle of a line connecting the camera and the target feature point relative to the imaging axis of the camera based on a position on the captured image of the target feature point detected by the feature point detection unit; a distance detection unit that detects a first distance from the ranging sensor to the target feature point based on sensor information acquired by the ranging sensor; and a position identification unit that identifies a three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit and the first distance detected by the distance detection unit. Effect of the Invention

[0007] According to the present disclosure, the position detection device is configured as described above, and therefore can identify the three-dimensional position of a person with higher accuracy than conventional techniques that estimate the three-dimensional position of a person by using the distance between reference feature points as a necessary prerequisite. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a configuration of a position detection device according to a first embodiment. [Diagram 2] 2A to 2C are diagrams showing an example of the installation position and orientation of a camera mounted on a vehicle, and an example of the installation position and orientation of a distance measuring sensor mounted on a vehicle in the first embodiment. [Diagram 3] 13A to 13C are diagrams showing another example of the installation position and orientation of the camera mounted on the vehicle, and another example of the installation position and orientation of the distance measuring sensor mounted on the vehicle in the first embodiment. [Figure 4]FIG. 4A is a diagram illustrating an example of the imaging angle of view and the feature point position angle of the camera relative to the horizontal direction, FIG. 4B is a diagram illustrating an example of the imaging angle of view and the feature point position angle of the camera relative to the vertical direction, and FIG. 4C is a diagram illustrating an image captured by the camera. [Diagram 5] 5A is a diagram for explaining an example of a method for identifying the three-dimensional position of a target feature point by a position identification unit in embodiment 1, where FIG. 5A is a diagram showing the camera's imaging situation and the distance measurement sensor's moving object detection situation when the inside of the vehicle cabin is viewed from above, and FIG. 5B is a diagram showing the camera's imaging situation and the distance measurement sensor's moving object detection situation when the inside of the vehicle cabin is viewed from the left side. [Figure 6] 4 is a flowchart for explaining the operation of the position detection device according to the first embodiment. [Figure 7] 7A and 7B are diagrams illustrating an example of a hardware configuration of the position detection device according to the first embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of the configuration of a position detection device according to a second embodiment. [Figure 9] 9A is a diagram for explaining an example of a method by which a position determination unit determines a post-determined distance in embodiment 2, where FIG. 9A is a diagram showing the relationship between the detection result of a radio wave sensor and distance resolution, and FIG. 9B, FIG. 9C, and FIG. 9D are diagrams showing the post-determined distance determined by the position determination unit using the second distance. [Figure 10] 10 is a flowchart for explaining the operation of the position detection device according to the second embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the configuration of a position detection device according to a third embodiment. [Figure 12] 13 is a flowchart for explaining the operation of the position detection device according to the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of the configuration of a position detection device according to a fourth embodiment. [Figure 14]14A and 14B are diagrams for explaining an example of a method in which the distance estimation unit sets a reference inter-feature point distance and estimates the second distance using the set reference inter-feature point distance in the fourth embodiment. [Figure 15] 13 is a flowchart for explaining the operation of the position detection device according to the fourth embodiment. [Figure 16] 10 is a flowchart for explaining the operation of the physique detection device in the physique detection system according to the first to fourth embodiments, in the case where physique determination is performed when it is determined that an occupant is in a properly seated state. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] Embodiment 1 The position detection device according to the first embodiment identifies the three-dimensional position of a person based on an image captured by a camera and information sensed by a distance measurement sensor. In the first embodiment, the position of a person is represented by the position of a predetermined part of the person's body. The part of the person's body is represented by a feature point associated with the part. That is, the three-dimensional position of a person is represented by the three-dimensional position of a feature point corresponding to the part of the person's body. In the first embodiment, as an example, the position of a person is represented by the position of the person's head. It is predetermined which point on the person's head is to be the feature point corresponding to the position of the person's head. In the first embodiment, feature points associated with body parts of a person that indicate the position of the person are referred to as "target feature points." In the following embodiment 1, as an example, the person whose three-dimensional position is to be identified by the position detection device is an occupant sitting in the driver's seat or the passenger seat of the vehicle. In the following embodiment 1, the occupant sitting in the driver's seat or the passenger seat is also simply referred to as an "occupant."

[0011] FIG. 1 is a diagram showing an example of the configuration of a position detection device 1 according to the first embodiment. The position detection device 1 is mounted on a vehicle 1000 and is connected to a camera 2, a distance measurement sensor 3, and a physique detection device 4. The camera 2 is composed of a visible light camera or an infrared camera mounted on the vehicle 1000. The camera 2 may be shared with a so-called "Driver Monitoring System (DMS)". The camera 2 has an imaging angle of view capable of imaging an area including an occupant seated in at least one of the driver's seat and the passenger seat (hereinafter referred to as the "front seat") of the vehicle 1000. The imaging angle of view is measured in degrees. The camera 2 outputs the captured image (hereinafter referred to as the “captured image”) to the position detection device 1.

[0012] The distance measurement sensor 3 is configured by a distance measurement sensor capable of measuring the distance to a moving object, such as a TOF (Time Of Flight) sensor, a radio wave sensor, or an ultrasonic sensor mounted on the vehicle 1000. The distance measurement sensor 3 acquires information on the distance to a moving object present in the vehicle cabin (hereinafter referred to as "sensor information"). The distance measurement sensor 3 outputs the acquired sensor information to the position detection device 1.

[0013] FIG. 2 is a diagram showing an example of the installation position and orientation of the camera 2 mounted on the vehicle 1000 and an example of the installation position and orientation of the distance measuring sensor 3 mounted on the vehicle 1000 in the first embodiment. 2 is a diagram of the vehicle 1000 as seen from the left side relative to the traveling direction of the vehicle 1000, with the front seats indicated as 501 and the rear seats indicated as 502. As shown in FIG. 2, the camera 2 is installed, for example, on the dashboard in an orientation that allows it to capture an image of the real space in which an occupant sitting in the front seat is assumed to be present, i.e., the real space in which the target feature points are assumed to be present. In the first embodiment, the position of the camera 2 is represented by the point where the lens surface and the imaging axis intersect. In the first embodiment, the imaging axis refers to the optical axis of the imaging unit, i.e., the optical axis of the optical system in the camera 2. In FIG. 2, the position of the camera 2 is shown as 2a for convenience.

[0014] In addition, the distance measuring sensor 3 may be Ba, O The distance measuring sensor 3 is installed in the overhead console in a direction that allows sensing of the real space in which a passenger seated in the front seat is assumed to exist, i.e., the real space in which the target feature points are assumed to exist. By installing the distance measuring sensor 3 in the overhead console, the effect of multipath noise can be suppressed, and the passenger (indicated by 201 in FIG. 2) can be detected with higher accuracy. In the first embodiment, the position of the distance measuring sensor 3 is represented by the center of the distance measuring sensor 3 which is determined in relation to the sensing range of the distance measuring sensor 3 . In FIG. 2, the position of the distance measuring sensor 3 is shown as 3a for the sake of convenience.

[0015] The camera 2 and the distance measuring sensor 3 are installed so that the imaging range of the camera 2 (indicated by F101 in FIG. 2) and the sensing range of the distance measuring sensor 3 (indicated by F102 in FIG. 2) overlap. In FIG. 2, the area where the imaging range of the camera 2 and the sensing range of the distance measuring sensor 3 overlap is indicated by F103. In the first embodiment, the coordinate system in the three-dimensional space of the camera 2 and the coordinate system in the three-dimensional space of the distance measuring sensor 3 are expressed by the same coordinate system. That is, the X-axis, Y-axis, and Z-axis in the coordinate system of the camera 2 are common to the X-axis, Y-axis, and Z-axis in the coordinate system of the distance measuring sensor 3. In the first embodiment, in the coordinate system in the three-dimensional space common to the camera 2 and the distance measuring sensor 3, the coordinates of the position of the camera 2 are (0,0,0).

[0016] In the following first embodiment, as an example, it is assumed that the camera 2 and the distance measuring sensor 3 are installed at positions as shown in FIG. 3, like Fig. 2, shows the vehicle 1000 as seen from the left side relative to the traveling direction of the vehicle 1000, with the front seats indicated as 501 and the rear seats indicated as 502. Also, in Fig. 3, the passengers are indicated as 201. Also, for convenience, in Fig. 3, the position of the camera 2 is indicated as 2a, and the position of the distance measuring sensor 3 is indicated as 3a. 3, the distance measuring sensor 3 is installed so that, among the parts of the occupant's body, the head of the occupant is located closest to the distance measuring sensor 3. The camera 2 is installed on the dashboard. The camera 2 and the distance measuring sensor 3 have the same installation axes in the left-right, up-down, and front-back directions, and the same installation coordinates in the left-right and front-back directions. The camera 2 and the distance measuring sensor 3 have different installation coordinates in the up-down direction. Specifically, the distance measuring sensor 3 is installed α [m] above the camera 2.

[0017] The camera 2 and the distance measuring sensor 3 are installed so that their orientations and positions match each other, thereby reducing the amount of calculation required for processing when identifying the three-dimensional position of the target feature point in the position detection device 1. In other words, when the camera 2 and the distance measuring sensor 3 are installed so that their orientations and positions match each other, it can be said that the camera 2 and the distance measuring sensor 3 are installed in optimal positions that can minimize the amount of calculation required for processing when identifying the three-dimensional position of the target feature point in the position detection device 1. Details of the position detection device 1 will be described later. In addition, in embodiment 1, when the camera 2 and the distance measurement sensor 3 are installed so that their positions match each other, it means that the camera 2 and the distance measurement sensor 3 are installed so that their horizontal position (X-axis direction), vertical position (Y-axis direction), depth position (Z-axis direction), and installation tilt all match each other in three-dimensional space. Here, in the above-mentioned optimal installation position of the camera 2 and the distance measuring sensor 3 that can minimize the amount of calculations required for processing when identifying the three-dimensional position of the target feature point in the position detection device 1, the position of the camera 2 and the distance measuring sensor 3 do not need to match perfectly, but only need to approximately match within a predetermined tolerance range. Also, in the above-mentioned optimal installation position of the camera 2 and the distance measuring sensor 3 that can minimize the amount of calculations required for processing when identifying the three-dimensional position of the target feature point in the position detection device 1, the orientation of the camera 2 and the distance measuring sensor 3 do not need to match perfectly, but only need to approximately match within a predetermined tolerance range. 2 and 3, the position of the camera 2 and the position of the distance measuring sensor 3 do not coincide. In embodiment 1, the camera 2 and the distance measurement sensor 3 need only be installed with at least one of the following positions in the horizontal direction (X-axis direction), the vertical direction (Y-axis direction), the depth direction (Z-axis direction), and the installation inclination in three-dimensional space being the same.

[0018] The position detection device 1 identifies the three-dimensional position of the target feature point based on the captured image captured by the camera 2 and the sensor information acquired by the distance measurement sensor 3. That is, the position detection device 1 identifies the three-dimensional position of the target feature point, thereby identifying the three-dimensional position of the occupant. When the position detection device 1 identifies the three-dimensional position of the target feature point, it outputs information on the three-dimensional position of the target feature point (hereinafter referred to as “position information”) to the physique detection device 4.

[0019] The physique detection device 4 is mounted on the vehicle 1000 and determines the physique of the occupant based on the position information output from the position detection device 1. The physique detection device 4 includes a position information acquisition unit 41 and a physique determination unit . The position information acquisition unit 41 acquires the position information from the position detection device 1 . The physique determining unit 42 determines the physique of the occupant based on the position information acquired by the position information acquiring unit 41. The physique determination unit 42 only needs to determine the physique of an occupant using well-known technology for determining the physique of a person from information indicating the person's position, so a detailed explanation of the method by which the physique determination unit 42 determines the physique of an occupant will be omitted. The position detection device 1 and the physique detection device 4 constitute a physique detection system 100.

[0020] Information relating to the physique of the occupant determined by the physique detection device 4 is output to a collision safety device 5 mounted on the vehicle 1000. The collision safety device 5 controls an airbag (not shown) or a seat belt pretensioner (not shown) mounted on the vehicle 1000 in accordance with the physique of the occupant determined by the physique detection device 4. An airbag or a seat belt pretensioner, etc., needs to be controlled in consideration of the physique of the occupant. For example, an airbag needs to be controlled based on the distance between the occupant and the airbag corresponding to the occupant's physique when the airbag is activated so that the occupant is not injured by the explosive force of the airbag. Also, for example, a seat belt pretensioner needs to be controlled in consideration of the position of the occupant's neck corresponding to the occupant's physique when the seat belt pretensioner is activated so that the seat belt does not strangle the occupant. Therefore, the physique detection device 4 is required to appropriately determine the occupant's physique so that the collision safety device 5 appropriately operates the control function of the airbag or the seat belt pretensioner, etc. That is, the position detection device 1 is required to accurately identify the position of the occupant, which is the basis for determining the occupant's physique. The position detection device 1 according to the first embodiment enables the position of the occupant to be identified with high accuracy. The collision safety device 5 may acquire the position information of the occupant outputted by the position detection device 1 and control an airbag, a seat belt pretensioner, or the like.

[0021] A position detection device 1 according to the first embodiment will be described. As shown in FIG. 1, the position detection device 1 includes a feature point detection unit 11, an angle calculation unit 12, a distance detection unit 13, and a position identification unit .

[0022] The feature point detection unit 11 acquires a captured image from the camera 2, and detects target feature points in the captured image based on the captured image captured by the camera 2. Here, the feature point detection unit 11 detects a target feature point in the captured image that corresponds to the head of an occupant. The feature point detection unit 11 can detect a plurality of feature points. For example, the feature point detection unit 11 may detect feature points indicating facial features such as the eyes and nose together, and may detect feature points indicating body parts such as the neck and shoulders together. The feature point detection unit 11 may detect feature points in a captured image using a well-known technique for detecting human feature points from a captured image, and therefore a detailed description of the method for detecting feature points by the feature point detection unit 11 will be omitted. Feature points in a captured image are represented by coordinates on the captured image. The feature point detection unit 11 outputs information on the detected target feature points (hereinafter referred to as “target feature point information”) to the angle calculation unit 12.

[0023] The angle calculation unit 12 calculates a “feature point position angle” that indicates the angle of the line connecting the camera 2 and the target feature point with respect to the imaging axis of the camera 2, based on the position of the target feature point on the captured image detected by the feature point detection unit 11. More specifically, the angle calculation unit 12 calculates a feature point position angle that indicates the angle of the direction from the camera 2 toward the three-dimensional position of the target feature point with respect to the direction of the imaging axis of the camera 2.

[0024] Here, the target feature points detected by the feature point detection unit 11 and the feature point position angles calculated by the angle calculation unit 12 will be described with reference to the drawings. FIG. 4 is a diagram for explaining the feature point position angles θcx, θcy calculated by the angle calculation section 12 and an example of a method for calculating the feature point position angles θcx, θcy by the angle calculation section 12 in the first embodiment. In FIG. 4, an occupant is indicated by 201 and the position of the occupant's head, that is, the position of the feature point corresponding to the occupant's head, is indicated by 202. Figure 4A is a diagram for explaining an example of the imaging angle of view θx and feature point position angle θcx of camera 2 relative to the horizontal direction, Figure 4B is a diagram for explaining an example of the imaging angle of view θy and feature point position angle θcy of camera 2 relative to the vertical direction, and Figure 4C is a diagram showing an image captured by camera 2.

[0025] As described above, the camera 2 has imaging angles of view θx, θy. Here, as shown in FIG. 4A, θx is the imaging angle of view with respect to the horizontal direction (the direction along an imaginary plane defined by the x-axis and z-axis in FIG. 4). As shown in the left diagram of Fig. 4A, θcx is the feature point position angle with respect to the horizontal direction. Note that for ease of understanding of θcx, the left diagram of Fig. 4A is shown, but here, for example, as shown in the right diagram of Fig. 4A, the feature point position angle θcx = 0 [degrees]. In other words, it is assumed that the horizontal direction of the straight line connecting the camera 2 and the target feature point coincides with the imaging axis of the camera 2 (indicated by IA in Fig. 4). As shown in FIG. 4B, θy is the imaging angle of view with respect to the vertical direction (the direction along an imaginary plane defined by the y-axis and z-axis in FIG. 4). As shown in FIG. 4B, θcy is the feature point position angle with respect to the vertical direction. The feature point position angles θcx and θcy are expressed in degrees.

[0026] When a target feature point is located at a position where the feature point position angles θcx and θcy are as shown in the right diagram of FIG. 4A and in FIG. 4B, but Photography Image The captured image is shown as the captured image 21 in FIG. 4C. The feature point detection unit 11 detects target feature points on a captured image such as that indicated by 21 in Fig. 4C. Here, the target feature points detected by the feature point detection unit 11 are represented by, for example, coordinates (Xc, Yc).

[0027] The angle calculation unit 12 calculates the feature point position angles θcx, θcy with respect to the imaging axis of the camera 2, based on the position of the target feature point on the captured image represented by the coordinates (Xc, Yc) detected by the feature point detection unit 11. When calculating the feature point position angles θcx, θcy, the angle calculation unit 12 assumes that the coordinates of the position in the captured image corresponding to the imaging axis of the camera 2, i.e., the position coordinates of the center part in the captured image, are set to (0, 0), as shown in Fig. 4C. Specifically, the values ​​of the imaging angles of view θx, θy, the value of the resolution in the first axis direction (x-axis direction in FIG. 4) in the captured image (indicated by m in FIG. 4C), and the value of the resolution in the second axis direction (y-axis direction in FIG. 4) in the captured image (indicated by n in FIG. 4C) are stored in advance in the angle calculation unit 12. The unit of resolution is pixels (px).

[0028] The angle calculation unit 12 calculates feature point position angles θcx, θcy based on the coordinates (Xc, Yc) of the position of the target feature point detected by the feature point detection unit 11, using the following equations (1) and (2). Xc:tanθcx=(m / 2):tan(θx / 2) ···(1) Yc:tanθcy=(n / 2):tan(θy / 2) ···(2) It can be said that the three-dimensional position coordinates of the target feature point lie on the straight line of the following equation (3). TIFF0007592159000001.tif14166 Note that here, the coordinates on the captured image are (Xc, Yc)=(0, Yc), and the feature point position angle θcx=0 (degrees).

[0029] The angle calculation section 12 outputs information on the calculated feature point position angle, more specifically, information on the above formula (3) obtained from the calculated feature point position angle, to the position identification section 14.

[0030] The distance detection unit 13 acquires sensor information from the distance measurement sensor 3, and detects an occupant based on the acquired sensor information. Specifically, the distance detection unit 13 detects the distance from the distance measurement sensor 3 to the target feature point (hereinafter referred to as "first distance") based on the sensor information. Note that the distance detection unit 13 may detect an occupant using a well-known technique for detecting a moving object from the sensor information, and therefore a detailed description of the method by which the distance detection unit 13 detects an occupant will be omitted. Distance detection unit 13 outputs information relating to the detected first distance (hereinafter referred to as “sensor distance information”) to position identification unit 14.

[0031] The position identification unit 14 identifies the three-dimensional position of the target feature point based on information on the feature point position angle calculated by the angle calculation unit 12, in other words, the information of the above formula (3), and the first distance detected by the distance detection unit 13.

[0032] Here, Fig. 5 is a diagram for explaining an example of a method for identifying a three-dimensional position of a target feature point by the position identification unit 14 in the first embodiment. Fig. 5A is a diagram showing the imaging situation of the camera 2 and the moving object detection situation of the distance measurement sensor 3 when the inside of the vehicle is viewed from above, and Fig. 5B is a diagram showing the imaging situation of the camera 2 and the moving object detection situation of the distance measurement sensor 3 when the inside of the vehicle is viewed from the left side. The camera 2 and the distance measurement sensor 3 are installed at the installation positions shown in Fig. 3. 2a and 3a in Fig. 5B indicate the positions of the camera 2 and the distance measurement sensor 3, respectively, and IA in Fig. 5B indicates the imaging axis of the camera 2. Also, in Fig. 5, the front seat 501 and the rear seat 502 are shown in a simplified manner. The positional relationship between the camera 2 and the target feature points is as shown in the diagram on the right side of FIG. 4A and in FIG. 5B.

[0033] Now, the distance measuring sensor 3 is installed so that the position of the occupant's head is the closest of all the parts of the occupant's body as viewed from the distance measuring sensor 3, and therefore the point indicated by 202 in Fig. 5 is the point indicating the occupant's head, in other words, the target feature point. The distance detection unit 13 detects the point indicated by 202 in Fig. 5 as the target feature point. Then, the distance detection unit 13 detects the distance from the distance measuring sensor 3 to the target feature point. Here, it is assumed that the first distance to the target feature point detected by the distance detection unit 13 is, for example, D [m].

[0034] The position specifying unit 14 receives the information of the above formula (3) from the angle calculation unit 12. Further, sensor distance information regarding the first distance is output from distance detection unit 13 to position identification unit 14. Here, information of the first distance=D[m] is output to position identification unit 14.

[0035] Since the first distance detected by the distance detection unit 13 is D [m], the target feature point is on a sphere with a radius of D [m] and its center is the position of the distance measurement sensor 3. In other words, if the center is (X0, Y0, Z0), the coordinates (X, Y, Z) of the three-dimensional position of the target feature point are expressed as (X-X0)^2+(Y-Y0)^2+(Z-Z0)^2=D^2. Here, in embodiment 1, in a coordinate system in three-dimensional space common to camera 2 and distance measurement sensor 3, the coordinates of the position of camera 2 are (0,0,0), and therefore the coordinates of the position of distance measurement sensor 3 are (0,α,0). In this case, the three-dimensional position of the target feature point expressed by the coordinates (X, Y, Z) will be on the sphere X^2+(Y-α)^2+Z^2=D^2.

[0036] The position identification unit 14 can calculate the coordinates (X, Y, Z) of the three-dimensional position of the target feature point by determining the intersection of the straight line expressed by the above formula (3) and the sphere of X^2+(Y-α)^2+Z^2=D^2. The coordinates (X, Y, Z) of the three-dimensional position of the target feature point become the coordinates indicating the three-dimensional position of the occupant. When the distance measuring sensor 3 is a sensor having multiple receiving elements, the distance detection unit 13 can detect not only the distance to an object but also the direction (angle) in which the object exists. Therefore, the distance measuring sensor can detect the distance to an object in the angular direction of the feature point captured by the camera 2. In this case, the position identifying unit 14 may calculate the coordinates (X, Y, Z) of the three-dimensional position of the target feature point from the intersection of two straight lines, the straight line represented by the above formula (3) and a straight line including a line segment indicating the distance to an object in the angular direction of the feature point detected by the distance detecting unit 13. Specifically, the position identifying unit 14 may determine the distance to the intersection of the above two straight lines, which is also detected by the distance detecting unit 13, as the distance to the target feature point, and calculate the coordinates (X, Y, Z) of the three-dimensional position of the target feature point. Therefore, if the distance measurement sensor 3 is a sensor having multiple receiving elements, the coordinates (X, Y, Z) of the three-dimensional position of the target feature point (here, the occupant's head) can be calculated even if the distance measurement sensor 3 is not installed so that it is closest to the target feature point.

[0037] Also, here we have explained a method in which the position identification unit 14 calculates the three-dimensional position of the target feature point when the positional relationship between the target feature point and the camera 2 and the distance measurement sensor 3 is as shown in Figures 3, 4, and 5, but this is merely one example. Even when the positional relationship between the target feature point and the camera 2 and the ranging sensor 3 is a positional relationship other than the above-mentioned example, the position identification unit 14 can similarly calculate the three-dimensional position of the target feature point from the equation of a straight line passing through the position of the camera 2 and the target feature point, and the equation of a sphere centered at the position of the ranging sensor 3 and with the first distance as its radius.

[0038] The position identifying unit 14 outputs to the physique detection device 4 position information relating to the identified three-dimensional position of the occupant.

[0039] The operation of the position detection device 1 according to the first embodiment will be described. FIG. 6 is a flowchart for explaining the operation of the position detection device 1 according to the first embodiment.

[0040] The feature point detection unit 11 acquires a captured image from the camera 2, and detects target feature points in the captured image based on the captured image captured by the camera 2 (step ST1). The feature point detection unit 11 outputs the target feature point information to the angle calculation unit 12.

[0041] The angle calculation unit 12 calculates the feature point position angle based on the position on the captured image of the target feature point detected by the feature point detection unit 11 in step ST1 (step ST2). The angle calculation section 12 outputs information relating to the calculated feature point position angle to the position identification section 14.

[0042] The distance detection unit 13 acquires sensor information from the distance measurement sensor 3, and detects an occupant based on the acquired sensor information. Specifically, the distance detection unit 13 detects a first distance based on the sensor information (step ST3). The distance detection unit 13 outputs the sensor distance information to the position identification unit 14.

[0043] The position identifying unit 14 identifies the three-dimensional position of the target feature point based on information about the feature point position angle calculated by the angle calculating unit 12 in step ST2 and the first distance detected by the distance detecting unit 13 in step ST3 (step ST4). The position identifying unit 14 outputs to the physique detection device 4 position information relating to the identified three-dimensional position of the target occupant.

[0044] In the flowchart shown in FIG. 6, the position detection device 1 performs the processes in the order of step ST1, step ST2, and step ST3, but this is merely an example. For example, the position detection device 1 may perform the processes in the order of step ST3, step ST1, and step ST2, or may perform the processes of steps ST1 and ST2 and the process of step ST3 in parallel.

[0045] In this way, by using sensor information from the ranging sensor 3, the position detection device 1 can accurately identify the three-dimensional position of a target feature point that cannot be directly measured from the image captured by the camera 2, in other words, the three-dimensional position of the occupant.

[0046] In the conventional methods for estimating the three-dimensional position of a person as described above, the distance between reference feature points is essential for estimating the three-dimensional position, and if the distance between the reference feature points is not appropriately set, there is a possibility that the three-dimensional position of the person will be erroneously estimated. For example, if the value of the reference inter-feature point distance is set based on a statistical value of actual measurements of multiple people, individual differences cannot be taken into consideration, and thus the three-dimensional position of the person may be erroneously identified. Even if the value of the reference inter-feature point distance is set when an occupant boards the vehicle, taking individual differences into consideration, the three-dimensional position of the person cannot be identified while the value of the reference inter-feature point distance is not set. Furthermore, if the position of the occupant's head is guided to a predetermined position to set the value of the reference inter-feature point distance, this causes inconvenience to the occupant. In contrast to this, as described above, the position detection device 1 according to the first embodiment can specify with high accuracy the three-dimensional positions of target feature points that cannot be directly measured from the captured image captured by the camera 2, in other words, the three-dimensional position of an occupant, by using the sensor information from the distance measurement sensor 3. In other words, the position detection device 1 can specify the three-dimensional position of a person with higher accuracy than the conventional technology that estimates the three-dimensional position of a person by necessarily using the distance between reference feature points.

[0047] For example, if the distance measuring sensor 3 of the position detection device 1 is a radio wave sensor and the three-dimensional position of the target feature point is detected using sensor information acquired from the radio wave sensor, the position detection device 1 can grasp the three-dimensional position of the occupant with higher accuracy than when the three-dimensional position of the target feature point is detected using sensor information acquired from a distance measuring sensor 3 of a type other than a radio wave sensor. This is because the radio wave sensor can detect the occupant even if there is an obstruction in front of the occupant, such as a thick jacket, hat, or baggage, through the obstruction and can detect the front and rear positions of the occupant with high accuracy.

[0048] 7A and 7B are diagrams illustrating an example of a hardware configuration of the position detection device 1 according to the first embodiment. In the first embodiment, the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 are realized by a processing circuit 91. That is, the position detection device 1 includes the processing circuit 91 for controlling detection of a three-dimensional position of a person from an image captured by the camera 2 and sensor information acquired by the distance measurement sensor 3. The processing circuitry 91 may be dedicated hardware as shown in FIG. 7A, or may be a processor 94 executing a program stored in a memory as shown in FIG. 7B.

[0049] When the processing circuitry 91 is dedicated hardware, the processing circuitry 91 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0050] When the processing circuit is the processor 94, the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 95. The processor 94 executes the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 by reading and executing the program stored in the memory 95. That is, the position detection device 1 includes a memory 95 for storing a program that, when executed by the processor 94, results in the execution of steps ST1 to ST4 in FIG. 6 described above. It can also be said that the program stored in the memory 95 causes a computer to execute the procedures or methods of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14. Here, memory 95 refers to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), etc., or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), etc.

[0051] It should be noted that the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the functions of the feature point detection unit 11 and the distance detection unit 13 may be realized by a processing circuit 91 as dedicated hardware, and the functions of the angle calculation unit 12 and the position identification unit 14 may be realized by a processor 94 reading and executing a program stored in a memory 95.

[0052] The position detection device 1 also includes a camera 2, a distance measurement sensor 3, a body size detection device 4, a collision safety device 5, and other devices, as well as an input interface device 92 and an output interface device 93 that perform wired or wireless communication.

[0053] In the above embodiment 1, the position of the occupant is indicated by the position of the head of the occupant, but this is merely an example. The position of the occupant may be indicated by multiple parts of the occupant's body. In other words, there may be multiple target feature points. The position detection device 1 can also detect the three-dimensional positions of multiple target feature points. In this case, in the position detection device 1, the distance detection unit 13 detects the first distances of the multiple target feature points. For example, the vehicle 1000 is provided with a distance measurement sensor 3 equipped with a receiving antenna having multiple receiving antenna elements that receive reflected waves of radio waves emitted from a transmitting antenna and are positioned differently in the width direction of the vehicle 1000, and the distance detection unit 13 detects the first distances of the multiple target feature points based on sensor information acquired from the distance measurement sensor 3. In more detail, for example, the distance detection unit 13 acquires a captured image from the camera 2, identifies the direction in which each target feature point exists based on the captured image, and the distance measurement sensor 3 can analyze the received waves by the multiple antennas to determine the direction of arrival of the reflected waves, so that the distance to a moving object in the direction in which each target feature point exists can be regarded as the distance to each target feature point.

[0054] In the above embodiment 1, as an example, the occupant whose three-dimensional position is to be specified is the occupant sitting in the driver's seat or the passenger seat of the vehicle 1000, but this is merely an example. The occupant whose three-dimensional position is to be specified may be an occupant in the back seat. The position detection device 1 can also specify the three-dimensional position of the occupant in the back seat. That is, the position detection device 1 can also specify the three-dimensional position of a target feature point corresponding to a body part indicating the position of the occupant in the back seat.

[0055] In the above embodiment 1, one camera 2 and one distance measuring sensor 3 are installed in the vehicle cabin, but this is merely an example. A plurality of cameras 2 may be installed in the vehicle cabin, and a plurality of distance measuring sensors 3 may be installed in the vehicle cabin. It is sufficient that the camera 2 and the distance measuring sensor 3 are installed so that the imaging range of the camera 2 and the sensing range of the distance measuring sensor 3 overlap.

[0056] In the above-described first embodiment, the position detection device 1 is an on-board device mounted on the vehicle 1000, and the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 are provided in the on-board device. However, without being limited to this, some of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 may be provided in the on-board device of the vehicle and the rest may be provided in a server connected to the on-board device via a network, and the on-board device and the server may form a position detection system. Moreover, the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, and the position identification unit 14 may all be provided in the server.

[0057] In the above-described first embodiment, the physique detection device 4 is an in-vehicle device mounted on the vehicle 1000, and the position information acquisition unit 41 and the physique determination unit 42 are provided in the in-vehicle device, but this is merely an example. Some of the position information acquisition unit 41 and the physique determination unit 42 may be provided in the in-vehicle device and the rest in the server, or the position information acquisition unit 41 and the physique determination unit 42 may be provided in the server. The collision safety device 5 may also be provided in the server.

[0058] In the above embodiment 1, the person whose three-dimensional position is to be detected by the position detection device 1 is an occupant of the vehicle 1000, but this is merely an example. The position detection device 1 can detect the three-dimensional positions of occupants in various vehicles. The position detection device 1 can also detect the three-dimensional positions of people present in real space, such as people present in a room, not limited to occupants of a vehicle.

[0059] As described above, according to the first embodiment, the position detection device 1 includes the feature point detection unit 11 that detects, based on an image captured by the camera 2, a target feature point corresponding to a body part of a person in the captured image, the angle calculation unit 12 that calculates a feature point position angle indicating an angle of a line connecting the camera 2 and the target feature point with respect to the imaging axis of the camera 2 based on the position of the target feature point on the captured image detected by the feature point detection unit 11, the distance detection unit 13 that detects a first distance from the distance measurement sensor 3 to the target feature point based on sensor information acquired by the distance measurement sensor 3, and the position identification unit 14 that identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit 12 and the first distance detected by the distance detection unit 13. Therefore, the position detection device 1 can identify the three-dimensional position of a person with higher accuracy than the conventional technology that estimates the three-dimensional position of a person by essentially using the distance between reference feature points.

[0060] Embodiment 2 In the second embodiment, an embodiment will be described in which a radio wave sensor is used as the distance measurement sensor, and the three-dimensional position of a person is identified with higher accuracy by taking into consideration the distance to the person estimated based on a captured image.

[0061] In the following second embodiment, as in the first embodiment, the person whose three-dimensional position is to be specified is assumed to be an occupant sitting in the driver's seat or the passenger seat of the vehicle, and the position of the occupant is indicated by the position of the occupant's head. Also, the camera and the distance measuring sensor are assumed to be installed at the positions as described in the first embodiment with reference to FIG. 3. However, in the second embodiment, it is assumed that the distance measuring sensor is a radio wave sensor. The radio wave sensor can detect an occupant through obstacles and can detect the front and rear positions of the occupant with high accuracy. Radio wave sensors detect at intervals of distance resolution. For example, if the distance resolution is 2 cm, the same value will be detected even if the occupant's position is 10 cm, 10.5 cm, or 11 cm from the radio wave sensor. This distance resolution is inversely proportional to the bandwidth used. For example, in the case of a 60 GHz radio wave sensor, the bandwidth used is 7 GHz, and the distance resolution is approximately 2.14 cm.

[0062] FIG. 8 is a diagram showing an example of the configuration of a position detection device 1a according to the second embodiment. In the configuration example of the position detection device 1a shown in FIG. 8, the same components as those of the position detection device 1 according to embodiment 1 described in embodiment 1 using FIG. 1 are given the same reference numerals and redundant explanations are omitted. The position detection device 1a according to the second embodiment differs from the position detection device 1 according to the first embodiment described with reference to FIG. Furthermore, in the position detection device 1a according to the second embodiment, the specific operation of the position identifying unit 14a is different from the specific operation of the position identifying unit 14 in the position detection device 1 according to the first embodiment.

[0063] Based on the plurality of feature points detected by feature point detection section 11, distance estimation section 15 estimates the distance from camera 2 to the target feature point (hereinafter referred to as “second distance”). In the second embodiment, the feature point detection unit 11 detects a plurality of feature points and outputs information on the detected plurality of feature points to the distance estimation unit 15. The distance estimation unit 15 may estimate a second distance to a target feature point based on multiple feature points detected from a captured image, for example, using a well-known technique such as that disclosed in International Publication No. 2019 / 163124. For example, the distance estimation unit 15 calculates the distance between any two feature points (for example, a feature point corresponding to the right eye of an occupant and a feature point corresponding to the left eye of the occupant; hereinafter referred to as "feature points for distance calculation") among a plurality of feature points. Note that the feature point distance is a distance in a captured image, and the unit of the feature point distance is pixels (px). The distance estimation unit 15 can also calculate the feature point distance taking into account the facial direction of the occupant (for example, see International Publication No. WO 2019 / 163124).

[0064] The distance estimation unit 15 estimates the three-dimensional position of the target feature point of the occupant in real space and estimates the second distance, for example, by using the feature point position angle calculated by the angle calculation unit 12, the inter-feature point distance, and a reference inter-feature point distance (hereinafter referred to as "first inter-feature point distance") stored in advance in the distance estimation unit 15. Note that the value of the first inter-feature point distance is set based on, for example, a statistical value of actual measurements for a plurality of people. Specifically, for example, the width between the parts corresponding to the distance calculation feature points (for example, the width between both eyes) is actually measured for each of the plurality of people. The distance estimation unit 15 stores in advance the average value of the measured widths as the value of the first inter-feature point distance. Distance estimation unit 15 outputs information relating to the estimated second distance (hereinafter referred to as "camera distance information") to position identification unit 14a.

[0065] Based on the first distance detected by the distance detection unit 13 and the second distance estimated by the distance estimation unit 15, the position identification unit 14a determines the distance to the target feature point (hereinafter referred to as the “determined distance”). In the second embodiment, the determined distance is a more likely distance to a target feature point determined from the first distance and the second distance, taking into account the distance resolution of the radio wave sensor.

[0066] Here, Fig. 9 is a diagram for explaining an example of a method in which the position specifying unit 14a determines the determined distance in embodiment 2. Fig. 9A is a diagram showing the relationship between the detection result of the radio wave sensor and the distance resolution, and Fig. 9B, Fig. 9C, and Fig. 9D are diagrams showing the determined distance determined by the position specifying unit 14a using the second distance. In FIG. 9, the distance resolution of the radio wave sensor is "2ω". In FIG. 9, the value of the first distance detected by the distance detection unit 13 based on the sensor information, which is the detection result of the radio wave sensor, is Dr [m]. In FIG. 9, the first distance is indicated as "detection result of the radio wave sensor". In FIG. 9, the value of the second distance estimated by the distance estimation unit 15 based on the image captured by the camera 2 is indicated as Dc [m]. In FIG. 9, the second distance is indicated as "estimated result of the camera".

[0067] As shown in FIG. 9A, if the first distance detected by the radio wave sensor is the intermediate value of the distance resolution (=2ω), the actual distance value to the target feature point is any value between "the first distance minus half the distance resolution" and "the first distance plus half the distance resolution". In other words, if the first distance detected by the radio wave sensor is the intermediate value of the distance resolution (=2ω), the actual distance value to the target feature point is any value between Dr-ω and Dr+ω [m]. Based on the first distance and the second distance, the position identification unit 14a determines the determined distance between Dr-ω and Dr+ω [m] taking into account the distance resolution of the radio wave sensor.

[0068] 9B, when the value of the second distance is smaller than the value obtained by subtracting half the distance resolution from the first distance, in this case, when Dc[m] is smaller than Dr-ω[m], the position specifying unit 14a determines the value of the determined distance to be "the value obtained by subtracting half the distance resolution from the first distance." That is, the position specifying unit 14a determines the value of the determined distance to be Dr-ω[m].

[0069] 9C, when the value of the second distance is greater than the value obtained by adding half the distance resolution to the first distance, in this case Dc[m] being greater than Dr+ω[m], the position specifying unit 14a determines that the value of the determined distance is "the value obtained by adding half the distance resolution to the first distance." That is, the position specifying unit 14a determines that the value of the determined distance is Dr+ω[m].

[0070] 9D, when the value of the second distance is equal to or greater than the value obtained by subtracting half the distance resolution from the first distance and equal to or less than the value obtained by adding half the distance resolution to the first distance, in this case, when Dc[m] is within the range of Dr-ω to Dr+ω[m], the position specifying unit 14a determines the value of the determined distance as the "value of the second distance." That is, the position specifying unit 14a determines the value of the determined distance to be Dc[m].

[0071] The process performed by the position specifying unit 14a to determine the post-determined distance based on the first distance and the second distance as described with reference to FIG. 9 is referred to as a "distance determination process."

[0072] When the position identifying unit 14a determines the post-determined distance, it identifies the three-dimensional position of the occupant based on the feature point position angle calculated by the angle calculation unit 12 and the post-determined distance. A specific method for identifying the three-dimensional position of the occupant by the position identifying unit 14a has been described in the first embodiment, and therefore a duplicated description will not be repeated. In the first embodiment, the position identifying unit 14a identifies the three-dimensional position of the occupant based on the first distance and the feature point position angle, but in the second embodiment, the position identifying unit 14a identifies the three-dimensional position of the occupant by replacing the first distance with the determined distance.

[0073] The operation of the position detection device 1a according to the second embodiment will be described. FIG. 10 is a flowchart for explaining the operation of the position detection device 1a according to the second embodiment. In the flowchart of FIG. 10, the specific processing contents of steps ST11 to ST12 and step ST14 are similar to the specific processing contents of steps ST1 to ST3 in FIG. 6 that have been described in the first embodiment, and therefore duplicated description will be omitted.

[0074] The distance estimation unit 15 estimates a second distance based on the plurality of feature points detected by the feature point detection unit 11 in step ST11 (step ST13). Distance estimation unit 15 outputs the camera distance information to position identification unit 14a.

[0075] The position identifying unit 14a executes a "distance determination process" based on the first distance detected by the distance detecting unit 13 in step ST14 and the second distance estimated by the distance estimating unit 15 in step ST13, and determines the determined distance. Specifically, first, the position specifying unit 14a determines whether or not the value of the second distance is smaller than a value obtained by subtracting half the distance resolution from the first distance (step ST15). If the value of the second distance is smaller than the value obtained by subtracting half the distance resolution from the first distance (if "YES" in step ST15), the position identification unit 14a determines that the value of the determined distance is "the value obtained by subtracting half the distance resolution from the first distance" (step ST16). Then, the operation of the position detection device 1a proceeds to the process of step ST20.

[0076] If the value of the second distance is greater than or equal to the value obtained by subtracting half the distance resolution from the first distance ("NO" in step ST15), the position identification unit 14a determines whether the value of the second distance is greater than the value obtained by adding half the distance resolution to the first distance (step ST17). If the value of the second distance is greater than the value obtained by adding half the distance resolution to the first distance (if "YES" in step ST17), the position identification unit 14a determines that the value of the determined distance is "the value obtained by adding half the distance resolution to the first distance" (step ST18). Then, the operation of the position detection device 1a proceeds to the process of step ST20.

[0077] If the value of the second distance is equal to or smaller than the value obtained by adding half the distance resolution to the first distance (if "NO" in step ST17), in other words, if the value of the second distance is equal to or larger than the value obtained by subtracting half the distance resolution from the first distance and equal to or smaller than the value obtained by adding half the distance resolution to the first distance, the position identifying unit 14a sets the value of the determined distance to the "value of the second distance" (step ST19). Then, the operation of the position detection device 1a proceeds to the processing of step ST20.

[0078] The above-mentioned processes from step ST15 to step ST19 constitute the "distance determination process" performed by the position specifying unit 14a.

[0079] In step ST20, the position identifying unit 14a identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit 12 in step ST12 and the post-determination distance determined in step ST16, step ST18, or step ST19.

[0080] In the flowchart shown in FIG. 10, the position detection device 1a performs the processes in the order of step ST11, step ST12, step ST13, and step ST14, but this is merely an example. For example, the position detection device 1a may perform the processes in the order of step ST14, step ST11, step ST12, and step ST13, or may perform the processes of steps ST11 to ST13 and step ST14 in parallel.

[0081] In this way, the position detection device 1a can determine the more likely distance to the target feature point (determined distance) with an accuracy equal to or higher than the distance resolution of the radio wave sensor. Therefore, the position detection device 1a can specify the three-dimensional position of the target feature point, in other words, the three-dimensional position of the occupant, with higher accuracy than when the distance resolution of the radio wave sensor is not taken into consideration.

[0082] The hardware configuration of the position detection device 1a according to the second embodiment is similar to the hardware configuration of the position detection device 1 according to the first embodiment described with reference to FIGS. 7A and 7B, and therefore is not illustrated. In the second embodiment, the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15 are realized by a processing circuit 91. That is, the position detection device 1a includes the processing circuit 91 for controlling detection of a three-dimensional position of a person from an image captured by the camera 2 and sensor information acquired by the radio wave sensor. The processing circuit 91 reads out and executes the programs stored in the memory 95, thereby executing the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15. That is, the position detection device 1a includes a memory 95 for storing a program which, when executed by the processing circuit 91, results in the execution of steps ST11 to ST20 in FIG. 10 described above. It can also be said that the programs stored in the memory 95 cause a computer to execute the procedures or methods of the processing of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15. The position detection device 1a also includes an input interface device 92 and an output interface device 93 that perform wired or wireless communication with devices such as the camera 2, the radio wave sensor, the body size detection device 4, or the collision safety device 5.

[0083] In the above-described second embodiment, the position detection device 1a can also detect the three-dimensional positions of a plurality of target feature points, similar to the position detection device 1 according to the first embodiment.

[0084] In the second embodiment, the position detection device 1a can also identify the three-dimensional positions of passengers in the rear seats.

[0085] In the second embodiment described above, for example, a plurality of cameras 2 may be installed in the vehicle cabin, and a plurality of distance measuring sensors 3 may be installed in the vehicle cabin.

[0086] In addition, in the above-mentioned embodiment 2, some of the feature point detection unit 11, angle calculation unit 12, distance detection unit 13, position identification unit 14a, and distance estimation unit 15 may be mounted on an in-vehicle device of a vehicle, and the rest may be provided on a server connected to the in-vehicle device via a network, so that a position detection system is formed by the in-vehicle device and the server. Moreover, the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15 may all be provided in the server.

[0087] In the above-described second embodiment, part of the position information acquisition unit 41 and the physique determination unit 42 may be provided in the in-vehicle device and the rest in the server, or the position information acquisition unit 41 and the physique determination unit 42 may be provided in the server. Also, the collision safety device 5 may be provided in the server.

[0088] In the second embodiment, the position detection device 1a can detect the three-dimensional positions of occupants in various vehicles. The position detection device 1a can detect the three-dimensional positions of people in the real space, such as people in a living room, not limited to occupants of a vehicle.

[0089] As described above, according to the second embodiment, the distance measuring sensor 3 is a radio wave sensor, and in the position detection device 1a, the feature point detection unit 11 detects a plurality of feature points including a target feature point, the position detection device 1a includes a distance estimation unit 15 that estimates a second distance from the camera 2 to the target feature point based on the plurality of feature points detected by the feature point detection unit 11, and the position identification unit 14a determines a post-determination distance from the radio wave sensor to the target feature point based on the first distance detected by the distance detection unit 13 and the second distance estimated by the distance estimation unit 15, and identifies the three-dimensional position of the target feature point based on the feature point position angle and the post-determination distance calculated by the angle calculation unit 12. Therefore, the position detection device 1a can identify the three-dimensional position of a person with higher accuracy than when the distance resolution of the radio wave sensor is not taken into consideration.

[0090] Embodiment 3 In the second embodiment, the distance measurement sensor is a radio wave sensor, and the three-dimensional position of a person is identified with higher accuracy by taking into consideration the distance from the camera to the person estimated based on the captured image. In the third embodiment, an embodiment will be described in which it is determined whether or not to take into account the distance from the camera to the person estimated based on the captured image depending on whether or not the person is wearing an accessory.

[0091] In the following third embodiment, as in the second embodiment, as an example, a person whose three-dimensional position is to be identified is an occupant sitting in the driver's seat or the passenger seat of a vehicle, and the position of the occupant is indicated by the position of the occupant's head. The camera and the distance measuring sensor are respectively installed at the positions as described in the first embodiment with reference to Fig. 3. In the third embodiment, as in the second embodiment, the distance measuring sensor is assumed to be a radio wave sensor.

[0092] FIG. 11 is a diagram showing an example of the configuration of a position detection device 1b according to the third embodiment. In the configuration example of the position detection device 1b shown in FIG. 11, the same components as those of the position detection device 1a according to the second embodiment described using FIG. 8 in the second embodiment are denoted by the same reference numerals, and duplicated explanations will be omitted. The position detection device 1b according to the third embodiment differs from the position detection device 1a according to the second embodiment described with reference to FIG. Furthermore, in the position detection device 1b according to the third embodiment, the specific operation of the position specifying unit 14b differs from the specific operation of the position specifying unit 14a in the position detection device 1a according to the second embodiment.

[0093] The attachment detection unit 16 acquires a captured image from the camera 2, and detects an attachment worn by the occupant based on the acquired captured image. In the third embodiment, the attachment is assumed to be, for example, sunglasses, a mask, or the like, which covers a feature point indicating a part of the occupant's body. The attachment detection unit 16 may detect the attachment worn by the occupant using a known technique such as a known image recognition technique. The attachment detection unit 16 outputs information indicating whether an attachment worn by the occupant has been detected (hereinafter referred to as "attachment detection information") to the position identification unit 14b.

[0094] Based on the accessory detection information output from the accessory detection unit 16, when the accessory detection unit 16 detects an accessory worn by an occupant, the position identification unit 14b identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit 12 and the first distance detected by the distance detection unit 13. Note that the specific method by which position identifying unit 14b identifies the three-dimensional position of the target feature point based on the feature point position angle and the first distance is similar to the specific method by which position identifying unit 14 identifies the three-dimensional position of the target feature point based on the feature point position angle and the first distance in embodiment 1, so a duplicated description will be omitted.

[0095] On the other hand, when the attachment detection unit 16 does not detect an attachment worn by the occupant, the position identification unit 14b executes a "distance determination process" and determines the determined distance based on the first distance detected by the distance detection unit 13 and the second distance estimated by the distance estimation unit 15. The details of the "distance determination process" have been explained in the second embodiment, so a duplicated explanation will be omitted. Then, the position identification unit 14b identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit 12 and the determined distance. Note that the specific method by which position identifying unit 14b identifies the three-dimensional position of the target feature point based on the feature point position angle and the post-determination distance is similar to the specific method by which position identifying unit 14a identifies the three-dimensional position of the target feature point based on the feature point position angle and the post-determination distance in embodiment 2, and therefore a duplicated description will be omitted.

[0096] The position identification unit 14a of the position detection device 1a according to embodiment 2 uniformly executes a "distance determination process" for each first distance detected by the distance detection unit 13 to determine a determined distance, and identifies the three-dimensional position of the target feature point from the determined distance and the feature point position angle. However, when the occupant is wearing an attachment, the number of feature points detected by the feature point detection unit 11 decreases. As a result, the error of the second distance estimated by the distance estimation unit 15 increases. Then, the position detection device 1a may not properly execute the "distance determination process," and the determined distance value may become less reliable than the first distance value. Therefore, in the third embodiment, the position identification unit 14b executes the "distance determination process" only when no attachment worn by the occupant is detected, in other words, when the occupant is not wearing any attachment.

[0097] The operation of the position detection device 1b according to the third embodiment will be described. FIG. 12 is a flowchart for explaining the operation of the position detection device 1b according to the third embodiment. In the flowchart of FIG. 12, the specific processing contents of steps ST111 to ST114 are similar to the specific processing contents of steps ST11 to ST14 in FIG. 10 that have been described in the second embodiment, and therefore duplicated description will be omitted.

[0098] The attachment detection unit 16 acquires a captured image from the camera 2, and detects an attachment worn by the occupant based on the acquired captured image (step ST115). The attachment detector 16 outputs attachment detection information to the position specifier 14b.

[0099] The position identifying unit 14b determines, based on the attachment detection information output from the attachment detecting unit 16 in step ST115, whether or not the attachment detecting unit 16 has detected an attachment worn by the occupant (step ST116). If the attachment detection unit 16 detects an attachment worn by the occupant (if "YES" in step ST116), the position identification unit 14b identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit 12 in step ST112 and the first distance detected by the distance detection unit 13 in step ST114 (step ST118).

[0100] On the other hand, if the attachment detection unit 16 does not detect an attachment worn by the occupant ("NO" in step ST116), the position identification unit 14b executes a "distance determination process" to determine the determined distance (step ST117; see steps ST15 to ST19 in FIG. 10). Then, the position identification unit 14b identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit 12 in step ST112 and the determined distance determined in step ST117 (step ST118).

[0101] In the flowchart shown in FIG. 12, the position detection device 1b performs the processes in the order of step ST111, step ST112, step ST113, step ST114, and step ST115, but this is merely an example. For example, the position detection device 1b may perform the process of steps ST111 to ST113, the process of step ST114, and the process of step ST115 in parallel. Furthermore, the position detection device 1b may perform the process of step ST114 or the process of step ST115 before the process of steps ST111 to ST113. Furthermore, the position detection device 1b may perform the process of step ST115 before the process of step ST114.

[0102] In this way, the position detection device 1b can determine a more likely distance to the target feature point (determined distance) with an accuracy equal to or higher than the distance resolution of the radio wave sensor, using the distance from the camera 2 to the target feature point (second distance) estimated from the captured image. However, when the occupant is wearing an attachment and it is assumed that there is a high possibility that an error will occur in the second distance, the position detection device 1b does not determine the post-determined distance. Therefore, the position detection device 1b can identify the three-dimensional position of the target feature point, in other words, the three-dimensional position of the occupant, with higher accuracy than in a case where it is not taken into account that the occupant is wearing an attachment.

[0103] The hardware configuration of the position detection device 1b according to the third embodiment is similar to the hardware configuration of the position detection device 1 according to the first embodiment described with reference to FIGS. 7A and 7B, and therefore is not illustrated. In the third embodiment, the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14b, the distance estimation unit 15, and the attachment detection unit 16 are realized by a processing circuit 91. That is, the position detection device 1b includes the processing circuit 91 for controlling detection of a three-dimensional position of a person from an image captured by the camera 2 and sensor information acquired by the radio wave sensor. The processing circuit 91 executes the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14b, the distance estimation unit 15, and the attachment detection unit 16 by reading and executing the programs stored in the memory 95. That is, the position detection device 1b includes a memory 95 for storing a program that, when executed by the processing circuit 91, results in the execution of steps ST111 to ST118 in FIG. 12 described above. It can also be said that the programs stored in the memory 95 cause a computer to execute the procedures or methods of the processing of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14b, the distance estimation unit 15, and the attachment detection unit 16. The position detection device 1b also includes an input interface device 92 and an output interface device 93 that perform wired or wireless communication with devices such as the camera 2, the radio wave sensor, the body size detection device 4, or the collision safety device 5.

[0104] In the above-described third embodiment, the position detection device 1b can also detect the three-dimensional positions of a plurality of target feature points, similar to the position detection device 1a according to the second embodiment.

[0105] In the third embodiment, the position detection device 1b can also identify the three-dimensional positions of passengers in the rear seats.

[0106] In the third embodiment, for example, a plurality of cameras 2 may be installed in the vehicle cabin, and a plurality of distance measuring sensors 3 may be installed in the vehicle cabin.

[0107] In addition, in the above-mentioned embodiment 3, some of the feature point detection unit 11, angle calculation unit 12, distance detection unit 13, position identification unit 14b, distance estimation unit 15, and attachment detection unit 16 may be mounted on an in-vehicle device of a vehicle, and the rest may be provided in a server connected to the in-vehicle device via a network, so that the in-vehicle device and the server form a position detection system. Furthermore, the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14b, the distance estimation unit 15, and the attachment detection unit 16 may all be provided in the server.

[0108] In the above-described third embodiment, part of the position information acquisition unit 41 and the physique determination unit 42 may be provided in the in-vehicle device and the rest in the server, or the position information acquisition unit 41 and the physique determination unit 42 may be provided in the server. Also, the collision safety device 5 may be provided in the server.

[0109] In the third embodiment, the position detection device 1b can detect the three-dimensional positions of occupants in various vehicles. The position detection device 1b can detect the three-dimensional positions of people in the real space, such as people in a room, not limited to occupants of a vehicle.

[0110] As described above, according to the third embodiment, the position detection device 1b includes the attachment detection unit 16 that detects an attachment worn by a person based on a captured image, and the position identification unit 14b is configured to identify the three-dimensional position of a target feature point based on the feature point position angle and the first distance when the attachment detection unit 16 detects an attachment, and to identify the three-dimensional position of the target feature point based on the feature point position angle and the post-determination distance when the attachment detection unit 16 does not detect an attachment. Therefore, when identifying the three-dimensional position of a person taking into account the distance resolution of the radio wave sensor, the position detection device 1b can identify the three-dimensional position of a person with higher accuracy than when not taking into account whether or not the person is wearing an attachment.

[0111] Embodiment 4 In the fourth embodiment, an embodiment will be described in which a reference inter-feature point distance used when estimating a distance from a camera to a person based on a captured image is set using a first distance detected from sensor information.

[0112] In the following embodiment 4, as in embodiment 2, as an example, a person whose three-dimensional position is to be specified is an occupant sitting in the driver's seat or the passenger seat of a vehicle, and the position of the occupant is indicated by the position of the occupant's head. Also, the camera and the distance measuring sensor are respectively installed at the positions as described in embodiment 1 using FIG. 3. Also, in the fourth embodiment, similarly to the second embodiment, it is assumed that the distance measuring sensor is a radio wave sensor.

[0113] FIG. 13 is a diagram showing an example of the configuration of a position detection device 1c according to the fourth embodiment. In the configuration example of the position detection device 1c shown in FIG. 13, the same components as those of the position detection device 1a according to embodiment 2 described using FIG. 8 in embodiment 2 are denoted by the same reference numerals, and duplicated explanations will be omitted. In the fourth embodiment, in a position detection device 1c, the specific operation of a distance estimation unit 15a is different from the specific operation of the distance estimation unit 15 in the position detection device 1a according to the second embodiment.

[0114] Distance estimation unit 15a estimates a second distance from camera 2 to a target feature point based on a reference inter-feature point distance (hereinafter referred to as a "second reference inter-feature point distance") and an inter-feature point distance, which is a distance between feature points for distance calculation among the multiple feature points detected by feature point detection unit 11. In the fourth embodiment, the second reference inter-feature point distance is set by distance estimation unit 15a based on the inter-feature point distance and the first distance detected by distance detection unit 13, and is stored in distance estimation unit 15a, before distance estimation unit 15a estimates the second distance.

[0115] Specifically, the distance estimation unit 15a first performs a "reference inter-feature point distance setting process" to set a second reference inter-feature point distance based on the inter-feature point distance, which is the distance between feature points for distance calculation among the multiple feature points detected by the feature point detection unit 11, and the first distance detected by the distance detection unit 13. After setting the second reference inter-feature point distance by performing the "reference inter-feature point distance setting process", the distance estimation unit 15a stores the set second reference inter-feature point distance. The "reference inter-feature point distance setting process" is performed until the distance estimation unit 15a estimates the second distance. Then, after performing the "reference inter-feature point distance setting process", distance estimation unit 15a estimates a second distance from camera 2 to the target feature point, based on the inter-feature point distance, which is the distance between feature points for distance calculation among the multiple feature points detected by feature point detection unit 11, and the stored second reference inter-feature point distance.

[0116] Here, FIGS. 14A and 14B are diagrams for explaining an example of a method in which distance estimation unit 15a sets a second reference inter-feature point distance and estimates the second distance using the set second reference inter-feature point distance in embodiment 4. Here, the feature points for distance calculation are a feature point corresponding to the left eye of the occupant and a feature point corresponding to the right eye of the occupant, and the distance between the feature points is the distance between the eyes. FIG. 14A is a diagram for explaining an example of the contents of the “reference inter-feature point distance setting process” performed by distance estimation unit 15a, and FIG. 14B is a diagram for explaining an example of a method in which distance estimation unit 15a estimates the second distance by using the second reference inter-feature point distance after setting the second reference inter-feature point distance as shown in FIG. 14A.

[0117] The "reference feature point distance setting process" performed by the distance estimation unit 15a will be described with reference to FIG. 14A. 14A, it is assumed that the distance estimation unit 15a calculates the inter-feature point distance, in other words, the interocular distance, to be 50 [px] from the captured image, and the distance detection unit 13 detects the first distance to be 1 [m] based on the sensor information. In this case, the actual interocular distance [m] of the occupant is expressed by the following equation (4). Interocular distance [m]: 1 Distance [m] = Interocular distance [px]: focal length of camera 2 [px] (4) The value of the focal length of the camera 2 is stored in advance in the distance estimation unit 15a. Here, the focal length is set to f.

[0118] That is, in the example shown in FIG. 14A, the interocular distance [m] is calculated as in the following formula (5). Interocular distance [m]: 1 [m] = 50 [px]: f [px] Interocular distance [m] × f [px] = 50 [px] × 1 [m] Interocular distance [m] = 50 / f [m] (5)

[0119] The distance estimation unit 15a stores the calculated interocular distance value (50 / f [m]) as the value of the second reference inter-feature point distance. The distance estimation unit 15a may execute the "reference inter-characteristic point distance setting process" as described with reference to FIG. 14A, for example, once while the vehicle 1000 is traveling. Furthermore, for example, the distance estimation unit 15a may execute the "reference feature point distance setting process" multiple times within a preset time period and set the average of the values ​​of the second reference feature point distances calculated in each second reference feature point distance setting process as the value of the second reference feature point distance, or may set the value of the second reference feature point distance by multiplying it by an update coefficient. The distance estimation unit 15a multiplies the reference inter-feature point distance by an update coefficient, gradually updates the reference inter-feature point distance, and sets the value of the second reference inter-feature point distance. For example, if the update coefficient is 0.2, the distance estimation unit 15a updates the reference inter-feature point distance by 20% at a time. As a specific example, it is assumed that the reference inter-feature point distance is 50 mm, and the second inter-feature point distance is 40 mm. In this case, the distance estimation unit 15a updates the reference inter-feature point distance by 2 mm, which is 20% of the difference of 10 mm between the reference inter-feature point distance and the second inter-feature point distance, and sets 48 mm, which is the updated value of the reference inter-feature point distance, as the value of the second inter-reference feature point distance. The distance estimation unit 15a repeats such an update process, and sets the value of the second inter-reference feature point distance in stages.

[0120] Next, an example of a method in which the distance estimation unit 15a estimates the second distance using the distance between the second reference feature points will be described with reference to Fig. 14B. Note that, here, it is assumed that the value of the distance between the second reference feature points is set to "50 / f[m]" as in the above example. When the distance estimation unit 15a sets and stores the second reference inter-feature point distance, the distance estimation unit 15a estimates the second distance based on the second reference inter-feature point distance and the inter-feature point distance, which is the inter-eye distance here. For example, as shown in FIG. 14B, it is assumed that the distance estimation unit 15a calculates the interocular distance to be 60 [px] based on the captured image. Here, if the second distance to be obtained is Dc1 [m], the distance estimation unit 15a calculates Dc1 [m] from the above equations (4) and (5) as follows. 50 / f[m]:Dc1[m]=60[px]:f[px] Dc1[m]×60[px]=50 / f[m]×f[px] Dc1 [m] = 50 / 60 = 0.83 [m] Thus, in the example shown in FIG. 14B, distance estimation unit 15a estimates the second distance to be 0.83 [m].

[0121] Distance estimation unit 15a outputs camera distance information regarding the estimated second distance to position identification unit 14a.

[0122] The operation of the position detection device 1c according to the fourth embodiment will be described. FIG. 15 is a flowchart for explaining the operation of the position detection device 1c according to the fourth embodiment. In the flowchart of FIG. 15, the specific processing contents of steps ST1111 to ST1113 and step ST1117 are similar to the specific processing contents of steps ST11 to ST12, step ST14 and step ST20 in FIG. 10, which have already been explained in embodiment 2, and therefore duplicate explanations will be omitted. In the flowchart of FIG. 15, the specific processing content of step ST1116 is similar to the specific processing content of steps ST15 to ST19 in FIG. 10 that have been explained in the second embodiment, so a duplicated explanation will be omitted.

[0123] The distance estimation unit 15a performs a "reference inter-feature point distance setting process" for setting a second reference inter-feature point distance based on the inter-feature point distance, which is the distance between the feature points for distance calculation among the multiple feature points detected by the feature point detection unit 11 in step ST1111, and the first distance detected by the distance detection unit 13 in step ST1113, and sets the second reference inter-feature point distance (step ST1114). Then, the distance estimation unit 15a stores the set second reference inter-feature point distance.

[0124] After setting and storing the second reference inter-feature point distance in step ST1114, the distance estimation unit 15a estimates a second distance from the camera 2 to the target feature point based on the inter-feature point distance, which is the distance between the feature points for distance calculation among the multiple feature points detected by the feature point detection unit 11 in step ST1111, and the stored second reference inter-feature point distance (step ST1115).

[0125] After the distance estimation unit 15a sets and stores the second reference inter-feature point distance, the position detection device 1c can omit the process of step ST1114. In addition, in the flowchart shown in FIG. 15, the position detection device 1c performs the processes in the order of step ST1111, step ST1112, and step ST1113, but this is merely an example. For example, the position detection device 1c may perform the processes in the order of step ST1113, step ST1111, and step ST1112, or may perform the processes of steps ST1111 to ST1112 and the process of step ST1113 in parallel.

[0126] In this manner, the position detection device 1c sets the second reference inter-feature point distance used when estimating the second distance from the camera 2 to the target feature point based on the captured image, by using the sensor information acquired from the radio wave sensor. If the value of the second reference inter-feature point distance is a value set without considering individual differences such as age or face shape, there is a possibility that an error will occur in the second distance estimated by distance estimation unit 15a. In contrast, position detection device 1c can estimate the second distance with high accuracy by considering individual differences such as age or face shape. As a result, when determining the three-dimensional position of the occupant by taking into account the distance resolution of the radio wave sensor, the position detection device 1c can determine the three-dimensional position of the target feature point, in other words, the three-dimensional position of the occupant, with higher accuracy.

[0127] The hardware configuration of the position detection device 1c according to the fourth embodiment is similar to the hardware configuration of the position detection device 1 according to the first embodiment described with reference to FIGS. 7A and 7B, and therefore is not illustrated. In the fourth embodiment, the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15a are realized by a processing circuit 91. That is, the position detection device 1c includes the processing circuit 91 for controlling detection of a three-dimensional position of a person from an image captured by the camera 2 and sensor information acquired by the radio wave sensor. The processing circuit 91 executes the functions of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15a by reading and executing the programs stored in the memory 95. That is, the position detection device 1c includes a memory 95 for storing a program that, when executed by the processing circuit 91, results in the execution of steps ST1111 to ST117 in FIG. 15 described above. It can also be said that the programs stored in the memory 95 cause a computer to execute the procedures or methods of the processing of the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15a. The position detection device 1c also includes an input interface device 92 and an output interface device 93 that perform wired or wireless communication with devices such as the camera 2, the radio wave sensor, the body size detection device 4, or the collision safety device 5.

[0128] In the above-described fourth embodiment, the position detection device 1c can also detect the three-dimensional positions of a plurality of target feature points, similar to the position detection device 1a according to the second embodiment.

[0129] In the fourth embodiment, the position detection device 1c can also identify the three-dimensional positions of passengers in the rear seats.

[0130] In the fourth embodiment, for example, a plurality of cameras 2 may be installed in the vehicle cabin, and a plurality of distance measuring sensors 3 may be installed in the vehicle cabin.

[0131] In addition, in the above-mentioned embodiment 4, some of the feature point detection unit 11, angle calculation unit 12, distance detection unit 13, position identification unit 14a, and distance estimation unit 15a may be mounted on an in-vehicle device of a vehicle, and the rest may be provided on a server connected to the in-vehicle device via a network, so that a position detection system is formed by the in-vehicle device and the server. Moreover, the feature point detection unit 11, the angle calculation unit 12, the distance detection unit 13, the position identification unit 14a, and the distance estimation unit 15a may all be provided in the server.

[0132] In the above-described fourth embodiment, part of the position information acquisition unit 41 and the physique determination unit 42 may be provided in the in-vehicle device and the rest in the server, or the position information acquisition unit 41 and the physique determination unit 42 may be provided in the server. Also, the collision safety device 5 may be provided in the server.

[0133] In the fourth embodiment, the position detection device 1c can detect the three-dimensional positions of occupants in various vehicles. The position detection device 1c can detect the three-dimensional positions of people in the real space, such as people in a living room, not limited to occupants of a vehicle.

[0134] As described above, according to the fourth embodiment, in the position detection device 1c, the distance estimation unit 15a estimates the second distance from the camera 2 to the target feature point based on the feature point distance, which is the distance between the feature points for distance calculation among the multiple feature points detected by the feature point detection unit 11, and the reference feature point distance (second reference feature point distance), and the reference feature point distance is set based on the feature point distance and the first distance detected by the distance detection unit 13. Therefore, the position detection device 1c can estimate the second distance with high accuracy, taking into account individual differences such as age or face shape. As a result, the position detection device 1c can specify the three-dimensional position of the occupant with high accuracy when specifying the three-dimensional position of the occupant taking into account the distance resolution of the radio wave sensor.

[0135] In the physique detection system 100 according to the above-described first to fourth embodiments, the physique detection device 4 may be configured to determine whether or not the occupant is properly seated, and to perform a physique determination when it is determined that the occupant is properly seated. In the first to fourth embodiments, the term "normal seated state" refers to a state in which a person's posture is not compromised. Hereinafter, the details of the physique detection device 4 that performs the physique determination when it is determined that the occupant is in a properly seated state in the first to fourth embodiments will be described.

[0136] In the physique detection device 4, the physique determination unit 42 determines whether or not the three-dimensional positions of the target feature points identified by the position identification units 14, 14a, and 14b in the position detection devices 1, 1a, 1b, and 1c are within the reference feature point region. The physique determination unit 42 can identify the three-dimensional positions of the target feature points identified by the position identification units 14, 14a, and 14b from the position information acquired by the position information acquisition unit 41. In the first to fourth embodiments, the "reference feature point region" is a region in real space in which the target feature points are assumed to exist when the occupant is in a normal seated position. The reference feature point area is set in advance as an area that includes all of the positions at which target feature points of multiple people having various different physiques are present when the multiple people are properly seated, and the physique determination unit 42 stores information regarding the reference feature point area.

[0137] The physique determining unit 42 determines the physique of the occupant when it determines that the three-dimensional position of the target feature point identified by the position identifying units 14, 14a, 14b is within the reference feature point region. On the other hand, if the physique determining unit 42 determines that the three-dimensional position of the target feature point identified by the position identifying units 14, 14a, 14b is not within the reference feature point region, the physique determining unit 42 does not determine the physique of the occupant.

[0138] When determining the physique of an occupant using the physique detection device 4, if the occupant is not in a properly seated position, there is a possibility that the physique of the occupant will be erroneously determined. The physical build detection device 4 determines whether the occupant is properly seated or not, and if it is determined that the occupant is properly seated, performs a physical build determination of the occupant, thereby reducing erroneous determination of the occupant's physical build.

[0139] FIG. 16 is a flowchart for explaining the operation of the physique detection device 4 in the physique detection system 100 according to the first to fourth embodiments, in the case where a physique determination is performed when it is determined that an occupant is in a properly seated state.

[0140] The position information acquisition unit 41 acquires position information from the position detection device 1 (step ST401). Based on the position information acquired by the position information acquisition unit 41 in step ST401, the physique determination unit 42 determines whether the three-dimensional position of the target feature point identified by the position identification units 14, 14a, 14b in the position detection devices 1, 1a, 1b, 1c is within the reference feature point area (step ST402).

[0141] In step ST402, if it is determined that the three-dimensional position of the target feature point identified by the position identification units 14, 14a, 14b is within the reference feature point area ("YES" in step ST402), the physique determination unit 42 determines the physique of the occupant (step ST403).

[0142] In step ST402, if it is determined that the three-dimensional position of the target feature point identified by the position identifying units 14, 14a, and 14b is not within the reference feature point area ("NO" in step ST402), the operation of the physique detection device 4 returns to the processing of step ST401.

[0143] In this way, in the physique detection device 4 of the physique detection system 100 according to embodiments 1 to 4, the physique determination unit 42 may perform physique determination of the occupant when the three-dimensional position of the target feature point identified by the position identification units 14, 14a, 14b of the position detection devices 1, 1a, 1b, 1c is within a reference feature point area in real space where the target feature point is expected to exist when the occupant is in a properly seated state. As a result, in the physique detection system 100, the physique detection device 4 can reduce erroneous determination of the occupant's physique.

[0144] An example of the hardware configuration of the physique detection device 4 according to embodiments 1 to 4 is similar to the example of the hardware configuration of the position detection devices 1, 1a, 1b, and 1c shown in Figures 7A and 7B, so it is not illustrated. In the first to fourth embodiments, the functions of the position information acquisition unit 41 and the physique determination unit 42 are realized by the processing circuit 91. That is, the physique detection device 4 includes the processing circuit 91 for determining the physique of an occupant based on the three-dimensional position of the occupant identified by the position detection devices 1, 1a, 1b, and 1c. The processing circuitry 91 may be dedicated hardware as shown in FIG. 7A, or may be a processor 94 executing a program stored in a memory as shown in FIG. 7B.

[0145] When the processing circuitry 91 is dedicated hardware, the processing circuitry 91 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0146] When the processing circuit is the processor 94, the functions of the position information acquisition unit 41 and the physique determination unit 42 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 95. The processor 94 executes the functions of the position information acquisition unit 41 and the physique determination unit 42 by reading and executing the program stored in the memory 95. That is, the physique detection device 4 includes a memory 95 for storing a program that, when executed by the processor 94, results in the execution of processes such as steps ST1 to ST4 in FIG. 16 described above. It can also be said that the program stored in the memory 95 causes a computer to execute the procedures or methods of the position information acquisition unit 41 and the physique determination unit 42. Here, memory 95 refers to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), etc., or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), etc.

[0147] The functions of the position information acquisition unit 41 and the physique determination unit 42 may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the function of the position information acquisition unit 41 may be realized by a processing circuit 91 as dedicated hardware, and the function of the physique determination unit 42 may be realized by a processor 94 reading and executing a program stored in a memory 95.

[0148] The physique detection device 4 also includes an input interface device 92 and an output interface device 93 that perform wired or wireless communication with devices such as the position detection devices 1, 1a, 1b, and 1c, or the collision safety device 5.

[0149] In addition, in embodiments 1 to 4, the physique detection device 4 can determine the physique of not only the occupants of the vehicle 1000, but also occupants of various vehicles or people existing in real space, based on position information regarding the three-dimensional position of the person detected by the position detection devices 1, 1a, 1b, and 1c.

[0150] In addition, the present disclosure allows free combination of the respective embodiments, modification of any of the components of each embodiment, or omission of any of the components of each embodiment. [Industrial Applicability]

[0151] The position detection device according to the present disclosure can identify the three-dimensional position of each individual person without performing personal authentication processing for any person. [Explanation of symbols]

[0152] 1, 1a, 1b, 1c position detection device, 2 camera, 3 distance measurement sensor, 4 body size detection device, 5 collision safety device, 11 feature point detection unit, 12 angle calculation unit, 13 distance detection unit, 14, 14a, 14b position identification unit, 15, 15a distance estimation unit, 16 attachment detection unit, 41 position information acquisition unit, 42 body size determination unit, 100 body size detection system, 1000 vehicle, 91 processing circuit, 92 input interface device, 93 output interface device, 94 processor, 95 memory.

Claims

1. a feature point detection unit that detects target feature points corresponding to body parts of a person in a captured image based on the captured image captured by the camera; an angle calculation unit that calculates a feature point position angle indicating an angle of a straight line connecting the camera and the target feature point with respect to an imaging axis of the camera, based on a position of the target feature point on the captured image detected by the feature point detection unit; a distance detection unit that detects a first distance from the distance measurement sensor to the target feature point based on sensor information acquired by the distance measurement sensor; a position specifying unit that specifies a three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit and the first distance detected by the distance detection unit; A position detection device comprising:

2. the distance measuring sensor is a radio wave sensor, The feature point detection unit detects a plurality of feature points including the target feature point, a distance estimation unit that estimates a second distance from the camera to the target feature point based on the plurality of feature points detected by the feature point detection unit, The position identification unit determines a post-determined distance from the radio wave sensor to the target feature point based on the first distance detected by the distance detection unit and the second distance estimated by the distance estimation unit, and identifies the three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit and the post-determined distance.

2. The position detection device according to claim 1.

3. a wearable item detection unit that detects an wearable item worn by the person based on the captured image, when the attachment detection unit detects the attachment, the position identification unit identifies the three-dimensional position of the target feature point based on the feature point position angle and the first distance; When the attachment detection unit does not detect the attachment, the three-dimensional position of the target feature point is identified based on the feature point position angle and the determined distance.

3. The position detection device according to claim 2.

4. the distance estimation unit estimates the second distance from the camera to the target feature point based on a feature point-to-feature point distance, which is a distance between feature points for distance calculation among the plurality of feature points detected by the feature point detection unit, and a reference feature point-to-feature point distance; The reference inter-feature point distance is set based on the inter-feature point distance and the first distance detected by the distance detection unit.

3. The position detection device according to claim 2.

5. the feature point detection unit detects the target feature point based on the captured image captured by the camera, which is installed at the same position in at least one of a lateral position, a height position, a depth position, and an installation inclination in a three-dimensional space as the distance measurement sensor, and is installed in an orientation capable of capturing an image of a real space in which the target feature point is assumed to exist; The distance detection unit detects the first distance based on the sensor information sensed by the distance measurement sensor, which is installed in the same position as the camera in at least one of a lateral position, a height position, a depth position, and an installation inclination in a three-dimensional space, and is installed in an orientation capable of sensing a real space in which the target feature point is assumed to exist.

2. The position detection device according to claim 1.

6. A position detection device according to any one of claims 1 to 5, a position information acquisition unit that acquires position information regarding the three-dimensional position of the target feature point identified by the position identification unit; a physique determination unit that determines a physique of the person based on the location information acquired by the location information acquisition unit, The physique determination unit determines the physique of the person when the three-dimensional position of the target feature point identified by the position identification unit is within a reference feature point area in real space in which the target feature point is assumed to exist when the person is in a normal seated state. A body size detection system comprising:

7. A feature point detection unit detects target feature points corresponding to body parts of a person in a captured image based on the captured image captured by the camera; an angle calculation unit calculating a feature point position angle indicating an angle of a straight line connecting the camera and the target feature point with respect to an imaging axis of the camera, based on a position on the captured image of the target feature point detected by the feature point detection unit; a distance detection unit detecting a first distance from the distance measurement sensor to the target feature point based on sensor information acquired by the distance measurement sensor; a position specifying unit specifying a three-dimensional position of the target feature point based on the feature point position angle calculated by the angle calculation unit and the first distance detected by the distance detection unit; A position detection method comprising:

Citation Information

Patent Citations

  • Laser data recognition method and device, robot and storage medium

    CN108710367A

  • Distance measuring device, distance measuring method, and program

    JP2017181209A

  • Sensing device

    JP2020044153A

  • Method and device for recognizing a known object in a field of view of a three-dimensional machine vision system

    US20160224858A1

  • Surveying Instrument

    US20160349051A1