Vehicle control system

Abstract

Provided is a vehicle control device capable of highly accurate sensing, by reducing incorrect grouping of millimeter-wave radars by using a camera x millimeter-wave radar algorithm, in driving assistance technologies. This vehicle control device comprises: a camera that recognizes a three-dimensional object around a host vehicle; and a millimeter-wave radar that recognizes peripheral targets of the host vehicle, sets the three-dimensional object as an independent target on the basis of the information on the three-dimensional object recognized by the camera, and when the millimeter-wave radar recognizes the three-dimensional object and another target as peripheral targets, recognizes the three-dimensional object as a target different from the other target.

Description

[Technical Field]

[0001] This invention relates to a vehicle control device. [Background technology]

[0002] In automotive driver assistance control systems, systems that use forward sensors, such as cameras and millimeter-wave radar, to detect obstacles ahead and automatically apply the brakes to avoid collisions or mitigate collision damage when there is a risk of collision are widely used.

[0003] In recent years, in order to further enhance safety, there has been a need to appropriately detect obstacles (pedestrians, bicycles, etc.) and apply the brakes even when obstacles suddenly appear in front of the vehicle from outside the detection range of the forward sensors at intersections and crosswalks. In addition, in vehicle assessments (e.g., EURO NCAP), protocols such as the intersection AEB forward-moving pedestrian protocol have emerged to avoid collisions with pedestrians crossing the sidewalk in the same direction as the vehicle turning at an intersection.

[0004] These systems need to be able to detect obstacles well before a collision occurs. However, for obstacles that suddenly appear in front of the vehicle from outside the detection range of the forward sensors, detection using forward sensors consisting of cameras or millimeter-wave radar is insufficient.

[0005] Currently, due to cost considerations, cameras are not installed on the sides, and sensing is almost always done using only millimeter-wave radar. However, while millimeter-wave radar is generally cheaper than cameras, its detection accuracy is lower, so controlling a vehicle based solely on millimeter-wave radar sensing results carries a high risk of malfunction. For example, when a vehicle is turning at an intersection, the millimeter-wave radar may mistakenly group pedestrians crossing the sidewalk in the same direction with nearby (stationary) objects, potentially leading to a failure to correctly detect pedestrians and a collision.

[0006] Therefore, as a countermeasure, camera-millimeter-wave radar fusion technology, which uses millimeter-wave radar and cameras to improve detection accuracy, is attracting attention, and for example, the following Patent Document 1 is disclosed.

[0007] Patent Document 1 discloses a technology that improves the obstacle detection rate by determining whether or not an object is moving on the road surface based on object information obtained by radar and vehicle driving information. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2012-103858 [Overview of the project] [Problems that the invention aims to solve]

[0009] Patent Document 1 assumes that the detection results of target information obtained by radar are correct and does not take into account radar misgrouping. Therefore, if the radar mistakenly groups moving targets such as pedestrians with stationary targets such as posts, the speed of the moving targets may be detected as smaller than it actually is.

[0010] The object of the present invention is to provide a vehicle control device that enables high-precision sensing in driver assistance technology by suppressing misgrouping of millimeter-wave radar using a camera × millimeter-wave radar algorithm. [Means for solving the problem]

[0011] To solve the above problems, the vehicle control device according to the present invention comprises a camera that recognizes three-dimensional objects in the vicinity of the vehicle, and a millimeter-wave radar that recognizes targets in the vicinity of the vehicle, wherein the three-dimensional object is set as an independent target based on the information of the three-dimensional object recognized by the camera, and when the millimeter-wave radar recognizes the three-dimensional object and other targets as surrounding targets, the three-dimensional object is recognized as a target different from the other targets. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide a vehicle control device that can suppress erroneous grouping of millimeter-wave radar.

[0013] Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0014] [Figure 1] A diagram illustrating an example of the hardware configuration of a system including a controller (vehicle control device) in Embodiment 1 of the present invention. [Figure 2] A flowchart showing the processing flow of the fusion program executed by the controller (vehicle control device) in Embodiment 1 of the present invention. [Figure 3] An overhead view of the vehicle and its surrounding objects at a certain time t0 in Embodiment 1 of the present invention. [Figure 4] An overhead view of the vehicle and its surrounding objects at a certain time t1, after a certain time has elapsed from time t0, in Embodiment 1 of the present invention. [Figure 5] This diagram illustrates the radar-detected point cloud as seen on a fixed coordinate system at the time (time t1) shown in Figure 4, in Embodiment 1 of the present invention. [Figure 6] An overhead view of the vehicle and its surrounding objects at a certain time t2, after a certain time has elapsed from time t1, in Embodiment 1 of the present invention. [Figure 7] A flowchart showing the processing flow of the fusion program executed by the controller (vehicle control device) in Embodiment 2 of the present invention. [Figure 8] An overhead view of the vehicle and its surrounding objects at a certain time t3 in Embodiment 2 of the present invention. [Figure 9] This is an overhead view of the vehicle and its surrounding objects at time t3 in Embodiment 2 of the present invention, with a dummy target placed in the camera occlusion region of a stationary object OOBJ. [Figure 10]An overhead view of the vehicle and its surrounding objects at a certain time t4, a certain time after time t3, in Embodiment 2 of the present invention. [Figure 11] This diagram illustrates the radar-detected point cloud as seen on a fixed coordinate system at the time (time t4) shown in Figure 10, in Embodiment 2 of the present invention. [Figure 12] An overhead view of the vehicle and its surrounding objects at a certain time t5, after a certain time has elapsed from time t4, in Embodiment 2 of the present invention. [Figure 13] A flowchart showing the processing flow of the fusion program executed by the controller (vehicle control device) in Embodiment 3 of the present invention. [Figure 14] An overhead view of the vehicle and its surrounding objects at a certain time t6 in Embodiment 3 of the present invention. [Figure 15] This is an overhead view of the vehicle and its surrounding objects at time t6 in Embodiment 3 of the present invention, with a dummy target placed in the camera occlusion area of ​​the pedestrian OPED. [Figure 16] An overhead view of the vehicle and its surrounding objects at a certain time t7, after a certain time has elapsed from time t6, in Embodiment 3 of the present invention. [Figure 17] An explanatory diagram of the radar-detected point cloud as seen on a fixed coordinate system at the time (time t7) shown in Figure 16, in Embodiment 3 of the present invention. [Figure 18] An overhead view of the vehicle and its surrounding objects at a certain time t8, a certain time after time t7, in Embodiment 3 of the present invention. [Modes for carrying out the invention]

[0015] Hereinafter, embodiments of the present invention will be described based on the drawings.

[0016] [Example 1] First, Example 1 of the present invention will be described with reference to Figures 1 to 6.

[0017] In this embodiment, the vehicle coordinate system is defined with the vehicle's front-to-back (length) direction as the X-axis, the vehicle's left-to-right (width) direction as the Y-axis, the center point of the front end of the vehicle body as the origin, and angles set to 0 degrees in the forward direction of the vehicle and positive in a counterclockwise direction.

[0018] Figure 1 is a diagram illustrating an example of the hardware configuration of a system including a controller (vehicle control device) in Embodiment 1 of the present invention.

[0019] This system consists of an external information acquisition unit H1 comprising a radar device H11 that detects the left and right front or side (hereinafter sometimes simply referred to as radar or millimeter-wave radar) and a camera device H12 that detects the front (hereinafter sometimes simply referred to as camera), etc. The vehicle information acquisition unit H2 consists of a speed sensor H21, a steering angle sensor H22, a yaw rate sensor H23, etc. A controller H3 calculates fusion target information by integrating the detection results of radar and camera based on external information from the external information acquisition unit H1 and vehicle information acquired by the vehicle information acquisition unit H2, and outputs warning commands to the driver to avoid collision with the fusion target, brake commands to stop the vehicle, steering commands to turn the vehicle, etc., as needed. The vehicle control unit H4 consists of an alarm device H41 that performs alarm control based on an alarm command calculated by the controller H3, a brake system H42 that performs brake control based on a brake command calculated by the controller H3, and a steering system H43 that performs steering control based on a steering command calculated by the controller H3.

[0020] Figure 2 shows the processing flow of the fusion program executed by the controller H3.

[0021] First, in process S101, the camera detection results (X position, Y position, width, target attributes, etc.) obtained from the external information acquisition unit H1 are acquired.

[0022] Next, in process S102, radar detection point information (X position, Y position, radar reflection intensity, etc.) obtained from the external information acquisition unit H1 is acquired.

[0023] Next, in process S103, the time axis of the radar detection point information and camera detection results acquired in processes S101 and S102 is unified, and the radar detection point information and camera detection results at the same time are calculated.

[0024] Next, in process S104, the position coordinates of the radar detection point information and camera detection results obtained in process S103 at the same time are unified to the center of the front end of the vehicle body.

[0025] Next, in process S105, it is determined whether or not a stationary three-dimensional object exists in the camera detection results obtained in process S104.

[0026] If the camera detects the presence of a stationary object (process S105: Yes), in process S106, the coordinate origin is fixed, and the X position, Y position, and width of the stationary object detected by the camera are stored.

[0027] Next, in process S108, the X and Y positions of the vehicle from a fixed coordinate origin, and the yaw angle when the yaw angle at the time of coordinate fixing is set to 0 degrees are estimated. Furthermore, the coordinate origin of the camera and radar detection results obtained in process S104 is moved to the fixed coordinate origin and rotated by the yaw angle.

[0028] On the other hand, if in process S105 it is determined that no stationary objects are detected by the camera (process S105: No), and then in process S107 it is determined that the coordinate origin is fixed (process S107: Yes), then process S108 is performed.

[0029] Next, in process S109, if it is determined that the vehicle has not passed the stationary object (stationary object detected by the camera) stored in process S106 (process S109: No), then in process S110, radar detection points near the stationary object (stationary object detected by the camera) stored in process S106 are excluded from grouping.

[0030] On the other hand, if process S109 determines that the vehicle has passed the stationary object stored in process S106 (process S109: Yes), the coordinate lock is released in process S111.

[0031] Next, in process S112, grouping is performed on radar detection points that are not to be grouped. This process treats radar detection points that are close together as the same target and combines them into a single representative detection point.

[0032] On the other hand, if in process S105 it is determined that no stationary objects are detected by the camera (process S105: No), and then in process S107 it is determined that the coordinate origin is not fixed (process S107: No), then process S112 is performed.

[0033] The process is now complete.

[0034] A diagram illustrates a scene to explain the process of Example 1 in more detail. As an example, we will describe a scene in chronological order in which, before the vehicle enters an intersection, there is a stationary object at the corner of the intersection, which is detected by the radar device H11 and camera device H12, and a pedestrian is walking on the sidewalk to the left of the vehicle in the same direction as the vehicle, and the pedestrian is near the stationary object when the vehicle turns at the intersection. Figure 3 is an overhead view of the vehicle and its surrounding objects at a certain time t0. In Figure 3, detection area A of radar device H11 L and detection area A of camera device H12 CAM In the region where they overlap, a stationary solid O OBJ There exists, and detection area A of radar device H11 L Pedestrian O PED There exists. In Figure 3, the stationary solid O OBJ If the camera can identify the object as a stationary object, in process S106, the coordinate origin is fixed and the X position, Y position, and width of the stationary object are recorded. The above process is performed when the camera can detect and identify the stationary object, and in Figure 3, the coordinates of the stationary object at the time immediately before the camera can no longer detect and identify the stationary object are recorded as (X'OBJ (t0), Y' OBJ (t0)), with a width of W OBJ and store it as such.

[0035] Figure 4 is an overhead view of the host vehicle and surrounding objects at a certain time t1 after a certain period has elapsed from time t0. In Figure 4, the detection area A of the camera device H12 CAM outside the detection area A of the radar device H11 L contains a stationary solid object O OBJ and a pedestrian O PED existing therein.

[0036] Figure 5 shows the radar detection point cloud viewed on the coordinates fixed at the time of Figure 4. In Figure 5, in process S110, the position of the stationary solid object recorded in process S106 (X' OBJ (t0), Y' OBJ (t0)) is centered, and all radar detection points within a circle with a diameter of W OBJ are given the attribute of being outside the grouping target. Next, in process S112, grouping processing is performed on radar detection points other than those with the outside-the-grouping-target attribute.

[0037] Figure 6 is an overhead view of the host vehicle and surrounding objects at a certain time t2 after a certain period has elapsed from time t1. In Figure 6, the detection area A of the camera device H12 CAM outside the detection area A of the radar device H11 L contains a stationary solid object O OBJ and a pedestrian O PED existing therein. Figure 6 shows an example scene where it is determined in process S109 that the host vehicle has passed by the stationary solid object recorded in process S106. The coordinates of the stationary solid object recorded in process S106 (X' OBJ (t0), Y' OBJ (t0)) are converted to coordinates (X'' OBJ (t0), Y'' OBJ (t0)) with the vehicle longitudinal direction as the X-axis, the vehicle lateral direction as the Y-axis, the center point of the front end of the vehicle body as the origin, and the vehicle forward direction as 0 degrees and positive counterclockwise with respect to the angle. At this time, the overall length of the vehicle is L OVERALLWhen equation (1) is true, it is determined that the vehicle has passed the stationary object.

number

[0038] In the description of Example 1 above, to avoid making the diagrams cluttered, only one stationary object is shown. However, if multiple stationary objects exist, the process in Example 1 is performed on all stationary objects detected by the camera.

[0039] In this embodiment, by assigning the attribute "not to be grouped" to stationary objects detected by the camera, it is possible to detect stationary objects and nearby moving objects such as pedestrians separately (as different targets) in the radar detection area outside the camera's detection area, without mistakenly grouping them together. Furthermore, although only one radar device is described in this embodiment, it can also be applied with multiple radar devices.

[0040] [Example 2] Next, Embodiment 2 of the present invention will be described with reference to Figures 7 to 12. In Embodiment 1, the pedestrian was at a distance from the object and moved closer to it. In Embodiment 2, however, the process will be explained using an example where the object and the pedestrian are already in close proximity when detected by the camera and radar devices. Note that the same parts and processes in Embodiment 2 and Embodiment 1 are denoted by the same reference numerals, and their descriptions will not be repeated.

[0041] In this embodiment, the vehicle coordinate system is defined with the vehicle's front-to-back (length) direction as the X-axis, the vehicle's left-to-right (width) direction as the Y-axis, the center point of the front end of the vehicle body as the origin, and angles set to 0 degrees in the forward direction of the vehicle and positive in a counterclockwise direction.

[0042] Figure 7 shows a flowchart of the fusion program processing performed by the controller H3 in Example 2.

[0043] Since the initial steps S101 through S108 are the same as in Example 1, the explanation will be omitted.

[0044] Next, in process S201, it is determined whether the area, height, or width of the stationary three-dimensional object detected by the camera in process S106 is greater than or equal to a threshold.

[0045] If the camera determines that the area, height, or width of a stationary object is greater than or equal to a threshold (process S201: Yes), the azimuth angle of the stationary object detected by the camera is calculated in process S202. The azimuth angle is calculated with the camera's installation position as the origin.

[0046] Next, in process S203, a dummy target is placed at a certain distance away from the position of the stationary object detected by the camera, relative to the azimuth angle obtained in process S202. This is equivalent to assuming that the camera has not detected a target in the blind spot of the stationary object, that is, that the target may be in the camera occlusion area of ​​the stationary object.

[0047] Next, in process S204, the dummy target set in process S203 is retained as a radar grouping object.

[0048] Next, process S109 is performed. If the result of process S109 is No, process S110 is performed. However, since processes S109 and S110 are the same as in Example 1, the explanation is omitted.

[0049] On the other hand, if it is determined that the vehicle has passed a stationary object (process S109: Yes), process S111 is performed, and then in process S205, the dummy target is removed from the radar's grouping of objects. Process S111 is the same as in Example 1, so its explanation is omitted.

[0050] Next, process S112 is performed, but since it is the same as in Example 1, the explanation will be omitted.

[0051] Next, in process S206, it is determined whether or not the grouped object obtained in process S112 exists near the dummy target.

[0052] If it is determined that a grouping object obtained in process S112 exists near the dummy target (process S206: Yes), the dummy target is removed from the radar's grouping objects in process S205, and the process is terminated.

[0053] On the other hand, if it is determined that there are no grouped objects obtained in process S112 near the dummy target (process S206: No), the process is terminated.

[0054] A diagram illustrates a scene to explain the process of Example 2 in more detail. As an example, we will describe a scene in chronological order in which, before the vehicle enters an intersection, a stationary object and a pedestrian are located near the corner of the intersection, and the pedestrian approaches the vehicle as it turns around the intersection. Figure 8 is an overhead view of the vehicle and its surrounding objects at a certain time t3. In Figure 8, the detection area A of the radar device H11 is shown. L and detection area A of camera device H12 CAM In the region where they overlap, a stationary solid O OBJ and pedestrian O PED There is a pedestrian O PED O is a stationary solid object OBJ Because it is located in the camera occlusion region, the camera device H12 can see the pedestrian O PED It is assumed that it has not been detected. In Figure 8, the stationary three-dimensional object O OBJ If the camera can identify the object as a stationary object, in process S106, the coordinate origin is fixed and the X position, Y position, and width of the stationary object are recorded. The above process is performed when the camera can detect and identify the stationary object, and in Figure 8, the coordinates of the stationary object at the time immediately before the camera can no longer detect and identify the stationary object are recorded as (X' OBJ (t3), Y' OBJ (t3), width W OBJ , height H OBJ Remember it as such.

[0055] Figure 9 is an overhead view of the vehicle and its surrounding objects at time t3. To avoid making the diagram too cluttered, pedestrians O, which were not detected by the camera, are omitted.PED Delete the stationary object O OBJ This figure shows a dummy target placed in the camera occlusion region. The dummy target placement process will now be explained. First, in process S201, it is determined whether the area, height, or width of a stationary object is greater than or equal to a threshold value based on equations (2), (3), and (4). The threshold values ​​for area, height, and width are S TH H TH , W TH Let's assume that.

number

number

number

[0056] Next, if any of equations (2), (3), or (4) is true, in process S202, based on equation (5), a stationary three-dimensional object O with the camera installation position as the origin is generated. OBJ Azimuth angle θ OBJ Calculate (t3).

number

[0057] Next, in process S203, the position (X') of the stationary object is determined. OBJ (t3), Y' OBJ (t3)) from the azimuth angle θ OBJ Distance L in the (t3) direction O Distant position (X' DUM (t3), Y' DUM A dummy target is placed at (t3). As mentioned above, radar grouping is the process of treating nearby radar detection points as a single object, and the grouping range is generally determined by a circle. Therefore, L O W is the width of the stationary object detected by the camera. OBJ and the diameter R of the radar grouping range DUM It is calculated using equation (6).

number

[0058] Also, X' DUM (t3), Y' DUM (t3) is calculated using equations (7) and (8), respectively.

number

number

[0059] Next, in process S204, it is assumed that the dummy target has been detected by the radar and is designated as a radar grouping object.

[0060] Figure 10 is an overhead view of the vehicle and its surrounding objects at a certain time t4, after a certain time has elapsed from time t3. In Figure 10, the detection area A of camera device H12 is shown. CAM Detection area A of the external radar device H11 L In a stationary solid object O OBJ and pedestrian O PED and dummy target O DUM It exists.

[0061] Figure 11 shows the radar-detected point cloud as seen on the fixed coordinates in Figure 10. In Figure 11, in processing S110, the position (X') of the stationary object recorded in processing S106 is shown. OBJ (t3), Y' OBJ (t3)) centered on diameter W OBJ All radar detection points within the circle are assigned the attribute that they are not to be grouped. Next, in process S112, grouping is performed on radar detection points that do not have the "not to be grouped" attribute. Next, in process S206, the position of the dummy target obtained in process S203 (X' DUM (t3), Y' DUM (t3)) centered on diameter R DUMIf a grouped object obtained in process S112 exists within the circle, it is determined that a dummy target actually exists, and the dummy target is deleted in process S205.

[0062] Figure 12 is an overhead view of the vehicle and its surrounding objects at a certain time t5, after a certain time has elapsed from time t4. In Figure 12, the detection area A of camera device H12 is shown. CAM Detection area A of the external radar device H11 L In a stationary solid object O OBJ and pedestrian O PED Such a thing exists. Figure 12, like Figure 6, shows an example scene where it is determined in process S109 that the vehicle has passed the stationary object recorded in process S106, so a detailed explanation of the process is omitted.

[0063] In the description of Example 2 above, to avoid making the diagrams cluttered, only one stationary object is shown. However, if multiple stationary objects exist, the process in Example 2 is performed on all stationary objects detected by the camera.

[0064] In this embodiment, by assigning the attribute "not for grouping" to stationary objects detected by the camera, and by assuming that a target exists in the camera occlusion region (the area near the object where millimeter-wave radar detection becomes difficult due to the object) when the stationary object is large, it is possible to detect stationary objects and nearby moving objects such as pedestrians separately (as different targets) in the radar detection area outside the camera's detection area, without mistakenly grouping them together. Furthermore, although only one radar device is described in this embodiment, it is also applicable to multiple radar devices.

[0065] [Example 3] Next, Embodiment 3 of the present invention will be described with reference to Figures 13 to 18. As a different scenario from Embodiments 1 and 2, the process will be explained using an example where a moving object such as a pedestrian is located between a stationary object and the vehicle, and the stationary object cannot be detected by the camera. Note that the same parts and processes in Embodiment 3 and Embodiments 1 and 2 are denoted by the same reference numerals, and their descriptions will not be repeated.

[0066] In this embodiment, the vehicle coordinate system is defined with the vehicle's front-to-back (length) direction as the X-axis, the vehicle's left-to-right (width) direction as the Y-axis, the center point of the front end of the vehicle body as the origin, and angles set to 0 degrees in the forward direction of the vehicle and positive in a counterclockwise direction.

[0067] Figure 13 shows a flowchart of the fusion program processing performed by the controller H3 in Example 3.

[0068] Since the initial steps S101 through S104 are the same as in Example 1, the explanation will be omitted.

[0069] Next, in process S301, it is determined whether or not a pedestrian (or two-wheeled vehicle) is present in the camera detection results obtained in process S104.

[0070] If the camera detects the presence of a pedestrian (or two-wheeled vehicle) (process S301: Yes), process S302 calculates the azimuth angle of the pedestrian (or two-wheeled vehicle) detected by the camera. The azimuth angle is calculated with the camera's installation position as the origin.

[0071] Next, in process S303, a dummy target is placed at a certain distance away from the pedestrian (or motorcycle) detected by the camera, relative to the azimuth angle obtained in process S302. This is equivalent to assuming that the camera has not detected a target in the pedestrian's (or motorcycle's) blind spot, meaning that the target may be within the camera's occlusion area.

[0072] Next, in process S304, the dummy targets placed in process S303 are held as radar grouping objects.

[0073] Next, in S305, the coordinate origin is fixed and the position of the dummy target is stored.

[0074] Next, process S108 is performed, but since process S108 is the same as in Example 1, the explanation will be omitted.

[0075] Next, in process S306, it is determined whether the vehicle has passed the dummy target obtained in process S305.

[0076] If the vehicle has not passed the dummy target (process S306: No), in process S307, radar detection points near the dummy target are excluded from grouping.

[0077] On the other hand, if it is determined that the vehicle has passed the dummy target (process S306: Yes), process S111 is performed, followed by process S205. Process S111 is the same as in Example 1, and process S205 is the same as in Example 2, so their explanations are omitted.

[0078] On the other hand, if process S301 determines that there are no pedestrians (or two-wheeled vehicles) detected by the camera (process S301: No), process S107 is performed. Process S107 is the same as in Example 1, so its explanation is omitted.

[0079] Next, process S112 is performed, but since it is the same as in Example 1, the explanation will be omitted.

[0080] The subsequent processes S206 and S205 are the same as in Example 2, so their explanation will be omitted.

[0081] This completes the process.

[0082] A scene example illustrating the processing of Embodiment 3 in more detail is shown in the figure. As an example, a scene is described in chronological order in which, before the vehicle enters an intersection, a pedestrian is moving on the sidewalk to the left front of the vehicle in the same direction as the vehicle's movement, and a three-dimensional object at the corner of the intersection remains within the pedestrian's camera occlusion area until it moves out of the camera's detection area. Figure 14 is an overhead view of the vehicle and its surrounding objects at a certain time t6. In Figure 14, detection area A of radar device H11 is shown. L and detection area A of camera device H12 CAM In the region where they overlap, a stationary solid O OBJ and pedestrian O PED There exists a stationary solid O. OBJ Pedestrian O PED Because it is located in the camera occlusion region, the stationary object O OBJ It is assumed that it has not been detected.

[0083] Figure 15 is an overhead view of the vehicle and its surrounding objects at time t6. To prevent the diagram from becoming cluttered, stationary objects O that were not detected by the camera are omitted. OBJ Delete and pedestrian O PED This is a diagram showing a dummy target placed in the camera occlusion region. The dummy target placement process will be explained. In Figure 15, pedestrian O PED If the camera can identify the person as a pedestrian, in process S302, based on equation (9), the pedestrian O is defined with the camera installation position as the origin. PED Azimuth angle θ PED Calculate (t6).

number

[0084] Next, in process S303, the pedestrian's position (X' PED (t6), Y' PED (t6)) from the azimuth angle θ PED (t6) distance L P Distant position (X' DUM (t6), Y' DUM Place a dummy target at (t6).P is the width W of the pedestrian detected by the camera in the same manner as in process S203 of Example 2 PED and the diameter R of the grouping range of the radar DUM which is calculated by equation (10). [Number]

[0085] Also, X' DUM (t6), Y' DUM (t6) are calculated by equations (11) and (12) respectively. [Number] [Number]

[0086] Next, in process S304, assume that a dummy target is detected by the radar and set it as the grouped object of the radar.

[0087] Next, in process S305, fix the coordinate origin and record the X position and Y position of the dummy target. The above-described process is performed when the pedestrian can be detected and identified by the camera. In FIG. 15, the coordinates of the dummy target at the time immediately before the pedestrian cannot be detected and identified by the camera are recorded as (X' DUM (t6), Y' DUM (t6)).

[0088] FIG. 16 is an overhead view of the host vehicle and surrounding objects at a certain time t7 after a certain time has elapsed from time t6. In FIG. 16, a stationary solid object O CAM and a pedestrian O L and a dummy target O OBJ exist in the detection area A PED of the radar device H11 outside the detection area A DUM of the camera device H12.

[0089] Figure 17 shows the radar detection point cloud as seen on the fixed coordinates in Figure 16. In Figure 17, in process S307, the position (X') of the dummy target recorded in process S305 is shown. DUM (t6), Y' DUM (t6)) centered on diameter R DUM All radar detection points within the circle are assigned the attribute "not to be grouped". Next, in process S112, grouping is performed on radar detection points that do not have the "not to be grouped" attribute. Next, in process S206, the position of the dummy target obtained in process S305 (X' DUM (t6), Y' DUM (t6)) centered on diameter R DUM If a grouped object obtained in process S112 exists within the circle, it is determined that a dummy target actually exists, and the dummy target is deleted in process S205.

[0090] Figure 18 is an overhead view of the vehicle and its surrounding objects at a certain time t8, after a certain time has elapsed from time t7. In Figure 18, the detection area A of camera device H12 is shown. CAM Detection area A of the external radar device H11 L In a stationary solid object O OBJ and pedestrian O PED There is a mechanism in place. Figure 18 shows an example of a scenario in which, similar to Figure 6, it is determined in process S306 that the vehicle has passed the dummy target recorded in process S305, so a detailed explanation of the process is omitted.

[0091] In the description of Example 3 above, to avoid making the diagrams cluttered, only one stationary object is shown. However, if multiple stationary objects exist, the process in Example 3 is performed on all stationary objects detected by the camera.

[0092] In this embodiment, if a moving object such as a pedestrian exists between the vehicle and a stationary object, and the stationary object is occluded by the camera and cannot be detected by the camera, a dummy target is placed in the camera occlusion area of ​​the pedestrian (the area near the moving object that is obscured by the moving object), and the dummy target is treated as a stationary object and given the attribute that it is not subject to grouping. This makes it possible to detect the stationary object and the nearby moving object such as a pedestrian separately (as different targets) without mistakenly grouping them in the radar detection area outside the camera's detection area. Furthermore, although only one radar device is described in this embodiment, it is also applicable with multiple radar devices.

[0093] As described above, the controller (vehicle control device) H3 of this embodiment has a camera that recognizes three-dimensional objects around the vehicle and a millimeter-wave radar that recognizes targets around the vehicle. Based on the information of the three-dimensional object recognized by the camera, the controller sets the three-dimensional object as an independent target (by assigning the attribute to the three-dimensional object that it is not subject to grouping by the millimeter-wave radar), and when the millimeter-wave radar recognizes the three-dimensional object and other targets as surrounding targets, it recognizes the three-dimensional object as a target different from the other targets.

[0094] Furthermore, a first virtual target (dummy target) is set in the vicinity of the three-dimensional object where detection by the millimeter-wave radar becomes difficult due to the three-dimensional object. (By assigning attributes to the three-dimensional object that exclude it from grouping (by the millimeter-wave radar), if the first virtual target is detected by the millimeter-wave radar due to the movement of the vehicle, the three-dimensional object and the first virtual target are recognized as different targets.

[0095] Furthermore, the first virtual target is placed if at least one of the area, width, and height of the three-dimensional object is greater than or equal to a predetermined value (threshold), and the first virtual target is not placed if it is less than the predetermined value (threshold).

[0096] Furthermore, when the camera recognizes a moving target, a second virtual target (dummy target) is placed in the vicinity of the moving target that is obscured by the moving target. (By assigning the second virtual target an attribute that excludes it from grouping (by the millimeter-wave radar),) if the second virtual target is detected by the millimeter-wave radar due to the movement of the vehicle, the moving target and the second virtual target are recognized as different targets.

[0097] In other words, this embodiment provides a vehicle control device that enables high-precision sensing by targeting a vehicle equipped with an external information acquisition device consisting of a millimeter-wave radar and a camera that acquires information such as the vehicle's position and driving environment, and in scenes where surrounding obstacles located in front of, to the side or to the front of the vehicle are detected, objects detected as arbitrary three-dimensional objects by the camera are excluded from grouping, and fusion is performed based on the grouping results of radar detection points that are not excluded (Embodiment 1); in addition to the above processing, dummy objects are placed in the camera occlusion area of ​​the arbitrary three-dimensional object detected by the camera according to the height, width, area, etc. of the object and fusion is performed (Embodiment 2); and when an object is detected as a pedestrian or two-wheeled vehicle by the camera, a dummy object is placed in its camera occlusion area, the dummy object is excluded from grouping, and fusion is performed based on the grouping results of radar detection points that are not excluded (Embodiment 3).

[0098] According to this embodiment, a vehicle control device capable of suppressing misgrouping of millimeter-wave radar can be provided.

[0099] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are explained in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described.

[0100] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software by having the processor interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD. [Explanation of symbols]

[0101] H1 External world information acquisition department H11 Radar system (millimeter-wave radar) H12 Camera device (camera) H2 Vehicle Information Acquisition Unit H3 Controller (Vehicle Control System) H4 Vehicle Control Unit

Claims

1. It has a camera that recognizes stationary three-dimensional objects around the vehicle, and a millimeter-wave radar that recognizes targets around the vehicle, When the camera can no longer recognize the stationary object due to the movement of the vehicle, the camera sets the stationary object as an independent target based on the information of the stationary object that the camera recognized before it could no longer recognize the stationary object, and when the millimeter-wave radar recognizes the stationary object and other targets as surrounding targets, it recognizes the stationary object as a target different from the other targets. A first virtual target is set in the vicinity of the stationary object in which detection by the millimeter-wave radar becomes difficult due to the stationary object. A vehicle control device characterized in that, when the first virtual target is detected by the millimeter-wave radar due to the movement of the vehicle, the stationary object and the first virtual target are recognized as different targets.

2. It has a camera that recognizes stationary three-dimensional objects around the vehicle, and a millimeter-wave radar that recognizes targets around the vehicle, When the camera can no longer recognize the stationary object due to the movement of the vehicle, the camera sets the stationary object as an independent target based on the information of the stationary object that the camera recognized before it could no longer recognize the stationary object, and when the millimeter-wave radar recognizes the stationary object and other targets as surrounding targets, it recognizes the stationary object as a target different from the other targets. When the camera recognizes a moving object, a second virtual object is placed in the vicinity of the moving object that is obscured by the moving object. A vehicle control device characterized in that, when the second virtual target is detected by the millimeter-wave radar due to the movement of the vehicle, the moving target and the second virtual target are recognized as different targets.

3. A vehicle control device according to claim 1, A vehicle control device characterized in that it places the first virtual target when at least one of the area, width, and height of the stationary three-dimensional object is greater than or equal to a predetermined value, and does not place the first virtual target when it is less than the predetermined value.

4. A vehicle control device according to claim 1 or 2, A vehicle control device characterized by recognizing the stationary object as a different target from the other targets by assigning an attribute to the stationary object that excludes it from grouping.

5. A vehicle control device according to claim 1, A vehicle control device characterized by recognizing the stationary object and the first virtual target as different targets by assigning an attribute to the stationary object that excludes it from grouping.

6. A vehicle control device according to claim 2, A vehicle control device characterized by recognizing the moving target and the second virtual target as different targets by assigning an attribute to the second virtual target that excludes it from grouping.

Images