Control apparatus for vehicle

Abstract

To provide a control apparatus for a vehicle capable of reducing a processing load in image recognition by narrowing down, in advance, a range where a target is likely to exist based on point group data in a radar.SOLUTION: A control apparatus for a vehicle according to the present disclosure includes a camera that captures an image of surroundings of a vehicle, and a radar that recognizes a reflection point of reflection from a target around the vehicle, and the control apparatus for a vehicle includes: a setting unit that sets a grid on a reference plane; a generation unit that generates a quadrangular prism according to height at which the reflection point is present in the grid in which a predetermined number or more of the reflection points detected by the radar are located; and an image processing unit that performs image processing on a region corresponding to the quadrangular prism in an image captured by the camera.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device. [Background technology]

[0002] A technology has been developed that reduces erroneous target recognition by integrating target recognition results using point cloud data from radar such as LiDAR (Light Detection And Ranging) with target recognition results from a camera (for example, using only target recognition results recognized by both a camera and LiDAR). Patent Document 1 also discloses a technology that matches point cloud data acquired from detection signals from an omnidirectional lidar with high-precision map data (point cloud data), which is data on a high-precision map, to estimate the position of a vehicle.

[0003] Here, the point cloud recognition process for recognizing targets from point cloud data includes ground removal, which removes point clouds due to reflections from the ground from the point cloud data; clustering, which groups point clouds based on points that are close to each other; boxing, which fits a rectangular prism to the shape of the grouped point clouds; and tracking, which monitors targets detected in the process up to boxing and calculates the relative speed of the targets. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-32069 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the above technology, image processing is performed on the camera's image capture results even when a target object does not exist, which places a heavy processing load on the system. There is also a method for determining the type of target object through image processing based on the position of the target object recognition result using LiDAR, but this increases processing time because image processing is performed on the camera's image capture results after the target recognition process using LiDAR is complete. If the target object cannot be recognized by LiDAR, image processing is not performed on the camera's image capture results, but target recognition processing using LiDAR is performed, which increases processing time.

[0006] The present invention has been made in consideration of the above, and aims to provide a vehicle control device that can reduce the processing load in image recognition by narrowing down the range in which targets are likely to be located in advance based on point cloud data from radar. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the vehicle control device of the present invention is a vehicle control device that includes a camera that photographs the area around the vehicle and a radar that recognizes reflection points from targets around the vehicle, and is equipped with a setting unit that sets a grid on a reference surface, a generation unit that generates a rectangular prism corresponding to the height at which the reflection points detected by the radar are located in the grid where a predetermined number or more of the reflection points are located, and an image processing unit that performs image processing on an area of ​​the image photographed by the camera that corresponds to the rectangular prism.

[0008] According to this configuration, the processing load in image recognition can be reduced by narrowing down the range where targets are likely to be located in advance based on the point cloud data from the radar.

[0009] In the vehicle control device according to the present invention, the image processing unit does not perform image processing on the region in which the number of reflection points is less than the predetermined number.

[0010] According to this configuration, clusters with less than a predetermined number of reflection points are highly likely to be noise that does not require image processing, and it is possible to avoid image processing of such clusters, thereby reducing the processing load. [Effects of the Invention]

[0011] The vehicle control device according to the present invention has the advantage of being able to reduce the processing load in image recognition by narrowing down in advance the range in which a target is likely to be located based on point cloud data from a radar. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of a system configuration of a vehicle equipped with a vehicle control device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the functional configuration of the autonomous driving ECU of the vehicle according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a target recognition process performed by the autonomous driving ECU of the vehicle according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The configuration of the embodiment described below and the actions and effects brought about by the configuration are merely examples, and the present invention is not limited to the following description.

[0014] 1 is a block diagram showing an example of a system configuration of a vehicle equipped with a vehicle control device according to an embodiment. The vehicle 1 is equipped with an automatic driving function and is capable of traveling by automatic driving without the need for driving operations by a user (driver). Note that automatic driving includes semi-automatic driving in which some of the operations for traveling of the vehicle 1 are automated (requiring partial driving operations by the user).

[0015] A plurality of ECUs (Electronic Control Units) are mounted on the vehicle 1 to control various parts. Each ECU has a microcontroller unit (microcomputer), and the microcomputer has built-in, for example, a CPU (Central Processing Unit), a non-volatile memory such as a flash memory, and a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0016] The multiple ECUs include a drive ECU 11, a steering ECU 12, a brake ECU 13, a meter ECU 14, and a body ECU 15. The drive ECU 11, the steering ECU 12, the brake ECU 13, the meter ECU 14, and the body ECU 15 are connected to each other so as to be able to communicate using a CAN (Controller Area Network) communication protocol, that is, to communicate via CAN.

[0017] The drive ECU 11 is a control unit that controls a drive unit 21 of the vehicle 1. The drive unit 21 may be configured to include an engine as a drive source, a motor as a drive source, or both an engine and a motor as drive sources. The drive unit 21 includes a transmission that changes the speed of the drive force from the drive source and outputs it as needed.

[0018] The steering ECU 12 is a control unit that controls a steering device 22 of the vehicle 1. The steering device 22 is, for example, an electric power steering device that applies torque from an electric motor to a steering mechanism. The steering mechanism includes, for example, a rack-and-pinion steering gear, and is configured so that when a rack shaft moves in the vehicle width direction due to the torque of the electric motor, the left and right steered wheels are turned left and right in accordance with the movement of the rack shaft.

[0019] The brake ECU 13 is a control unit that controls a braking device 23 of the vehicle 1. The braking device 23 may be hydraulic or electric. The hydraulic braking device 23 includes a brake actuator, and the function of this brake actuator distributes hydraulic pressure to wheel cylinders of the brakes provided on each wheel, and the hydraulic pressure applies braking force from each brake to the wheels, including the drive wheels.

[0020] The meter ECU 14 is a control unit that controls each part of a meter panel (not shown) of the vehicle 1. The meter panel is provided with indicators such as a liquid crystal display for displaying various information, as well as instruments that display vehicle speed and engine RPM. An emergency stop switch 24 that is operated to issue an emergency stop command for the autonomous driving is also connected to the meter ECU 14.

[0021] The body ECU 15 is a control unit that controls various parts that need to operate even when the ignition switch of the vehicle 1 is off, such as the left and right turn signals and door lock motors.

[0022] The plurality of ECUs also include an autonomous driving ECU 31, a lidar ECU 32, and a monocular camera ECU 33 as control units for the autonomous driving function.

[0023] The autonomous driving ECU 31 is a control center for autonomous driving control. The autonomous driving ECU 31 is an example of a vehicle control device. The autonomous driving ECU 31 is connected to the drive ECU 11, the steering ECU 12, the brake ECU 13, the meter ECU 14, and the body ECU 15 so as to be able to communicate via CAN.

[0024] An omnidirectional LiDAR (Light Detection and Ranging) 34 is connected to the autonomous driving ECU 31 via, for example, an Ethernet (registered trademark) communication cable. The omnidirectional LiDAR 34 is an example of a radar that recognizes reflection points of reflections from targets around the vehicle 1. Here, the targets are objects present around the vehicle 1, including vehicles, pedestrians, buildings, and obstacles such as curbs. In this embodiment, the omnidirectional LiDAR 34 is capable of acquiring ranging information indicating the distances from the vehicle 1 to targets present around the vehicle 1. The omnidirectional LiDAR 34 emits laser light in all directions (360°), receives reflected light from targets present within a search range with an optical sensor, and outputs a detection signal corresponding to the reflected light as ranging information. The ranging information may take the form of, for example, point cloud data, which is a collection of distances from the vehicle 1 to each of multiple reflection points of the targets at each position (voxel) in three-dimensional space. The autonomous driving ECU 31 receives a detection signal from the omnidirectional LiDAR 34. In this embodiment, a lidar is used as an example of a radar, but the present invention is not limited to this, and a millimeter wave radar or the like may also be used as an example of a radar.

[0025] Furthermore, a GPS receiver 35 is connected to the autonomous driving ECU 31 via, for example, a USB (Universal Serial Bus) standard communication cable. The GPS receiver 35 is a receiver that receives positioning signals from GPS (Global Positioning System) satellites. The GPS receiver 35 is capable of acquiring positioning information indicating a position on Earth (e.g., latitude, longitude, and altitude). The positioning signals received by the GPS receiver 35 are input from the GPS receiver 35 to the autonomous driving ECU 31 as positioning information.

[0026] The LIDAR ECU 32 is communicatively connected to the autonomous driving ECU 31 via, for example, an Ethernet communication cable. Six LIDARs 36 are connected to the LIDAR ECU 32. Each LIDAR 36 emits laser light into a search range around the vehicle 1, receives reflected light from targets within the search range using an optical sensor, and outputs a detection signal corresponding to the reflected light. The LIDARs 36 are disposed, for example, at the left, center, and right ends of the front bumper and the left, center, and right ends of the rear bumper of the vehicle 1. The LIDARs 36 are an example of a distance measurement sensor. The LIDAR ECU 32 receives detection signals output from each LIDAR 36. The LIDAR ECU 32 processes the detection signals output from each LIDAR 36 and transmits data obtained by the processing to the autonomous driving ECU 31. In this embodiment, an example will be described in which the omnidirectional LIDAR 34 functions as an example of a radar, but the LIDAR 36 may also function as an example of a radar.

[0027] The monocular camera ECU 33 is communicatively connected to the autonomous driving ECU 31 via, for example, a USB standard communication cable. A monocular camera 37 is connected to the monocular camera ECU 33. The monocular camera 37 is an example of a camera that captures images of the surroundings of the vehicle. In this embodiment, the monocular camera 37 is a camera that can continuously capture still images of a search range ahead of the vehicle 1 at a predetermined frame rate. Image signals of still images continuously output from the monocular camera 37 are input to the monocular camera ECU 33. The monocular camera ECU 33 processes the image signals input from the monocular camera 37 and transmits image data obtained by this processing to the autonomous driving ECU 31.

[0028] Fig. 2 is a block diagram showing an example of the functional configuration of an autonomous driving ECU of a vehicle according to this embodiment. The autonomous driving ECU 31 of this embodiment includes an image recognition processing unit 41, a ground surface removal unit 42, an image range determination unit 43, a point cloud recognition processing unit 44, and an integration processing unit 45. These functional units 41 to 45 are configured by cooperation between hardware and software (programs, etc.) that configure the vehicle 1 as shown in Fig. 1. Furthermore, at least one of these functional units 41 to 45 may be configured by dedicated hardware (circuits, etc.). However, the functional configuration of the autonomous driving ECU 31 is not limited to this.

[0029] The ground surface removal unit 42 is an example of a setting unit that sets a grid on a reference plane around the vehicle 1. Here, the reference plane may be a horizontal plane parallel to the horizontal direction with the position of the vehicle 1 as a reference, or may be the road surface on which the vehicle 1 is located. In addition, the ground surface removal unit 42 acquires point cloud data from the omnidirectional LIDAR 34, and removes point cloud data due to reflection from the ground surface (ground point cloud data) from the acquired point cloud data.

[0030] Then, the ground removal unit 42 notifies the image range determination unit 43 and the point cloud recognition processing unit 44 of the point cloud data obtained by excluding the ground point cloud data, i.e., the point cloud data (target point cloud data) indicating the distance from the vehicle 1 to each of multiple reflection points of reflections from targets other than the ground.

[0031] The image range determination unit 43 uses the target point cloud data to determine a target existence range (image range) in which targets exist within the surroundings of the vehicle 1 (within the image (hereinafter referred to as the captured image) captured by the monocular camera 37). Then, the image range determination unit 43 notifies the image recognition processing unit 41 of the determined target existence range.

[0032] The image recognition processing unit 41 identifies grids where a predetermined number or more of reflection points detected by the omnidirectional LIDAR 34 are located, among the grids set by the ground surface removal unit 42. Next, the image recognition processing unit 41 functions as an example of a generating unit that generates a quadrangular prism (e.g., a cube) according to the height at which the reflection points are located in the identified grid. Specifically, the image recognition processing unit 41 calculates the maximum height of the point cloud data of the identified grid, and generates a quadrangular prism of the calculated maximum height.

[0033] Furthermore, the image recognition processing unit 41 integrates the rectangular prisms of adjacent grids. This allows for simple clustering processing with a low load. Alternatively, the image recognition processing unit 41 may integrate grids that exist within a predetermined range (for example, within three grids) based on the grid where the rectangular prism was generated.

[0034] The image recognition processing unit 41 functions as an example of an image processing unit that performs image processing on a region corresponding to a rectangular prism in the image captured by the monocular camera 37. This makes it possible to determine in advance whether or not a target exists using point cloud data, thereby eliminating unnecessary image processing when no target exists and reducing the processing load of image recognition. Furthermore, since it is possible to narrow down the range in which a target may exist by simple processing using only the number of points in a grid or point height information, it becomes possible to perform parallel processing of point cloud recognition processing and image processing after narrowing down the range of the target, thereby reducing processing time.

[0035] In this embodiment, the image recognition processing unit 41 executes image processing to recognize a target from an area corresponding to a rectangular prism in the captured image, and notifies the integration processing unit 45 of the target recognition result (target recognition result (1)) obtained by the image processing. Furthermore, in this embodiment, the image recognition processing unit 41 executes the target recognition processing by image processing in parallel with the target recognition processing from target point cloud data by the point cloud recognition processing unit 44, which will be described later.

[0036] The image recognition processing unit 41 does not perform image processing on areas of the captured image that correspond to rectangular prisms and have less than a predetermined number of point cloud data (i.e., reflection points). This makes it possible to remove noise contained in the captured image. For example, the image recognition processing unit 41 deletes rectangular prisms in areas of the captured image that correspond to rectangular prisms and have less than a predetermined number of point cloud data. Alternatively, the image recognition processing unit 41 may delete rectangular prisms whose height is equal to or less than a preset height threshold. Furthermore, the image recognition processing unit 41 does not perform image processing on areas of the captured image where no rectangular prisms exist.

[0037] The point cloud recognition processing unit 44 recognizes targets existing around the vehicle 1 from the target point cloud data. Then, the point cloud recognition processing unit 44 notifies the integration processing unit 45 of the target recognition result (target recognition result (2)) from the target point cloud data. In this embodiment, the point cloud recognition processing unit 44 executes the target recognition process from the target point cloud data in parallel with the image processing by the image recognition processing unit 41.

[0038] The integration processing unit 45 receives the target recognition result (1) and the target recognition result (2). The integration processing unit 45 then outputs only targets that are recognized in both the target recognition result (1) and the target recognition result (2) as the target recognition result (3). This reduces erroneous target recognition.

[0039] 3 is a diagram illustrating an example of a target recognition process performed by the autonomous driving ECU of a vehicle according to an embodiment. In this embodiment, the ground surface deletion unit 42 first generates target point cloud data (e.g., point cloud data of pedestrians) by deleting ground surface point cloud data from point cloud data acquired from the omnidirectional lidar 34. The ground surface deletion unit 42 also divides a reference plane (XY plane) such as a horizontal plane into 1 m square sections (grids). Next, the image range determination unit 43 determines a target presence region in which a target exists in the captured image based on the target point cloud data.

[0040] The image recognition processing unit 41 calculates the number of points (reflection points) in each grid based on the target point cloud data. Next, the image recognition processing unit 41 generates a rectangular prism corresponding to the height of the target for a grid where a predetermined number of reflection points or more are located. Furthermore, the image recognition processing unit 41 integrates the generated rectangular prisms that exist in adjacent grids into a single rectangular prism. At this time, if the heights of the rectangular prisms in the adjacent grids are different, the image recognition processing unit 41 changes the rectangular prism with the shorter height to the rectangular prism with the greatest height and integrates them. In other words, it is sufficient that the rectangular prism is set within a range that includes the target. Furthermore, the image recognition processing unit 41 discards rectangular prisms with a number of reflection points less than a predetermined number.

[0041] As described above, according to the vehicle 1 of this embodiment, by using point cloud data to determine in advance whether or not a target exists, it is possible to eliminate unnecessary image processing when no target exists, thereby reducing the processing load of image recognition. Also, since it is possible to narrow down the range in which a target may exist by simple processing using only the number of points in a grid or point height information, it is possible to perform parallel processing of the point cloud recognition process and image processing after narrowing down the range of the target, thereby reducing processing time. [Explanation of symbols]

[0042] 1 vehicle 31 Autonomous Driving ECU 34 Omnidirectional Lidar 37 Monocular Camera 41 Image recognition processing section 42 Ground removal section 43 Image range determination unit 44 Point cloud recognition processing section 45 Integrated Processing Unit

Claims

1. A vehicle control device including a camera that captures images of the surroundings of a vehicle and a radar that recognizes reflection points of reflections from targets around the vehicle, a setting unit for setting a grid on a reference surface; a generation unit that generates a rectangular prism corresponding to a height at which a predetermined number or more of the reflection points detected by the radar are located in the grid; an image processing unit that performs image processing on a region of the image captured by the camera that corresponds to the rectangular prism; A vehicle control device comprising:

2. The vehicle control device according to claim 1 , wherein the image processing unit does not perform image processing on the region in which the number of reflection points is less than the predetermined number.

Images