External perception device

Abstract

To reduce the processing load required to recognize the external environment surrounding the vehicle. [Solution] The external environment recognition device 50 includes an on-board detector 5, a recognition unit 111 that recognizes the road surface and three-dimensional objects on the road on which the vehicle is traveling as road surface information based on point cloud data for each frame acquired by the on-board detector 5, a determination unit 113 that determines the interval of detection points necessary for the point cloud data of the next frame based on the size of a predetermined three-dimensional object set in advance as a recognition target and the measured distance from the vehicle to the three-dimensional object based on the point cloud data, and a gradient prediction unit 114 that predicts the gradient of the road surface not recognized by the recognition unit 111 based on gradient information associated with map information on which the road is recorded. The determination unit 113 further determines the interval of detection points necessary for the point cloud data of the next frame based on the size of a predetermined three-dimensional object, the map information and the estimated distance from the vehicle to the three-dimensional object estimated from the gradient, for a range from the furthest distance to the required distance.

Description

Technical Field

[0001] The present invention relates to an external situation recognition device for recognizing the external situation of a vehicle.

Background Art

[0002] As this type of device, there is known a device that performs scanning by changing the irradiation angle of laser light irradiated from a lidar around a first axis parallel to the height direction and a second axis parallel to the horizontal direction, and detects the outside of the vehicle based on the position information of each detection point (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above device, there are many detection points obtained by scanning, and the processing burden for acquiring position information based on each detection point is large. Detecting the external situation of a vehicle enables smooth movement of the vehicle, leading to improved traffic convenience and safety. Thereby, it can contribute to the development of a sustainable transportation system.

Means for Solving the Problems

[0005] An external environment recognition device according to one aspect of the present invention includes an on-board detector that scans and irradiates electromagnetic waves in a first direction and a second direction intersecting the first direction within the field of view, and acquires point cloud data including three-dimensional position information of detection points on the surface of objects based on reflected waves from objects around the vehicle, frame by frame; a recognition unit that recognizes the road surface and three-dimensional objects on the road on which the vehicle is traveling as road surface information based on the point cloud data for each frame; and a detection unit that, based on the size of a predetermined three-dimensional object to be recognized and the measured distance from the vehicle to the three-dimensional object based on the point cloud data, detects the necessary detections for the point cloud data of the next frame. The system includes a determination unit that determines the interval between points, and a gradient prediction unit that, when the furthest distance to the road surface in the direction of travel recognized by the recognition unit is shorter than the required distance based on the vehicle speed, predicts the gradient of the road surface not recognized by the recognition unit based on gradient information associated with map information on which the road is recorded. The determination unit further determines the interval between detection points required for the point cloud data of the next frame, based on the size of a predetermined three-dimensional object and the estimated distance from the vehicle to the three-dimensional object estimated from the map information and gradient, for a range from the furthest distance to the required distance. [Effects of the Invention]

[0006] According to the present invention, it becomes possible to reduce the processing load required to recognize the external environment surrounding the vehicle. [Brief explanation of the drawing]

[0007] [Figure 1A] A diagram showing a vehicle traveling on a road. [Figure 1B] A schematic diagram showing an example of detection data by a lidar. [Figure 2] A block diagram illustrating the main components of a vehicle control system. [Figure 3A] A diagram showing the position of point cloud data in 3D space using a 3D coordinate system. [Figure 3B] A diagram illustrating the mapping of point cloud data from 3D space to 2D XZ space. [Figure 3C] A schematic diagram showing point cloud data divided into grids. [Figure 3D]A schematic diagram showing the road surface gradient in the depth direction. [Figure 4A] A schematic diagram showing the projection angle and depth distance. [Figure 4B] A schematic diagram showing the distance measured by a rider. [Figure 5A] A schematic diagram illustrating an example of the relationship between depth and vertical projection angle. [Figure 5B] A schematic diagram illustrating an example of the relationship between depth and vertical angular resolution. [Figure 6A] A schematic diagram showing an example of an illumination point when the lidar's illumination light is emitted using a raster scanning method. [Figure 6B] A schematic diagram showing an example of illumination points when the lidar's illumination light is directed only to predetermined grid points arranged in a grid pattern within the detection area. [Figure 7] This figure shows an example of the irradiation sequence when irradiating the irradiation point exemplified in Figure 6B with irradiation light. [Figure 8A] A diagram illustrating an example of a prediction. [Figure 8B] A diagram illustrating an example of a prediction. [Figure 9] A flowchart showing an example of the processing performed by the controller's CPU in Figure 2. [Figure 10] A flowchart illustrating the process in S20 of Figure 9. [Modes for carrying out the invention]

[0008] Embodiments of the invention will be described below with reference to the drawings. The external environment recognition device according to the embodiment of the invention can be applied to a vehicle having an automatic driving function, i.e., an autonomous vehicle. The vehicle to which the external environment recognition device according to this embodiment is applied may be referred to as "the vehicle itself" to distinguish it from other vehicles. The vehicle itself may be an engine-powered vehicle with an internal combustion engine as its driving source, an electric vehicle with a drive motor as its driving source, or a hybrid vehicle with both an engine and a drive motor as its driving sources. The vehicle itself can operate not only in an automatic driving mode that does not require driver operation, but also in a manual driving mode with driver operation.

[0009] When an autonomous vehicle is traveling in an autonomous driving mode (hereinafter referred to as autonomous driving or self-driving), it recognizes the external situation around the host vehicle based on the detection data of in-vehicle detectors such as cameras and lidars (LiDAR: Light Detection and Ranging). Based on the recognition result, the autonomous vehicle generates a driving trajectory (target trajectory) from the current time to a time beyond a predetermined time, and controls a driving actuator so that the host vehicle travels along the target trajectory.

[0010] FIG. 1A is a diagram showing a state in which a host vehicle 101, which is an autonomous vehicle, travels on a road RD. FIG. 1B is a schematic diagram showing an example of detection data obtained by a lidar mounted on the host vehicle 101 and directed in the traveling direction of the host vehicle 101. A measurement point (which may also be referred to as a detection point) by the lidar is point information where the irradiated laser is reflected from a certain point on the surface of an object and returns. The point information includes the distance from the laser source to that point, the intensity of the laser reflected and returned, and the relative velocity between the laser source and that point. Further, data composed of a plurality of detection points as shown in FIG. 1B is referred to as point cloud data. In FIG. 1B, point cloud data based on the detection points on the surface of an object included in the field of view (hereinafter referred to as FOV) of the lidar among the objects in FIG. 1A is shown. The FOV may be, for example, 120 deg in the horizontal direction (which may also be referred to as the road width direction) of the host vehicle 101 and 40 deg in the vertical direction (which may also be referred to as the up-down direction). The value of the FOV may be appropriately changed based on the specifications of the external recognition device. The host vehicle 101 recognizes the external situation around the vehicle, more specifically, the road structure and objects around the vehicle, based on the point cloud data as shown in FIG. 1B, and generates a target trajectory based on the recognition result.

[0011] By the way, as a method for sufficiently recognizing the external situation around the vehicle, it is conceivable to increase the number of irradiation points of the electromagnetic wave irradiated from an in-vehicle detector such as a lidar (in other words, increase the irradiation point density of the electromagnetic wave to increase the number of detection points constituting the point cloud data). On the other hand, increasing the number of irradiation points of the electromagnetic wave (increasing the number of detection points) may increase the processing load for controlling the in-vehicle detector or increase the capacity of the detection data (point cloud data) obtained by the in-vehicle detector, thereby increasing the processing load for the point cloud data. In particular, in a situation where many objects exist on the road or beside the road, the capacity of the point cloud data further increases. Therefore, in consideration of the above points, in the embodiment, an external recognition device is configured as follows.

[0012] <Summary> The external recognition device according to the embodiment intermittently irradiates the traveling direction of the host vehicle 101 with irradiation light as an example of electromagnetic wave from the lidar of the host vehicle 101 traveling on the road RD, and discretely acquires point cloud data at different positions on the road RD. The irradiation range of the irradiation light irradiated from the lidar is set so that no data blank interval occurs in the traveling direction of the road RD between the point cloud data of the previous frame acquired by the lidar in the previous irradiation and the point cloud data of the next frame acquired by the lidar in the current irradiation. The detection point density within the irradiation range is set to be high, for example, for the road surface far from the host vehicle 101 and low for the road surface close to the host vehicle 101. Compared with the case where a high detection point density is set for all road surfaces within the irradiation range, the total number of detection points used for the recognition process is suppressed. As a result, it is possible to reduce the number of detection points used for the recognition process without degrading the recognition accuracy of the position (distance from the host vehicle 101) and size of an object or the like recognized based on the point cloud data. In addition, it is also possible to configure the lidar to be small and inexpensive, such as reducing the number of laser elements provided in the lidar. Such an external recognition device will be described in more detail.

[0013] <Configuration of Vehicle Control Device> Figure 2 is a block diagram showing the main components of a vehicle control device 100, including an external environment recognition device. This vehicle control device 100 includes a controller 10, a communication unit 1, a positioning unit 2, an internal sensor group 3, a camera 4, a lidar 5, and a driving actuator AC. The vehicle control device 100 also includes an external environment recognition device 50, which constitutes part of the vehicle control device 100. The external environment recognition device 50 recognizes the external environment around the vehicle based on detection data from on-board detectors such as the camera 4 and the lidar 5.

[0014] Communication unit 1 communicates with various servers (not shown) via a network including wireless communication networks such as the Internet and mobile phone networks, and acquires map information, driving history information, and traffic information from the servers periodically or at arbitrary times. The network includes not only public wireless communication networks but also closed communication networks established for each designated management area, such as wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc. The acquired map information is output to storage unit 12, and the map information is updated. The map information is associated with a road surface gradient map, which will be used for predicting road surface gradients, as will be described in detail later. The positioning unit (GNSS unit) 2 has a positioning sensor that receives positioning signals transmitted from positioning satellites. Positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 2 uses the positioning information received by the positioning sensor to measure the current position (latitude, longitude, altitude) of the vehicle 101.

[0015] The internal sensor group 3 is a collective term for multiple sensors (internal sensors) that detect the driving state of the vehicle 101. For example, the internal sensor group 3 includes a vehicle speed sensor that detects the vehicle speed (driving speed) of the vehicle 101, acceleration sensors that detect the longitudinal acceleration and lateral acceleration (lateral acceleration) of the vehicle 101, respectively, a rotation speed sensor that detects the rotation speed of the driving power source, and a yaw rate sensor that detects the rotational angular velocity of the vehicle 101 around the vertical axis of its center of gravity. Sensors that detect the driver's driving operations in manual driving mode, such as operation of the accelerator pedal, brake pedal, and steering wheel, are also included in the internal sensor group 3.

[0016] Camera 4 has an image sensor such as a CCD or CMOS and captures images of the area around the vehicle 101 (front, rear, and sides). LiDAR 5 receives scattered light from the illuminated light and measures the distance from the vehicle 101 to surrounding objects, the position of the objects, their shape, etc.

[0017] Actuator AC is a drive actuator used to control the movement of the vehicle 101. When the drive source is an engine, Actuator AC includes a throttle actuator that adjusts the opening degree (throttle opening) of the engine's throttle valve. When the drive source is a drive motor, the drive motor is included in Actuator AC. Brake actuators that operate the vehicle 101's braking system and steering actuators that drive the steering system are also included in Actuator AC.

[0018] The controller 10 is comprised of an electronic control unit (ECU). More specifically, the controller 10 includes a computer comprising an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as ROM or RAM, and other peripheral circuits (not shown) such as an I / O interface. While multiple ECUs with different functions, such as an engine control ECU, a drive motor control ECU, and a braking system ECU, can be provided separately, in Figure 2, for convenience, the controller 10 is shown as a collection of these ECUs.

[0019] The memory unit 12 is capable of storing highly accurate and detailed map information (referred to as high-precision map information). This high-precision map information includes road location information, road shape information (curvature, etc.), road gradient information, intersection and branching point location information, number of lanes (driving lanes), lane width and location information for each lane (information on the center position of the lane and the boundary lines of the lane positions), location information of landmarks (traffic lights, signs, buildings, etc.) as markers on the map, and road surface profile information such as road surface irregularities. In addition to the 2D map information described later, the memory unit 12 can also store various control programs, threshold information used in the programs, and setting information for on-board detectors such as the LiDAR 5 (illumination point information described later). Furthermore, since this embodiment does not necessarily require highly accurate and detailed map information, it is not necessary to store detailed map information in the storage unit 12.

[0020] The calculation unit 11 has a functional configuration that includes a recognition unit 111, a setting unit 112, a determination unit 113, a prediction unit 114, and a driving control unit 115. As shown in Figure 2, the recognition unit 111, setting unit 112, determination unit 113, and prediction unit 114 are included in the external environment recognition device 50. As described above, the external environment recognition device 50 recognizes the external environment around the vehicle based on detection data from on-board detectors such as the camera 4 and lidar 5. Details of the recognition unit 111, setting unit 112, determination unit 113, and prediction unit 114 included in the external environment recognition device 50 will be described later.

[0021] In automatic driving mode, the driving control unit 115 generates a target trajectory based on the external conditions around the vehicle recognized by the external environment recognition device 50, and controls the actuator AC so that the vehicle 101 travels along that target trajectory. In manual driving mode, the driving control unit 115 controls the actuator AC in response to driving commands (such as steering operations) from the driver acquired by the internal sensor group 3.

[0022] Let me explain more about the Rider 5. <Detection Area> The Rider 5 is mounted facing forward of the vehicle 101 so that its field of view (FOV) includes the area that should be monitored while driving. Since the Rider 5 receives light scattered by the three-dimensional object that illuminates it, the FOV of the Rider 5 corresponds to the illumination range and detection area of ​​the illuminated light. In other words, the illuminated point within the illumination range corresponds to the detected point within the detection area. In this embodiment, the term "three-dimensional objects, etc." includes the road surface shape, including unevenness, steps, and undulations; three-dimensional objects located on the road RD (equipment related to the road RD, such as traffic lights, signs, ditches, walls, fences, guardrails, etc.); objects on the road RD (including other vehicles and obstacles on the road surface); and lane markings provided on the road surface. Lane markings include white lines (including lines of different colors such as yellow), curb lines, road studs, etc., and may also be called lane marks. Furthermore, three-dimensional objects, etc. that have been pre-set as detection targets are called detection targets.

[0023] <Example of a coordinate system> Figure 3A is a diagram that shows the position of point cloud data in three-dimensional space using a three-dimensional coordinate system. In Figure 3A, the positive x-axis direction corresponds to the direction of travel of the vehicle 101, the positive y-axis direction corresponds to the horizontal left of the vehicle 101, and the positive z-axis direction corresponds to the vertical upward. Furthermore, the x-axis component of the position of data P is called the depth distance X, the y-axis component of the position of data P is called the horizontal distance Y, and the z-axis component of the position of data P is called the height Z. If D is the distance measured by the LIDA 5, in other words, the distance from the LIDA 5 to a point on the object being detected, then the coordinates (X, Y, Z) indicating the position of data P are calculated using the following formula. X = D × cosθ × cosφ (1) Y = D × sinθ × cosφ (2) Z = D × sinφ (3) The angle θ is called the horizontal projection angle, and the angle φ is called the vertical projection angle. The horizontal projection angle θ and the vertical projection angle φ are set in the lidar 5 by the setting unit 112.

[0024] Figure 3B illustrates the mapping of point cloud data from 3D space to 2D XZ space. In this embodiment, in order to calculate the road surface gradient of road RD, each data point constituting the point cloud data is mapped from the data P in 3D space to the data P' in XZ space. This mapping converts the 3D point cloud data into 2D point cloud data in XZ space. In XZ space, information indicating the horizontal distance Y is omitted, while information indicating the depth distance X and height Z remains. Next, the XZ space is divided into grids of a predetermined size (e.g., 50 cm square), and the number of data points P' contained in each grid is counted. Figure 3C is a schematic diagram showing the point cloud data divided by grid. Note that the actual number of grids based on the data points P' is much larger than the number shown. Figure 3C shows the position data (depth distance X) for each grid, the height Z for each grid, and the number of data points P' in each grid. In this embodiment, since the data of three-dimensional objects has been separated and excluded in advance, the grid data consists mainly of X and Z values ​​for the road surface. Therefore, by sequentially extracting the grid with the maximum number of data points P' in the depth distance X direction, a row of grids showing the road surface height Z, i.e., the road surface gradient in the depth distance X direction, as shown in Figure 3D, is obtained. Focusing on each grid, the following equation (4) holds between the vertical projection angle α with respect to the road surface point (corresponding to the illumination point mentioned above) of that grid, the depth distance X of that road surface point, and the height Z of the road surface. Also, the following equation (5) holds between the distance DL from rider 5 to that road surface point, the depth distance X of that road surface point, and the height Z of the road surface. tanα = Z / X (4) DL=(X 2 + Z 2 ) 1 / 2 (5)

[0025] In this embodiment, it is assumed that the pitch angle, roll angle, and yaw angle of the Rider 5 installed on the vehicle 101 are fixed. Furthermore, the road surface gradient map described later may be generated based on the road surface gradient (a row of grids indicating the road surface height Z as described above) obtained by the vehicle 101 or another vehicle equipped with a Rider 5 in the same manner as the vehicle 101, while driving so that the center line of the width of each vehicle follows the driving route. Each vehicle may, for example, periodically or at any time, send data showing the relationship between the height Z of the road surface along the acquired driving route and the depth distance X to an external server device via the communication unit 1. The external server device associates the data showing the relationship between the average height Z of each road surface point along the driving route and the depth distance X, sent from multiple vehicles, with the 2D map information containing the road RD of the driving route as a road gradient map. External server equipment may calculate the difference in height for each road surface point between the average value data of height Z for each road surface point on the travel route newly transmitted from multiple vehicles and the data of the existing road surface gradient map. For areas with a depth distance X where the difference exceeds a predetermined threshold, the existing road surface gradient map may be updated with the new average value data. Furthermore, instead of associating the road surface gradient map with 2D map information, external server devices may add data showing the relationship between the average height Z of each road surface point along the driving route and the depth distance X to the 2D map information as 1D road surface point height information for each route. Each vehicle, including its own vehicle 101, may periodically or at any time obtain the latest road surface gradient map information (or height information for each one-dimensional road surface point) from an external server device or the like via the communication unit 1 and store it in the storage unit 12.

[0026] <Lighting angle and depth> Figure 4A is a schematic diagram showing the vertical projection angle α (angle of the illuminated light relative to the horizontal direction) and depth distance X of the lidar 5. The external environment recognition device 50 changes the direction of the illuminated light vertically by changing the projection angle α, thereby moving the position of the illuminated point vertically. In Figure 4A, for example, if the light is projected onto a road RD at a depth distance X2 of 10m, the road surface will be illuminated at a projection angle α2. Similarly, if the light is projected onto a road RD at a depth distance X1 of 40m, the road surface will be illuminated at a projection angle α1. Furthermore, if the light is projected onto a road RD at a depth distance X0 of 100m, the road surface will be illuminated at a projection angle α0. Generally, the greater the projection angle relative to the road surface, the less scattered light returns from the road surface to the Rider 5. Therefore, in most cases, the received level of scattered light for light illuminating a point at a depth distance X0 will be the lowest.

[0027] Figure 4B is a schematic diagram showing the distance DL measured by the lidar 5. As described above with reference to Figures 3A to 3D, the external environment recognition device 50 uses the projection angle α set for the lidar 5, the distance DL (optical path length of the irradiated light) measured by the lidar 5, and the above equations (4) and (5) to calculate the depth distance X to the road surface point irradiated by the light and the height Z of that road surface point.

[0028] The external environment recognition device 50 adjusts the projection angle α upwards when it is desired to increase the depth distance beyond its current value, and downwards when it is desired to decrease the depth distance beyond its current value. For example, if the depth distance is to be changed from 70m to 100m, the external environment recognition device 50 adjusts the projection angle α upwards so that the light is projected onto the 100m point. Also, if the road RD is on a downhill slope and the light is not projected onto the road RD, the external environment recognition device 50 adjusts the projection angle α downwards so that the light is projected onto the road RD.

[0029] Figure 5A is a schematic diagram showing an example of the relationship between the depth distance X and the vertical projection angle α. The horizontal axis represents the depth distance X (in meters), and the vertical axis represents the vertical projection angle α (in degrees). The projection angle α may also be called the vertical angle. As illustrated in Figure 5A, the external environment recognition device 50 lowers the projection angle α if it wants to shorten the depth distance X, and raises the projection angle α if it wants to lengthen the depth distance X. The symbol N will be explained later.

[0030] <FOV and depth distance> In the embodiment, the road surface condition from the depth distance corresponding to the lower end of the FOV of the lidar 5 (for example, X2 in FIG. 4A) to the depth distance corresponding to the upper end of the FOV (for example, X0 in FIG. 4A) is detected. The depth distance corresponding to the lower end of the FOV is referred to as the first predetermined distance, and the depth distance corresponding to the upper end of the FOV is referred to as the second predetermined distance. Generally, the camera 4 is superior to the lidar 5 in terms of resolution at close range, and the lidar 5 is superior to the camera 4 in terms of distance measurement accuracy and relative speed measurement accuracy. Therefore, when the viewing angle of the camera 4 is wider than the FOV of the lidar 5 in the vertical direction, the camera 4 may be responsible for detecting the road surface condition for the road surface below the lower end of the FOV of the lidar 5 (in other words, the road surface closer to the host vehicle 101).

[0031] <Number of irradiation points of irradiation light> The external recognition device 50 calculates the position of the irradiation point that irradiates the irradiation light of the lidar 5 within the FOV of the lidar 5. More specifically, the external recognition device 50 calculates the irradiation point according to the angular resolution calculated based on the minimum size of the three-dimensional object (for example, 15 cm both in the vertical direction and the horizontal direction) and the required depth distance (for example, 100 m) specified in advance as the detection target (which may also be called the recognition target). The three-dimensional object corresponds to, for example, a stone or a concrete piece on the road. The required depth distance corresponds to the braking distance of the host vehicle 101 that changes according to the vehicle speed. In the embodiment, based on the idea that the host vehicle 101 in motion should detect the road surface condition of the road in the traveling direction at least up to a distance beyond the braking distance, a value obtained by adding a predetermined margin to the braking distance is referred to as the required depth distance. The vehicle speed of the host vehicle 101 is detected by the vehicle speed sensor of the internal sensor group 3. The relationship between the vehicle speed and the required depth distance is stored in the storage unit 12 in advance. The symbol N in FIG. 5A indicates the required depth distance when the vehicle speed is, for example, 100 km / h. As an example of the angular resolution required when detecting a 15cm target at a required depth of 100m, as shown in Figure 5B and described later, an angular resolution of 0.05 degrees is required in both the vertical and horizontal directions. Note that when detecting targets smaller than 15cm, or when detecting a 15cm target at a depth X longer than 100m, it is necessary to increase the number of illumination points within the FOV by increasing the angular resolution.

[0032] The external environment recognition device 50 calculates the positions of the illumination points so that they are arranged in a grid pattern within the FOV, and the vertical and horizontal spacing of the grid points corresponds to the vertical and horizontal angular resolutions, respectively. To increase the vertical angular resolution, the FOV is divided vertically by a number based on the angular resolution, and the vertical grid spacing is narrowed to increase the number of illumination points. In other words, the spacing of the illumination points is made denser. Conversely, to decrease the vertical angular resolution, the FOV is divided vertically by a number based on the angular resolution, and the vertical grid spacing is widened to decrease the number of illumination points. In other words, the spacing of the illumination points is made coarser. The same applies to the horizontal direction. The external environment recognition device 50 generates information indicating the position of the illumination point calculated according to the angular resolution (hereinafter referred to as illumination point information) and stores it in the storage unit 12 in association with position information indicating the current driving position of the vehicle 101.

[0033] <Angular resolution and depth of field> Figure 5B is a schematic diagram illustrating an example of the relationship between depth distance X and vertical angular resolution, showing the angular resolution (which may also be called required angular resolution) necessary to recognize a detection target of the aforementioned size (15 cm in both length and width). The horizontal axis represents depth distance X (in meters), and the vertical axis represents vertical angular resolution (in degrees). Generally, the shorter the depth distance X (in other words, the closer the detection target is to the vehicle 101), the larger the viewing angle to the detection target, making it possible to detect the target even with low angular resolution. Conversely, the longer the depth distance X (in other words, the farther the detection target is from the vehicle 101), the smaller the viewing angle to the detection target, requiring high angular resolution to detect the target. Therefore, as illustrated in Figure 5B, the external environment recognition device 50 lowers the angular resolution (increases the value) as the depth distance X decreases, and increases the angular resolution (decreases the value) as the depth distance X increases. Although not shown in the diagram, the same principle applies to the relationship between the depth distance X and the horizontal angular resolution. In Figure 5B, the symbol N indicates the required depth when the vehicle speed is, for example, 100 km / h.

[0034] When the vehicle 101 is driving in automatic driving mode, the external environment recognition device 50 controls the lidar 5 to set a predetermined illumination point (detection point) within the FOV and to emit illumination light. As a result, the illumination light from the lidar 5 is directed towards the set illumination point (detection point).

[0035] Furthermore, the illumination light from the lidar 5 may be irradiated to all illumination points (detection points) arranged in a grid pattern within the FOV using a raster scanning method, or the illumination light may be irradiated intermittently so that the illumination light is irradiated only to predetermined illumination points (detection points), or it may be irradiated in any other manner.

[0036] Figure 6A is a schematic diagram showing an example of an illumination point when the illumination light from the lidar 5 is emitted using a raster scanning method. When the external environment recognition device 50 emits illumination light from the lidar 5, it sets the required angular resolution at the required depth distance N for the entire FOV and controls the illumination direction of the illumination light. For example, if the required angular resolution for recognizing a target located at a required depth distance N on a road RD is 0.05 degrees in both the vertical and horizontal directions, the external environment recognition device 50 controls the direction of illumination light to shift by 0.05 degrees in both the vertical and horizontal directions across the entire FOV. In other words, in Figure 6A, each black circle on the grid corresponds to an illumination point (detection point), and the intervals between vertical and horizontal illumination points (detection points) correspond to an angular resolution of 0.05 degrees, respectively. The actual number of illumination points within the FOV is far greater than the number of black circles shown in Figure 6A. For example, if the FOV of Lida 5 is 120 degrees horizontally, there are 2400 black circles corresponding to illumination points (detection points) arranged horizontally at 0.05-degree intervals. Similarly, if the FOV is 40 degrees vertically, there are 800 black circles corresponding to illumination points (detection points) arranged vertically at 0.05-degree intervals.

[0037] The external environment recognition device 50 acquires detection data for detection points corresponding to the illumination points in Figure 6A each time it scans the illumination light for one frame across the FOV, and extracts detection point data based on the angular resolution required for recognition of the target object from this detection data. More specifically, for areas within the FOV where the depth distance X is shorter than the required depth distance N, and where an angular resolution of 0.1 deg is sufficient instead of 0.05 deg, data is extracted so that the vertical and horizontal data intervals are wider than 0.05 deg intervals. Similarly, for areas within the FOV corresponding to the sky, data is extracted so that the vertical and horizontal data intervals are wider because there are no roads RD. The intervals of the detection points extracted in this way are the same as the intervals of the detection points shown by black circles in Figure 6B, which will be described later. By having the external environment recognition device 50 extract data from the detection points, it becomes possible to reduce the total amount of detection data used in the recognition process.

[0038] Figure 6B is a schematic diagram showing an example of irradiation points when the light emitted from the lidar 5 is emitted only to predetermined irradiation points (detection points) arranged in a grid within the field of view (FOV). When the external environment recognition device 50 emits light from the lidar 5, it sets the spacing between the irradiation points (detection points) within the FOV to an interval corresponding to the required angular resolution and controls the direction of the emitted light. For example, if the required angular resolution for recognizing a target located at a required depth distance N on a road RD is 0.05 degrees in both the vertical and horizontal directions, the external environment recognition device 50 controls the direction of the illumination light to shift in both the vertical and horizontal directions by 0.05 degree intervals in the region corresponding to the required depth distance N (a long, strip-shaped region in the left-right direction). Furthermore, in areas of the FOV where the depth distance X is shorter than the required depth distance N, and where a required angular resolution of 0.1 degrees is sufficient, the direction of the illumination light is controlled to widen the spacing between vertical and horizontal detection points. In addition, in areas of the FOV corresponding to the sky, the direction of the illumination light is controlled to widen the spacing between vertical and horizontal detection points, since there is no road RD. As an example, illumination is started with the road surface being flat, or with the density distribution of illumination points (detection points) as measured at the end of the previous measurement. The external environment recognition device 50 controls the spacing between detection points (in other words, controls the spacing (density) of irradiation points during scanning irradiation), thereby making it possible to suppress the total number of detection data used in the recognition process. Furthermore, the actual number of illumination points within the FOV is far greater than the number of black circles shown in Figure 6B.

[0039] Figure 7 shows an example of the irradiation sequence when irradiating the irradiation point illustrated in Figure 6B. In Figure 7, the irradiation direction of the light is controlled in the direction of the arrows, from the upper left of the FOV towards the irradiation point in the lower right. The letters P1 to P3 written next to the vertical arrows indicate the magnitude of the interval between the irradiation points (detection points). P1 indicates, for example, the interval between irradiation points (detection points) corresponding to an angular resolution of 0.05 degrees. P2 indicates, for example, the interval between irradiation points (detection points) corresponding to an angular resolution of 0.1 degrees. P3 indicates, for example, the interval between irradiation points (detection points) corresponding to an angular resolution of 0.2 degrees. Figure 7 illustrates an example of switching the angular resolution in three stages, but it is not limited to three stages; the system can be configured to switch between two or more angular resolution stages as appropriate. For example, in addition to the intervals P1, P2, and P3 between the irradiation points (detection points), an additional interval P4 corresponding to an angular resolution of 0.3 degrees could be added to create a four-stage switching system.

[0040] <Configuration of the external environment recognition device> Details of the external environment recognition device 50 will be explained. As described above, the external environment recognition device 50 includes a recognition unit 111, a setting unit 112, a determination unit 113, a prediction unit 114, and a lidar 5. <Recognition part> The recognition unit 111 generates three-dimensional point cloud data using time-series detection data detected in the FOV of the lidar 5. Furthermore, the recognition unit 111 recognizes the road structure in the direction of travel of the road RD on which the vehicle 101 is traveling, and the detected object on the road RD in the direction of travel, based on the detection data measured by the rider 5. The road structure refers to, for example, straight roads, curved roads, branching roads, tunnel entrances and exits, etc. Furthermore, the recognition unit 111 detects lane markings by, for example, applying brightness filtering to data indicating a flat road surface. In this case, the recognition unit 111 may determine that a lane marking is present if the height of the road surface where the brightness exceeds a predetermined threshold is approximately the same as the height of the road surface where the brightness does not exceed the threshold.

[0041] <Understanding Road Structure> An example of road structure recognition by the recognition unit 111 will be described. The recognition unit 111 recognizes curbs, walls, ditches, guardrails, or lane markings of the road RD ahead, which is the direction of travel, as boundary lines RL and RB (Figure 1A) of the road RD, and recognizes the road structure in the direction of travel indicated by boundary lines RL and RB. As described above, lane markings include white lines (including lines of different colors), curb lines, road studs, etc., and the driving lanes of the road RD are defined by these markings. In this embodiment, the boundary lines RL and RB of the road RD defined by the above markings are called lane markings.

[0042] The recognition unit 111 recognizes the area enclosed by boundary lines RL and RB as the area corresponding to road RD. However, the method of recognizing road RD is not limited to this, and other methods may be used. Furthermore, the recognition unit 111 separates the generated point cloud data into point cloud data representing flat road surfaces and point cloud data representing three-dimensional objects, etc. For example, among the three-dimensional objects, etc. on the road in the direction of travel included in the point cloud data, it recognizes road surface shapes such as bumps, steps, and undulations that are larger than 15 cm in size, and objects that are larger than 15 cm in length and width, as detection targets. 15 cm is just an example of the size of the detection target and can be changed as appropriate.

[0043] <Settings section> The setting unit 112 sets the vertical projection angle φ of the illumination light to the lidar 5. If the FOV of the lidar 5 is 40 degrees in the vertical direction, the vertical projection angle φ is set in increments of 0.05 degrees within the range of 0 to 40 degrees. Similarly, the setting unit 112 sets the horizontal projection angle θ of the illumination light to the lidar 5. If the FOV of the lidar 5 is 120 degrees in the horizontal direction, the horizontal projection angle θ is set in increments of 0.05 degrees within the range of 0 to 120 degrees. The setting unit 112 sets the number of illumination points within the FOV (corresponding to the number of black circles in Figures 6A and 6B, and indicating the illumination point density) to the lidar 5 based on the angular resolution determined by the determination unit 113, as will be described later. As mentioned above, the vertical and horizontal spacing of the illumination points (detection points) arranged in a grid within the FOV corresponds to the vertical and horizontal angular resolutions, respectively.

[0044] <Decision Section> The determination unit 113 determines the scanning angle resolution set by the setting unit 112. First, the determination unit 113 calculates the vertical projection angle α at each depth distance X and the distance DL to the road surface point at each depth distance X. Specifically, as explained with reference to Figure 3D, the depth distance X is calculated based on the distance DL to the road surface point measured by the lidar 5 and the projection angle α set on the lidar 5 during measurement. The determination unit 113 calculates the relationship between the calculated depth distance X and the vertical angle (Figure 5A). The determination unit 113 also calculates the relationship between the depth distance X and the distance DL. Furthermore, as illustrated in Figure 5B, the determination unit 113 calculates the relationship between the depth distance X and the vertical angle resolution based on the size of the detection target and the depth distance X. In this way, the vertical angle resolution is calculated based on the size of the detection target and the distance DL, and the relationship between the depth distance X and the vertical angle resolution is calculated based on the distance DL and the depth distance X.

[0045] Next, the determination unit 113 determines the vertical angular resolution necessary to recognize the detection target of the above size. For example, in Figure 5B, for a depth distance X where the vertical angular resolution is less than 0.1 degrees, 0.05 degrees, which is smaller than 0.1 degrees, is determined as the required angular resolution. Similarly, for a depth distance X where the vertical angular resolution is 0.1 degrees or more but less than 0.2 degrees, 0.1 degrees, which is smaller than 0.2 degrees, is determined as the required angular resolution. In the same manner, for depth distances X where the vertical angular resolution is 0.2 degrees or more but less than 0.3 degrees, and for depth distances X where the vertical angular resolution is 0.3 degrees or more but less than 0.4 degrees, the smaller values ​​of 0.2 degrees and 0.3 degrees, respectively, are determined as the required angular resolution.

[0046] The determined required vertical angular resolution can be reflected as the vertical spacing of the detected points when acquiring the 3D point cloud data for the next frame. Furthermore, the determination unit 113 may determine the required horizontal angular resolution for recognizing the target object, depending on the size and depth distance X of the target object. The required horizontal angular resolution can also be reflected as the horizontal spacing of the detection points when acquiring the 3D point cloud data for the next frame. Furthermore, the required angular resolution in the horizontal direction may be set to match the required angular resolution in the vertical direction that was determined earlier. In other words, on the same horizontal line as a detection point where the required angular resolution in the vertical direction was determined to be 0.05 degrees, the required angular resolution in the horizontal direction should be set to 0.05 degrees. Similarly, on the same horizontal line as a detection point where the required angular resolution in the vertical direction was determined to be 0.1 degrees, the required angular resolution in the horizontal direction should be set to 0.1 degrees. In addition, for other required angular resolutions, on the same horizontal line as a detection point where the required angular resolution in the vertical direction was determined, the required angular resolution in the horizontal direction should be set to the same value as the required angular resolution in the vertical direction.

[0047] <Prediction Section> For example, if the road surface RD on which vehicle 101 is traveling is flooded with rainwater, etc., the rider 5 may not be able to receive scattered light up to the required depth distance N. In such cases, the furthest depth distance X that can be detected by the rider 5 is called the maximum depth distance L. The maximum depth distance may also be called the maximum road surface detection distance. In addition, if the road surface RD is on a downhill slope in the direction of travel, or if the vehicle speed of vehicle 101 is high and the required depth distance N is long, the rider 5 may not be able to receive scattered light up to the required depth distance N.

[0048] The prediction unit 114 predicts the height Z (road surface gradient) of the road surface from the maximum depth distance L to the required depth distance N using the road surface gradient map described above, if the required depth distance N calculated from the vehicle speed of the vehicle 101 exceeds the maximum depth distance L (for example, if the required depth distance N is 115m and the maximum depth distance L = 80m). An example of a prediction will be explained with reference to Figures 8A and 8B. Figures 8A and 8B are two-dimensional graphs illustrating the relationship between the depth distance Xr on the travel route and the road surface height Z. The horizontal axis represents the depth distance Xr (in meters), and the vertical axis represents the road surface height Z (in meters). The scale on the horizontal axis is based on the current position of the vehicle 101, with negative values ​​indicating areas closer to the vehicle 101 than the current position, and positive values ​​indicating areas further in the depth direction than the current position.

[0049] The prediction unit 114 aligns the road surface height Z measured by the rider 5 with the road surface gradient map associated with the map information. More specifically, with respect to the point cloud data represented based on the position of the vehicle 101 acquired using the positioning unit 2, the prediction unit adjusts the position on the Xr axis by relatively shifting the road surface gradient map forward or backward on the Xr axis in a two-dimensional graph of depth distance Xr and height Z on the driving route. This minimizes the discrepancy ΔZ between the measured result of the road surface height Z (measured data based on point cloud data) and the road surface height Z according to the road surface gradient map.

[0050] In Figure 8A, the measurement data showing the road surface height Z measured by Rider 5, specifically the range from negative tens of meters to positive tens of meters (e.g., 10m) on the vehicle 101 side (shown as a solid line), represents the road surface height Z based on point cloud data from the past few frames measured chronologically by Rider 5. Furthermore, the range from positive tens of meters (e.g., 10m) to the maximum depth distance L (shown as a double line) beyond the solid line shown above represents the road surface height Z based on point cloud data for the current frame (newly acquired frame). In addition, the data shown as a dashed line represents the road surface height Z based on the road surface gradient map. Generally, if the current position of the vehicle 101 obtained using the positioning unit 2 contains errors, the measurement data on the driving route and the position on the road surface gradient map will not match. As a result, the measured result of the road surface height Z (displayed as a solid or double line) and the road surface height Z according to the road surface gradient map (displayed as a dashed line) will not match, causing a discrepancy ΔZ in the height direction.

[0051] As an example of how to suppress the displacement ΔZ, the prediction unit 114 shifts the road surface gradient map data (shown as a dashed line) along the Xr axis to match the data (shown as a double line) for a predetermined section (e.g., 5m to 10m) of the road surface height Z (measured data) based on the point cloud data for the current frame. The unit searches for the position that minimizes the least squares sum based on the magnitude of the displacement ΔZ, and then shifts the road surface gradient map relative to that position.

[0052] As shown in Figure 8B, after shifting the position of the road surface gradient map relative to the current position of the vehicle 101 on the Xr axis, the measured result of the road surface height Z (shown as a solid or double line) and the road surface height Z according to the road surface gradient map (shown as a dashed line) coincide, and the height deviation ΔZ is kept below a predetermined value. As explained above, by shifting the road surface gradient map data in Figure 8A in the forward and backward direction along the depth distance Xr axis on the driving route, it becomes possible to align the position of the road surface gradient map with the position of the vehicle 101 (Figure 8B).

[0053] <Generating location data> The external environment recognition device 50 can generate continuous position data by mapping data indicating the position of a detected object, which is detected based on time-series point cloud data measured in real time by the lidar 5, onto a two-dimensional map, for example, in XY space. In XY space, information indicating height Z is omitted, while information on depth distance X and horizontal distance Y remains. The recognition unit 111 acquires positional information of three-dimensional objects, etc., on the two-dimensional map stored in the memory unit 12, and calculates the relative position of the three-dimensional objects, etc., by performing a coordinate transformation around the position of the vehicle 101 based on the vehicle's speed and direction of movement (e.g., azimuth angle). Each time point cloud data is acquired by the lidar 5 through measurement, the recognition unit 111 records the relative position of the three-dimensional objects, etc., based on the acquired point cloud data by performing a coordinate transformation around the position of the vehicle 101 on the two-dimensional map.

[0054] <Explanation of the flowchart> Figure 9 is a flowchart showing an example of a process executed by the arithmetic unit 11 of the controller 10 in Figure 2 according to a predetermined program. The process shown in the flowchart of Figure 9 is repeated at predetermined intervals, for example, while the vehicle 101 is driving in automatic driving mode.

[0055] First, in step S10, the calculation unit 11 has the lidar 5 acquire 3D point cloud data and proceeds to step S20. In step S20, the calculation unit 11 calculates the road surface gradient and maximum depth distance L in the direction of travel of the road RD based on the point cloud data acquired by the rider 5, and proceeds to step S30. Details of the process in step S20 will be described later with reference to Figure 10.

[0056] In step S30, the prediction unit 114 of the calculation unit 11 determines whether the maximum depth distance L is shorter than the required depth distance N. If the maximum depth distance L is shorter than the required depth distance N, the calculation unit 11 affirms step S30 and proceeds to step S40. If the maximum depth distance L is longer than the required depth distance N, it denies step S30 and proceeds to step S50.

[0057] In step S40, the prediction unit 114 of the calculation unit 11 predicts the road surface gradient from the maximum depth distance L to the required depth distance N, and proceeds to step S50. An example of the road surface gradient prediction result is shown in Figure 8B.

[0058] In step S50, the calculation unit 11 calculates the vertical projection angle α and the distance DL to the road surface point for each depth distance X, and proceeds to step S60. The relationship between the vertical angle and the depth distance X is illustrated in Figure 5A. The relationship between the depth distance X, the road surface height Z, and the distance DL to the road surface is illustrated in Figure 4B.

[0059] In step S60, the calculation unit 11 calculates the required angular resolution for each depth distance X and proceeds to step S70. The required angular resolution is the angular resolution necessary to detect a detection target of a predetermined size. The relationship between the depth distance X and the angular resolution is illustrated in Figure 5B.

[0060] In step S70, the calculation unit 11 determines the vertical angular resolution to the required angular resolution using the determination unit 113 and proceeds to step S80. In this embodiment, the vertical angular resolution is determined before the horizontal angular resolution.

[0061] In step S80, the determination unit 113 of the calculation unit 11 determines the horizontal angular resolution to the required angular resolution and proceeds to step S90. By determining the horizontal angular resolution after the vertical angular resolution, it becomes easier to make the horizontal angular resolution match the vertical angular resolution.

[0062] In step S90, the calculation unit 11 determines the coordinates of the detection point. More specifically, it determines the coordinates indicating the position of the detection point, as illustrated by the black circle in Figure 6B. Based on the detection data detected at the position of the detection point determined in step S90, the recognition unit 111 recognizes three-dimensional objects, etc., in the direction of travel of the road RD on which the vehicle 101 is traveling.

[0063] Furthermore, each time point cloud data is acquired in step S10, the calculation unit 11 generates continuous position data in two dimensions by mapping the relative positions of three-dimensional objects, etc., based on the point cloud data onto a two-dimensional XY map. The relative positions of three-dimensional objects, etc., based on the point cloud data can then be converted to coordinates around the position of the vehicle 101 and recorded on the two-dimensional map.

[0064] In step S100, the calculation unit 11 determines whether or not to terminate the process. If the vehicle 101 is continuing to drive in automatic driving mode, the calculation unit 11 negates step S100 and returns to step S10, repeating the process described above. By returning to step S10, the measurement of three-dimensional objects, etc., based on point cloud data is periodically repeated while the vehicle 101 is driving. On the other hand, if the vehicle 101 has finished driving in automatic driving mode, the calculation unit 11 affirms step S100 and terminates the process shown in Figure 9.

[0065] Figure 10 is a flowchart illustrating the details of the process in step S20 (Figure 9) performed by the calculation unit 11. The calculation unit 11 performs the processing shown in Figure 10 on the point cloud data of the detected points determined by the determination unit 113. In step S210, the calculation unit 11 performs separation processing on the point cloud data and proceeds to step S220. More specifically, by detecting and separating data of three-dimensional objects on the road RD from the point cloud data, it obtains point cloud data representing a flat road surface and point cloud data representing three-dimensional objects. Three-dimensional objects include obstacles on the road, curbs, walls, ditches, guardrails, etc., provided at the left and right ends of the road RD, as well as other vehicles such as motorcycles in motion.

[0066] An example of separation processing is described below. The calculation unit 11 transforms the relative positions of the point cloud data into coordinates centered on the position of the vehicle 101, and represents the road RD on a two-dimensional XY map corresponding to the depth direction and road width direction, for example, as if viewed from above, and grids the two-dimensional map to a predetermined size. If the difference between the maximum and minimum values ​​of the data in each grid is smaller than a predetermined threshold, the calculation unit 11 determines that the data in that grid represents a flat road surface. On the other hand, if the difference between the maximum and minimum values ​​of the data in the grid is larger than a predetermined threshold, the calculation unit 11 determines that the data in that grid represents a three-dimensional object or the like. Note that other methods may be used to determine whether the point cloud data corresponds to road surface data or to three-dimensional objects, etc.

[0067] In step S220, the calculation unit 11 determines whether the data to be processed is road surface data or not. If the data is grid data separated as road surface data, the calculation unit 11 affirms step S220 and proceeds to step S230. On the other hand, if the data is grid data separated as data of a three-dimensional object, the calculation unit 11 negates step S220 and proceeds to step S250.

[0068] If the process proceeds to step S250, the recognition unit 111 of the calculation unit 11 performs a coordinate transformation on the 2D map based on the point cloud data of the grid, representing the relative position of the 3D object, etc., centered on the position of the vehicle 101. Then, the process shown in Figure 10 is completed, and the process proceeds to step S30 in Figure 9.

[0069] If the process proceeds to step S230, the prediction unit 114 of the calculation unit 11 calculates the road surface gradient of road RD. An example of the road surface gradient calculation process is explained with reference to Figures 3A to 3D. Note that other methods may be used to calculate the road surface gradient.

[0070] In step S240, the prediction unit 114 of the calculation unit 11 obtains the maximum depth distance L, completes the processing shown in Figure 10, and proceeds to step S30 in Figure 9. As described above, the maximum depth distance L is the furthest depth distance that can be detected by the rider 5. The prediction unit 114 of the calculation unit 11 obtains the depth distance corresponding to the data of the grid furthest from the position of the vehicle 101 among the grids extracted during the road surface gradient calculation process as the maximum depth distance L.

[0071] According to the embodiments described above, the following effects and advantages are achieved. (1) The external environment recognition device 50 includes a lidar 5 as an on-board detector that scans and irradiates light as electromagnetic waves in the horizontal direction as a first direction and the vertical direction as a second direction intersecting the first direction within the field of view (FOV), and acquires point cloud data including 3D position information of detection points on the surface of objects based on reflected waves from objects around the vehicle 101, frame by frame; a recognition unit 111 that recognizes the road surface and three-dimensional objects on the road RD on which the vehicle 101 is traveling as road surface information based on the point cloud data for each frame; and a determination unit 11 that determines the interval of detection points necessary for the point cloud data of the next frame based on the size of a predetermined three-dimensional object to be recognized and the measured distance from the vehicle 101 to the three-dimensional object based on the point cloud data. 3. The system includes a prediction unit 114, which is a gradient prediction unit that predicts the gradient of the road surface not recognized by the recognition unit 111, based on a road surface gradient map as gradient information associated with map information on which the road RD is recorded, when the maximum depth distance L, which is the furthest distance on the road surface in the direction of travel of the vehicle 101 recognized by the recognition unit 111, is shorter than the required depth distance N, which is the required distance based on the vehicle speed of the vehicle 101. The determination unit 113 further determines the interval between detection points required for the point cloud data of the next frame, based on the size of a predetermined three-dimensional object and the estimated distance from the vehicle 101 to the three-dimensional object estimated from the map information and gradient, for the range from the maximum depth distance L to the required depth distance N. Generally, the shorter the depth distance X, the larger the viewing angle to the object to be recognized, making it possible to recognize the object even with low angular resolution. Conversely, the longer the depth distance X, the smaller the viewing angle to the object to be recognized, requiring high angular resolution for recognition. In this embodiment, the lidar 5 acquires the depth distance X to the road surface of the road RD in the direction of travel at each detection point, and the determination unit 113 determines the interval of detection points necessary for the recognition unit 111 to recognize the three-dimensional object at that depth distance X. Furthermore, for road surfaces where the depth distance X cannot be acquired by the lidar 5, the prediction unit 114 predicts the road surface based on a road surface gradient map, and the determination unit 113 determines the interval of detection points necessary for the recognition unit 111 to recognize the three-dimensional object at the predicted depth distance X on the road surface. With this configuration, the determination unit 113 appropriately controls the spacing between detection points in the 3D point cloud data used for recognition processing by the recognition unit 111, thereby suppressing the total number of detection data used for recognition processing. In other words, the processing load on the calculation unit 11 can be reduced without decreasing the recognition accuracy of the position and size of objects, etc., that the external recognition device 50 is trying to detect. Furthermore, in this embodiment, even when the vehicle 101 travels on a road RD not included in high-precision map information, a road RD being traveled on for the first time without high-precision map information, or a road RD that has changed in a manner different from the high-precision map information due to construction or the like, it is possible to determine the interval between detection points necessary for the recognition unit 111 to recognize the three-dimensional object at each depth distance X by using the lidar 5 to acquire the depth distance X to the road surface of the road RD in the direction of travel at each detection point.

[0072] (2) The external environment recognition device 50 described in (1) above further includes a positioning unit 2 as a position detection unit that detects the position of the vehicle 101 based on information from satellites, and a prediction unit 114 as an adjustment unit that relatively shifts the position of the vehicle 101 detected by the positioning unit 2 with the positions of map information and gradient information. With this configuration, even if the current position of the vehicle 101 acquired using the positioning unit 2 contains an error, causing the position on the Xr axis to not match between the measurement data from the Lidar 5 (data represented based on the position of the vehicle 101 detected by the positioning unit 2) and the road surface gradient map data (Figure 8A), it is possible to effectively correct the error on the Xr axis (Figure 8B). As a result, compared to the case where the prediction unit 114 does not function as an adjustment unit, it becomes possible to predict the road surface gradient that is not recognized by the recognition unit 111 with greater accuracy.

[0073] (3) In the external environment recognition device 50 described in (2) above, gradient information is created based on the height Z of the road surface of the road RD measured by the Rider 5 of the own vehicle 101 and / or other vehicles, and the prediction unit 114, acting as an adjustment unit, relatively shifts the position of the gradient map relative to the position of the own vehicle 101 so that the difference between the height Z of the road surface of a predetermined section (e.g., 10m) including the maximum depth distance L of the road RD based on point cloud data and the height Z of the road surface of the section corresponding to the maximum depth distance L of the road surface for which the gradient has been predicted by the prediction unit 114 is less than or equal to a predetermined value. With this configuration, after relatively shifting the position of the road surface gradient map, the measured result of the road surface height Z (shown as a solid or double line in Figure 8B) and the road surface height Z calculated by the road surface gradient map (shown as a dashed line in Figure 8B) coincide, and the height deviation ΔZ is kept below a predetermined value. This makes it possible to accurately predict the road surface gradient that has not been recognized by the recognition unit 111.

[0074] (4) In the external environment recognition device 50 described in (1) above, the determination unit 113 further determines the interval between detection points required for the point cloud data of the next frame as the scanning angle resolution of the illumination light, and makes the scanning angle resolution coarser as the target area of ​​the illumination light within the field of view (FOV) of the lidar 5 becomes farther than the required depth distance N. With this configuration, for example, it becomes possible to ensure higher recognition accuracy in the region corresponding to the required depth distance N, while lowering the recognition accuracy in the empty region above it, thereby suppressing the total number of detection data used for recognition processing by the recognition unit 111. In other words, it is possible to reduce the processing load of the calculation unit 11 without reducing the recognition accuracy of the vertical position and size of the object to be recognized by the external environment recognition device 50.

[0075] (5) In the external environment recognition device 50 described in (4) above, the determination unit 113 further reduces the scanning angle resolution as the scanning target of the illumination light within the field of view (FOV) of the lidar 5 gets closer to the required depth distance N. This configuration prevents the creation of an excessive number of detection points for recognition targets close to the vehicle 101. In other words, it reduces the processing load on the calculation unit 11 without reducing the accuracy of recognizing the horizontal position and size of objects, etc., that the external environment recognition device 50 recognizes.

[0076] The above embodiment can be modified into various forms. Modifications will be described below. (Variation 1) In the embodiment described above, an example was explained in which the external environment recognition device 50 causes the lidar 5 to detect the road surface conditions in the direction of travel of the vehicle 101. Alternatively, for example, the vehicle 101 may be equipped with a lidar 5 having a field of view (FOV) capable of detecting a 360-degree area around it, and the lidar 5 may be configured to detect the road surface conditions all around the vehicle 101.

[0077] The above description is merely an example, and the present invention is not limited by the embodiments and modifications described above, as long as they do not impair the features of the present invention. It is also possible to combine the above embodiments and modifications. [Explanation of Symbols]

[0078] 1 Communication unit, 2 Positioning unit, 3 Internal sensor group, 4 Camera, 5 LiDAR, 10 Controller, 11 Calculation unit, 12 Memory unit, 50 External environment recognition device, 100 Vehicle control device, 101 Own vehicle, 111 Recognition unit, 112 Setting unit, 113 Decision unit, 114 Prediction unit, 115 Driving control unit, AC actuator

Claims

1. An on-board detector scans and irradiates electromagnetic waves in a first direction and a second direction intersecting the first direction within the field of view, and acquires point cloud data, including three-dimensional position information of detection points on the surface of objects based on reflected waves from objects surrounding the vehicle, frame by frame. A recognition unit recognizes the road surface and three-dimensional objects on the road through which the vehicle is traveling as road surface information based on the point cloud data for each frame, A determination unit determines the interval of the detection points required for the point cloud data of the next frame, based on the size of a predetermined three-dimensional object defined in advance as the object to be recognized and the measured distance from the vehicle to the three-dimensional object based on the point cloud data. The system includes a gradient prediction unit that, when the furthest distance to the road surface in the direction of travel of the vehicle recognized by the recognition unit is shorter than the required distance based on the vehicle speed of the vehicle, predicts the gradient of the road surface, which has not been recognized by the recognition unit, based on gradient information associated with map information on which the road is recorded. The determination unit further determines, when the gradient is predicted by the gradient prediction unit, the interval between the detection points required for the point cloud data of the next frame, based on the predetermined size of the three-dimensional object and the estimated distance from the vehicle to the three-dimensional object estimated from the map information and the gradient, within the range from the furthest distance to the required distance. An external environment recognition device characterized by the following features.

2. In the external environment recognition device according to claim 1, A position detection unit that detects the position of the vehicle based on information from a satellite, The system further includes an adjustment unit that relatively shifts the position of the vehicle detected by the position detection unit with the positions of the map information and the gradient information. An external environment recognition device characterized by the following features.

3. In the external environment recognition device according to claim 2, The gradient information is created based on the height of the road surface measured by the vehicle-mounted detector of the vehicle itself and / or other vehicles. The adjustment unit shifts the position of the gradient map obtained from the map information and the gradient information relative to the position of the vehicle so that the difference between the height of the road surface in a predetermined section including the furthest distance of the road based on the point cloud data and the height of the road surface in the section corresponding to the furthest distance of the road surface for which the gradient is predicted by the gradient prediction unit is less than or equal to a predetermined value. An external environment recognition device characterized by the following features.

4. In the external environment recognition device according to claim 1, The determination unit further determines the interval between the detection points required for the point cloud data of the next frame as the scanning angle resolution of the electromagnetic wave, and makes the scanning angle resolution coarser as the scanning irradiation target of the electromagnetic wave in the field of view of the vehicle detector becomes farther than the required distance. An external environment recognition device characterized by the following features.

5. In the external environment recognition device according to claim 4, The determination unit further reduces the scanning angle resolution as the scanning irradiation target of the electromagnetic wave within the field of view of the on-board detector approaches the required distance. An external environment recognition device characterized by the following features.

6. In the external environment recognition device according to any one of claims 1 to 5, An external environment recognition device characterized in that the on-board detector is a lidar.

Images