Object detection device

Abstract

To accurately detect a mobile object, while reducing a processing load.SOLUTION: An object detection device 50 includes: a rider 5; a data acquisition part 111 for acquiring point group data indicating a detection result of the rider 5 for each predetermined time, including first speed information indicating positional information of measurement points of the surface of an object and relative movement speed of the measurement points; an estimation part 112 for acquiring second speed information indicating absolute movement speed of a mobile body; a speed calculation part 113 for calculating absolute movement speed of each of the plurality of measurement points on the basis of the first speed information and the second speed information; a classification part 114 for classifying the point group data into movement point group data corresponding to the measurement points in which the absolute value of the absolute movement speed is equal to or higher predetermined speed, and quiescent point group data other than the movement point group data; an extraction part 115 for extracting, from the quiescent point group data, change point group data corresponding to the measurement points in which the movement amount is equal to or higher than a predetermined threshold for a predetermined time; and a detection part 116 for detecting an object around the mobile body, on the basis of the movement point group data and the change point group data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an object detection device that detects objects around a vehicle. [Background technology]

[0002] As this type of device, a device that detects moving objects from three-dimensional point cloud data acquired by a lidar using machine learning is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-027736 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in a method using machine learning like the device described in Patent Document 1, the detection accuracy depends on the reliability of the learning model, so there is a risk that the detection accuracy of moving objects may not be sufficiently guaranteed. [Means for solving the problem]

[0005] An object detection device according to one aspect of the present invention includes a detector mounted on a moving body, which irradiates electromagnetic waves around the moving body and detects the external environment around the moving body based on the reflected waves; a data acquisition unit that acquires point cloud data for each predetermined time period indicating the detection results of the detector, including position information of measurement points on the surface of the object from which reflected waves are obtained and first speed information indicating the relative movement speed of the measurement points; a speed acquisition unit that acquires second speed information indicating the absolute movement speed of the moving body; a calculation unit that calculates the absolute movement speed of each of a plurality of measurement points corresponding to the point cloud data based on the first speed information and the second speed information; a classification unit that classifies the point cloud data into moving point cloud data corresponding to measurement points for which the absolute value of the absolute movement speed calculated by the calculation unit is equal to or greater than a predetermined speed, and still point cloud data other than the moving point cloud data; an extraction unit that calculates the amount of movement of the still point cloud data over a predetermined time period, and extracts from the still point cloud data changed point cloud data corresponding to measurement points for which the amount of movement is equal to or greater than a predetermined threshold; and a detection unit that detects objects around the moving body based on the moving point cloud data and the changed point cloud data. Effect of the Invention

[0006] According to the present invention, it is possible to accurately detect a moving object while reducing the processing load. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a main part of a vehicle control device including an object detection device according to an embodiment of the present invention. [Diagram 2] 1 is a diagram for explaining the relationship between the relative movement speed of a moving object measured by a lidar and the movement direction of the moving object. FIG. [Diagram 3] 4 is a flowchart showing an example of processing executed by a CPU of the controller of FIG. 1; [Figure 4] FIG. 1 is a diagram illustrating an example of point cloud data acquired by a lidar. [Figure 5A] FIG. 13 is a diagram illustrating an example of still point cloud data of a previous frame. [Figure 5B] FIG. 4 is a diagram illustrating an example of still point cloud data of a current frame. [Figure 6A]11 is a diagram showing a schematic diagram of a state in which still point cloud data of a previous frame and still point cloud data of a current frame are aligned with each other; FIG. [Figure 6B] 13A and 13B are diagrams showing still point cloud data of a previous frame and still point cloud data of a current frame after alignment. [Figure 7] FIG. 11 is a diagram illustrating another example of point cloud data acquired by a lidar. [Figure 8] FIG. 13 is a diagram showing how change points are extracted from still point cloud data. [Figure 9] FIG. 4 is a diagram for explaining a movement vector of a moving object. [Figure 10] FIG. 11 is a diagram showing an example of information indicating a detection result of a moving object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An object detection device according to an embodiment of the present invention can be applied to a vehicle having an automatic driving function, that is, an automatic driving vehicle. The vehicle to which the object detection device according to the present embodiment is applied may be called the host vehicle to distinguish it from other vehicles. The host vehicle may be any of an engine vehicle having an internal combustion engine (engine) as a driving source, an electric vehicle having a driving motor as a driving source, and a hybrid vehicle having an engine and a driving motor as driving sources. The host vehicle can run not only in an automatic driving mode in which no driving operation by the driver is required, but also in a manual driving mode in which the driver operates the vehicle.

[0009] When an autonomous vehicle is traveling in an autonomous driving mode (hereinafter referred to as autonomous driving or self-driving), it recognizes the external environment around the vehicle based on detection data from on-board detectors such as cameras and LiDAR (Light Detection and Ranging). Based on the recognition results, the autonomous vehicle generates a traveling trajectory (target trajectory) for a predetermined time ahead from the current time, and controls the traveling actuators so that the vehicle travels along the target trajectory.

[0010] 1 is a block diagram showing the main configuration of a vehicle control device 100 including an object detection device. The vehicle control device 100 has a controller 10, a communication unit 1, a positioning unit 2, an internal sensor group 3, a camera 4, a lidar 5, and an actuator AC for driving. The vehicle control device 100 also has an object detection device 50 that constitutes a part of the vehicle control device 100. The object detection device 50 detects objects around the vehicle based on detection data from the lidar 5.

[0011] The communication unit 1 communicates with various servers (not shown) via a network including a wireless communication network such as the Internet network or a mobile phone network, and acquires map information, driving history information, traffic information, and the like from the server periodically or at any timing. The network includes not only a public wireless communication network but also a closed communication network provided for each predetermined management area, such as a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like. The acquired map information is output to the storage unit 12, and the map information is updated. The positioning unit (GNSS unit) 2 has a positioning sensor that receives a positioning signal transmitted from a positioning satellite. The positioning satellite is an artificial satellite such as a GPS satellite or a quasi-zenith satellite. The positioning unit 2 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

[0012] The internal sensor group 3 is a collective term for a plurality of sensors (internal sensors) that detect the driving state of the vehicle. For example, the internal sensor group 3 includes a vehicle speed sensor that detects the vehicle speed of the vehicle, an acceleration sensor that detects the acceleration in the forward and backward directions and the acceleration in the left and right directions (lateral acceleration) of the vehicle, a rotation speed sensor that detects the rotation speed of the driving source, a yaw rate sensor that detects the rotation angular velocity around the vertical axis of the center of gravity of the vehicle, etc. The internal sensor group 3 also includes sensors that detect the driving operations of the driver in the manual driving mode, such as the operation of the accelerator pedal, the operation of the brake pedal, the operation of the steering wheel, etc.

[0013] The camera 4 has an imaging element such as a CCD or a CMOS, and captures the surroundings (front, rear, and sides) of the vehicle. The lidar 5 irradiates electromagnetic waves (reflected waves) into a three-dimensional space around the vehicle and detects the external situation around the vehicle based on the reflected waves. More specifically, the electromagnetic waves (laser light, etc.) irradiated by the lidar 5 are reflected at a certain point (measurement point) on the surface of the object and return, thereby measuring the distance from the laser source to that point, the intensity of the reflected electromagnetic waves, the relative speed of the object located at that measurement point, and the like. The electromagnetic waves of the lidar 5 attached to a predetermined position (front) of the vehicle are scanned in the horizontal and vertical directions around the vehicle (front), thereby detecting the position, shape, relative movement speed, and the like of an object in front of the vehicle (moving objects such as other vehicles and stationary objects such as road surfaces and structures). In the following, the above three-dimensional space is represented by an X-axis along the traveling direction of the vehicle, a Y-axis along the width direction of the vehicle, and a Z-axis along the height direction of the vehicle. Therefore, the above three-dimensional space is sometimes called an XYZ space.

[0014] The actuators AC are driving actuators for controlling the driving of the host vehicle. When the driving source for driving is an engine, the actuators AC include a throttle actuator that adjusts the opening of a throttle valve (throttle opening) of the engine. When the driving source for driving is a driving motor, the actuators AC include the driving motor. The actuators AC also include a brake actuator that operates the braking device of the host vehicle and a steering actuator that drives the steering device.

[0015] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 includes a computer having a calculation unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM, a RAM, and other peripheral circuits (not shown) such as an I / O interface. Note that although a plurality of ECUs with different functions, such as an engine control ECU, a travel motor control ECU, and a braking device ECU, can be provided separately, the controller 10 is shown in FIG. 1 as a collection of these ECUs for the sake of convenience.

[0016] The memory unit 12 stores highly accurate, detailed map information (referred to as high-accuracy map information). The high-accuracy map information includes road position information, road shape (curvature, etc.) information, road gradient information, intersection and branch point position information, number of lanes (driving lanes) information, lane width and position information for each lane (information on lane center positions and lane position boundary lines), position information of landmarks (traffic lights, signs, buildings, etc.) as markers on the map, and road surface profile information such as road surface unevenness. The memory unit 12 also stores various control programs, information such as thresholds used in the programs, and setting information for vehicle-mounted detectors such as the LIDAR 5.

[0017] The calculation unit 11 has, as functional components, a data acquisition unit 111, an estimation unit 112, a speed calculation unit 113, a classification unit 114, an extraction unit 115, a detection unit 116, a determination unit 117, a vector calculation unit 118, and a driving control unit 119. As shown in Fig. 1, the data acquisition unit 111, the estimation unit 112, the speed calculation unit 113, the classification unit 114, the extraction unit 115, the detection unit 116, the determination unit 117, and the vector calculation unit 118 are included in the object detection device 50. Details of the data acquisition unit 111, the estimation unit 112, the speed calculation unit 113, the classification unit 114, the extraction unit 115, the detection unit 116, the determination unit 117, and the vector calculation unit 118 included in the object detection device 50 will be described later.

[0018] In the automatic driving mode, the driving control unit 119 generates a target trajectory based on the external conditions around the vehicle including the size, position, relative moving speed, etc. of an object detected by the object detection device 50. Specifically, the driving control unit 119 generates a target trajectory based on the size, position, relative moving speed, etc. of the object detected by the object detection device 50 so as to avoid collision or contact with the object or to follow the object. The driving control unit 119 controls the actuator AC so that the host vehicle travels along the target trajectory. Specifically, the actuator AC is controlled along the target trajectory to adjust the accelerator opening and drive the braking device and the steering device. In the manual driving mode, the driving control unit 119 controls the actuator AC according to a driving command (such as a steering operation) from the driver acquired by the internal sensor group 3.

[0019] The object detection device 50 will be described in detail. As described above, the object detection device 50 includes the data acquisition unit 111, the estimation unit 112, the speed calculation unit 113, the classification unit 114, the extraction unit 115, the detection unit 116, the determination unit 117, and the vector calculation unit 118. The object detection device 50 further includes a LIDAR 5.

[0020] The data acquisition unit 111 acquires four-dimensional data (hereinafter referred to as point cloud data) including position information indicating three-dimensional position coordinates of measurement points on the surface of an object from which reflected waves of the LIDAR 5 are obtained, and speed information indicating the relative movement speed of the measurement points, as detection data of the LIDAR 5. The point cloud data is acquired by the LIDAR 5 on a frame-by-frame basis, specifically, at predetermined time intervals (at time intervals determined by the frame rate of the LIDAR 5).

[0021] As a method for detecting a moving object from the detection data (point cloud data) of the lidar 5, there is a method for detecting a moving object by performing a clustering process on the point cloud data and classifying the point cloud data into measurement points corresponding to moving objects and other measurement points. However, the point cloud data also includes information on measurement points corresponding to stationary objects. Therefore, if the point cloud data acquired from the lidar 5 is used directly in the clustering process, not only the measurement point clouds corresponding to moving objects but also the measurement point clouds corresponding to stationary objects are subject to classification, so that the calculation load in the clustering process may increase. Therefore, in order to reduce the calculation load, a method is conceivable in which the relative movement speed indicated by the speed information of each measurement point is converted into an absolute speed (hereinafter referred to as absolute movement speed), the point cloud data is classified into stationary point cloud data corresponding to stationary objects and moving point cloud data corresponding to moving objects based on the absolute movement speed, and the clustering process is performed on the classified moving point cloud data.

[0022] However, depending on the moving direction of the moving object, there are cases where the relative moving speed of the moving object cannot be measured by the LIDAR 5. FIG. 2 is a diagram for explaining the relationship between the relative moving speed of the moving object measured by the LIDAR 5 and the moving direction of the moving object. In FIG. 2, the outlined arrows show how the moving objects M1 and M2 move in the directions of the arrows. The solid arrows show typically how the electromagnetic waves irradiated by the LIDAR 5 are reflected by the surfaces of the moving objects M1 and M2 and returned. The dashed lines CL1 are imaginary lines connecting points equidistant from the LIDAR 5 mounted on the host vehicle 101.

[0023] The lidar 5 cannot detect the velocity in the direction perpendicular to the projection angle. Therefore, when the moving objects M1 and M2 move on the dashed line CL1 as shown in FIG. 2, the moving direction is perpendicular to the projection angle of the lidar 5, so the relative moving velocity on the moving objects M1 and M2 measured by the lidar 5 remains zero and does not change. As a result, the moving objects M1 and M2 are detected as stationary objects rather than moving objects, and the measurement points corresponding to the moving objects M1 and M2 are classified into stationary point cloud data. Therefore, in a method of performing clustering processing only on the moving point cloud data, there is a risk that the moving objects M1 and M2 moving in the direction perpendicular to the projection angle of the lidar 5 will be excluded from the clustering processing. In this case, the moving objects M1 and M2 will be lost and the moving objects M1 and M2 will not be tracked accurately. Therefore, in this embodiment, the clustering processing on the point cloud data is performed as follows so that the moving objects can be detected accurately while reducing the calculation load.

[0024] Fig. 3 is a flowchart showing an example of processing executed by the CPU of the controller 10 in Fig. 1 in accordance with a predetermined program. The processing shown in this flowchart is repeated at a predetermined cycle while the object detection device 50 is running. More specifically, the processing is repeated every time the data acquisition unit 111 acquires detection data of the lidar 5, that is, at predetermined time intervals.

[0025] When the detection data (point cloud data) of the lidar 5 is acquired, first, a process (step S1) is executed to classify the point cloud data into moving point cloud data and stationary point cloud data. Next, a process (step S2) of detecting a moving object for the moving point cloud data and a process (steps S31 to S33) of detecting a moving object for the stationary point cloud data are executed in parallel. Note that the detection process (step S2) and the detection process (steps S31 to S33) may not be executed in parallel, and one of them may be executed before the other. Next, a process (step S4) of identifying the same object between frames (between the current frame and the previous frame) is executed for the detected moving object, and finally, a process (step S5) of calculating the movement vector of the moving object identified as the same object is executed.

[0026] In the moving object detection process for the moving point cloud data (step S2), a clustering process is performed on the moving point cloud data. In the moving object detection process for the stationary point cloud data (steps S31 to S33), first, in step S31, a predetermined scan matching process is performed to superimpose the stationary point cloud data of the previous frame on the stationary point cloud data of the current frame, thereby estimating the azimuth angle difference and the movement vector of the vehicle 101. The movement vector represents the movement direction between frames of the representative point (center of gravity, etc.) of the vehicle 101 and the movement speed in the movement direction. The azimuth angle difference is the angle difference between the azimuth in the current frame and the azimuth (traveling direction) of the vehicle 101 in the previous frame. In the following, the axis along the traveling direction of the moving object is the X-axis, and the horizontal and height directions with respect to the traveling direction are the Y-axis and the Z-axis, respectively. The movement vector may be two-dimensional (X, Y) or three-dimensional (X, Y, Z). The azimuth angle difference may be a one-axis angle (Z-axis rotation angle), a two-axis angle (X-axis rotation angle, Z-axis rotation angle), or a three-axis angle (X-axis rotation angle, Y-axis rotation angle, Z-axis rotation angle). The scan matching process may use ICP (Iterative Closest Point) or NDT (Normal Distributions Transform), or other methods. Next, in step S32, the still point cloud data of the previous frame and the still point cloud data of the current frame are superimposed by the scan matching process in step S31. The distance difference value of each measurement point of the previous frame and each measurement point of the current frame that correspond in the superposition is calculated, and the measurement point whose distance difference value is equal to or greater than a predetermined threshold value is extracted as a change point. Finally, in step S33, a clustering process is performed on the extracted measurement point group (change point group). The azimuth angle difference and movement vector of the vehicle 101 estimated in step S31 are accumulated, and a self-position estimation process (not shown) is executed to estimate the vehicle's position (the traveling position of the vehicle 101) based on the accumulated azimuth angle difference and movement vector.

[0027] The process at each step in FIG. 3 will now be described in detail. <Point Cloud Classification (S1)> The estimation unit 112 estimates the absolute moving speed of the host vehicle 101 based on the point cloud data acquired by the data acquisition unit 111. Here, the estimation of the absolute moving speed of the host vehicle 101 by the estimation unit 112 will be described.

[0028] The estimation unit 112 extracts point cloud data from the point cloud data acquired by the data acquisition unit 111, excluding information on measurement points corresponding to three-dimensional objects, that is, point cloud data corresponding to the road surface around the vehicle 101 (hereinafter referred to as road surface point cloud data). i 4-dimensional data (x i , y i ,z i ,v i ) i , y i ,z i ) based on the relative movement speed v i The unit vector e indicating the direction of i Specifically, the estimation unit 112 calculates the unit vector e i Calculate.

[0029]

number

[0030] Next, the estimation unit 112 estimates the moving speed (absolute moving speed) V self Specifically, the estimation unit 112 estimates the measurement point P i The relative movement speed v i The transformation equation for converting the measured point P into the absolute moving speed is set as the objective function L, and the optimization problem is solved to optimize the objective function L so that it approaches zero. i are measurement points on the road surface, the absolute speed of each measurement point should be zero. Therefore, by optimizing the objective function L to approach zero, the correct V self can be estimated. V selfis expressed by the velocity components in the XYZ axis directions as shown in the following equation (ii). The objective function L is expressed by the following equation (iii). By solving the above optimization problem, V self is searched. Note that V self The initial value of may be set to zero, or it may be set to the estimated value of V self may be set.

[0031]

number

number

[0032] In equation (iii), A is the unit vector e of n measurement points corresponding to the road surface. i In addition, in the equation (iii), V is a matrix of n measurement points P i is a 1×n matrix representing the velocity components (relative movement velocities) of the vectors V, V, and V are expressed by the formula (v). The estimation unit 112 estimates V obtained by solving the optimization problem. self is obtained as an estimate of the absolute moving speed of the host vehicle 101 in the current frame.

[0033]

number

number

[0034] The speed calculation unit 113 calculates the absolute moving speed V of the vehicle 101 estimated by the estimation unit 112. self Based on the above, the absolute moving speed of all the measurement points, more specifically, all the measurement points including the measurement points corresponding to the three-dimensional object, is calculated. Here, the calculated absolute moving speed has a negative value when approaching the host vehicle, and has a positive value when moving away.

[0035] The classification unit 114 classifies the point cloud data acquired by the data acquisition unit 111 into moving point cloud data corresponding to measurement points where the absolute value of the absolute moving speed calculated by the speed calculation unit 113 is equal to or greater than a predetermined speed Th_V, and still point cloud data corresponding to measurement points where the absolute value is less than the predetermined speed Th_V.

[0036] <Estimation of movement vector (S31)> 4, 5A, 5B, 6A, and 6B are diagrams for explaining the process (step S31) of estimating the movement vector and azimuth angle of the vehicle 101 between frames. In FIG. 4, the arrow attached to the vehicle 101 indicates the movement direction (traveling direction) of the vehicle 101, that is, the X-axis direction. The lateral direction (left and right direction in FIG. 4) and the height direction (direction from back to front in FIG. 4) with respect to the X-axis direction indicate the Y-axis direction and the Z-axis direction. Note that the definitions of the X-axis direction, Y-axis direction, and Z-axis direction are similar in other figures, so the notation of the X-axis, Y-axis, and Z-axis is omitted in those figures. FIG. 4 shows an example of a schematic diagram of point cloud data acquired by the lidar 5 at a past time point (time t1) viewed from above (Z-axis direction). Regions N1 to N6 typically indicate measurement point clouds corresponding to stationary objects, more specifically, the positions and sizes of the measurement point clouds. The stationary objects include the road surface on which the vehicle 101 is traveling, structures such as walls and median strips installed on the sides of the road, and other vehicles parked on the road shoulders. The areas M31 and M32 represent the measurement point clouds corresponding to the moving objects, more specifically, the positions and sizes of the measurement point clouds. In the following, the objects detected by the lidar 5 are referred to as objects, including people. Therefore, the moving objects include not only vehicles such as moving cars and bicycles, but also moving people (pedestrians, etc.). The arrows attached to the areas M31 and M32 represent the moving directions of the moving objects.

[0037] Fig. 5A shows a schematic diagram of stationary point cloud data classified from the point cloud data of Fig. 4, i.e., point cloud data acquired by the LIDAR 5 at a past time (time t1). Fig. 5B shows a schematic diagram of stationary point cloud data classified from the point cloud data acquired by the LIDAR 5 at a present time (time t2) a predetermined time after a predetermined time in the past (time t1).

[0038] The estimation unit 112 first performs alignment between the still point cloud data of the previous frame (FIG. 5A) and the still point cloud data of the current frame (FIG. 5B) to estimate the azimuth angle difference and the movement vector of the vehicle 101 at a predetermined time (between frames). FIGS. 6A and 6B are schematic diagrams showing how the still point cloud data of the previous frame (FIG. 5A) and the still point cloud data of the current frame (FIG. 5B) are aligned. In FIG. 6A, the still point cloud data of the previous frame is shown by a broken line, and the still point cloud data of the current frame is shown by a solid line. The estimation unit 112 searches (estimates) the azimuth angle difference and the movement vector of the vehicle 101 such that the measurement point clouds N1 to N6 of the previous frame shown by the broken line overlap (match) with the measurement point clouds N1 to N6 of the current frame shown by the solid line. FIG. 6B shows the still point cloud data of the previous frame and the still point cloud data of the current frame after alignment. In FIG. 6B, angle MA represents the azimuth angle difference of the host vehicle 101 between frames. An open arrow MV represents the movement vector of the host vehicle 101 between frames. The open circle in the figure typically represents the center of gravity of the host vehicle 101. The estimation unit 112 solves an optimization problem that minimizes the deviation (error) between the positions of the measurement point group N1 to N6 of the previous frame after alignment and the positions of the measurement point group N1 to N6 of the current frame, and outputs a final search result (estimation result) of the azimuth angle difference and movement vector of the host vehicle 101. Note that, in order to reduce the calculation load, the above alignment may be performed after converting the three-dimensional point cloud data into two-dimensional point cloud data represented in an XY coordinate system.

[0039] <Extraction of change points (S32)> 7 and 8 are diagrams for explaining the extraction process of the change point (step S32). FIG. 7 shows another example of a schematic diagram of the point cloud data acquired by the LIDAR 5 at a past time point (time t1) viewed from above (Z-axis direction). The point cloud data shown in FIG. 7 is similar to the point cloud data shown in FIG. 4, but the point cloud data shown in FIG. 7 includes a measurement point cloud (area M33) corresponding to a moving object moving in a direction perpendicular to the projection angle of the LIDAR 5. The shaded area in FIG. 7 represents a measurement point cloud classified as stationary point cloud data. As described above, the LIDAR 5 cannot detect the speed in the direction perpendicular to the projection angle, so that the object corresponding to the area M33 is erroneously detected as a stationary object rather than a moving object, as shown in FIG. 7. Therefore, in order to detect the object corresponding to the area M33 as a moving object, the extraction unit 115 calculates the amount of change (amount of movement) of the position of each measurement point of the stationary point cloud data between frames (a predetermined time), and extracts the measurement point whose amount of change is equal to or greater than a predetermined threshold as a change point. FIG. 8 shows how change points are extracted from still point cloud data. In FIG. 8, the still point cloud data of the previous frame (FIG. 7) is shown by a dashed line, and the still point cloud data of the current frame is shown by a solid line. FIG. 8 also shows a state in which the point cloud data of the previous frame is aligned with the point cloud data of the current frame using the movement vector and azimuth angle difference of the host vehicle estimated by the estimation unit 112. As shown in FIG. 8, the position of the measurement point group M33 corresponding to the moving object erroneously detected as a still object changes significantly between frames compared with the other measurement point groups N1 to N6. Therefore, each measurement point of the measurement point group M33 is extracted as a change point by the extraction unit 115. Hereinafter, the point cloud data corresponding to the change points (change point group) extracted by the extraction unit 115 is called change point group data.

[0040] <Clustering process (S2, S33)> In step S2, the detection unit 116 executes a clustering process on the moving point cloud data (moving point cloud data classified from the point cloud data of the current frame in step S1). In addition, in step S33, the detection unit 116 executes a clustering process on the change point cloud data extracted from the still point cloud data (still point cloud data classified from the point cloud data of the current frame in step S1). As a result, bounding boxes (circumscribed areas) corresponding to each of the measurement point clouds M31 and M32 are detected from the moving point cloud data, and a bounding box corresponding to the measurement point cloud M33 is detected from the change point cloud data. The detection unit 116 detects the position and size of the detected bounding box as the position and size of the moving object. In this way, the moving object included in the three-dimensional space around the host vehicle 101 is detected. The detection unit 116 outputs information (such as image information) indicating the detection result of the moving object to a display device (not shown) or the like. The clustering process by the detection unit 116 may use any method, such as density-based spatial clustering of applications with noise (DBSCAN) or the K-means method.

[0041] <Identification of the same object (S4)>

[0042] The determination unit 117 determines whether or not the moving object detected in the previous frame and the moving object detected in the current frame are the same object.

[0043] Specifically, first, the determination unit 117 executes an offset rotation process to offset (translate) and rotate each bounding box detected by the clustering process by the detection unit 116 in the previous frame based on the movement vector and azimuth angle difference of the host vehicle 101 between frames estimated in step S31. In the offset rotation process, the determination unit 117 first offsets each bounding box detected in the previous frame according to the movement vector and rotates it by the azimuth angle difference. Next, the determination unit 117 further offsets each bounding box that has been offset and rotated based on the movement vector of the corresponding moving object. Specifically, the bounding boxes of the measurement point groups M31, M32, and M33 are further offset by the movement amount obtained by multiplying the vector amount of the movement vector of the moving object corresponding to the measurement point groups M31, M32, and M33 by the frame time (predetermined time). The movement vector of the moving object will be described later.

[0044] As described above, the determination unit 117 superimposes each bounding box of the previous frame, which has been subjected to the offset rotation process based on the movement vector and azimuth angle difference of the vehicle 101, and the offset process based on the movement vector of the moving object, on the current frame. When a bounding box that overlaps with the bounding box of the previous frame that has been subjected to the offset rotation process and the offset process exists in the current frame as a result of the superposition, the determination unit 117 determines that the moving objects corresponding to the overlapping bounding boxes are the same object. Note that the determination of whether the bounding boxes overlap each other may be performed based on whether the overlap rate is equal to or greater than a predetermined threshold, or based on whether the distance between the centers of gravity of the bounding boxes is less than a predetermined length.

[0045] <Calculation of movement vector (S5)> The vector calculation unit 118 calculates the movement vector of the moving object based on the determination result of the determination unit 117. The calculation of the movement vector of the moving object will be described with reference to FIG. 9. FIG. 9 is a diagram for explaining the movement vector of the moving object. In FIG. 9, the point cloud data of the previous frame is indicated by a broken line, and the point cloud data of the current frame is indicated by a solid line. FIG. 9 also shows a state in which the point cloud data of the previous frame, which has been subjected to an offset rotation process based on the movement vector and azimuth angle difference of the vehicle 101, and the point cloud data of the current frame are superimposed. The white circles G1, G2, and G3 represent the centers of gravity of the measurement point groups M31, M32, and M33, more specifically, the centers of gravity of the bounding boxes of the measurement point groups M31, M32, and M33 detected by the clustering process by the detection unit 116. The vector calculation unit 118 calculates the movement vector of the moving object corresponding to the measurement point groups M31, M32, M33 based on the positional relationship between the centers of gravity G1, G2, G3 between frames (between the previous frame and the current frame). The vector calculation unit 118 stores the calculated movement vector of the moving object in the storage unit 12 together with information (identifier) ​​capable of identifying the moving object. The movement vector of the moving object stored in the storage unit 12 is used in the offset process based on the movement vector of the next moving object, i.e., the offset process in step S4 executed for the next frame. This allows the same object to be identified with high accuracy in the next frame, and the position, moving direction, and moving speed of the moving object to be tracked well.

[0046] FIG. 10 is a diagram showing an example of information indicating a detection result of a moving object output by the detection unit 116. Note that FIG. 10 shows image information in which bounding boxes BB1 to BB5 of moving objects (pedestrians P1 to P5 moving in different directions) detected by the clustering process by the detection unit 116 are superimposed on the captured image IM acquired by the camera 4 as information indicating the detection result of the moving object. The captured image IM of the camera 4 includes stationary objects such as buildings in addition to the pedestrians P1 to P5, but these objects are not depicted in FIG. 10 to simplify the drawing. Note that instead of the captured image IM of the camera 4, image information in which bounding boxes of moving objects are superimposed on an image showing detection data of the lidar 5 may be output as information indicating the detection result of the moving object.

[0047] FIG. 10 shows the detection results of moving objects from time t11 to time t13. The dashed line CL2 in the figure is a virtual line connecting points equidistant from the LIDAR 5. The dashed line CL2 is for the purpose of explanation and is not actually included in the detection results of moving objects. In this embodiment, as described above, the clustering process is also performed on the change point cloud data extracted from the stationary point cloud data, so that moving objects around the vehicle 101 can be tracked without losing sight of them, including moving objects that move perpendicularly to the projection angle of the LIDAR 5, such as the pedestrian P1 in FIG. 10.

[0048] According to the embodiment described above, the following advantageous effects are achieved. (1) The object detection device 50 is mounted on the host vehicle 101 and includes a LIDAR 5 that irradiates electromagnetic waves around the host vehicle 101 and detects the external environment around the moving object based on the reflected waves, a data acquisition unit 111 that acquires point cloud data for each predetermined time period that indicates the detection results of the LIDAR 5, including position information of measurement points on the surface of the object from which the reflected waves are obtained and speed information (referred to as first speed information) that indicates the relative moving speed of the measurement points, an estimation unit 112 that acquires speed information (referred to as second speed information) that indicates the absolute moving speed of the host vehicle 101, and an estimation unit 112 that estimates a plurality of measurement points corresponding to the point cloud data based on the first speed information and the second speed information. The apparatus includes a speed calculation unit 113 that calculates the absolute moving speed of each point, a classification unit 114 that classifies the point cloud data into moving point cloud data corresponding to measurement points where the absolute value of the absolute moving speed calculated by the speed calculation unit 113 is equal to or greater than a predetermined speed, and still point cloud data other than the moving point cloud data, an extraction unit 115 that calculates the amount of movement of the still point cloud data at a predetermined time and extracts, from the still point cloud data, change point cloud data corresponding to measurement points where the amount of movement is equal to or greater than a predetermined threshold, and a detection unit 116 that detects moving objects around the moving body based on the moving point cloud data and the change point cloud data. The detection unit 116 executes a predetermined clustering process on the moving point cloud data and the change point cloud data to detect a circumscribing area of ​​the object from each of the moving point cloud data and the change point cloud data, and detects the position and size of the object based on the position and size of the circumscribing area. This allows the moving object to be detected with high accuracy while reducing the processing load. In addition, a moving object that moves in a direction perpendicular to the projection angle of the lidar 5 can also be detected with high accuracy.

[0049] (2) The object detection device 50 includes an estimation unit 112 as an information acquisition unit that acquires first movement information indicating the movement state of the vehicle 101 from a past time to the present time based on still point cloud data at the current time and still point cloud data at a time prior to the present time, a memory unit 12 that stores second movement information indicating the movement state of a moving object from a past time to the present time, and a detection unit 116 that detects a circumscribing area (first circumscribing area) from the movement point cloud data and change point cloud data at a past time based on the first movement information acquired by the estimation unit 112 and the second movement information stored in the memory unit 12. The vehicle 101 includes a determination unit 117 that performs alignment to superimpose the current movement point cloud data and the change point cloud data on a circumscribing area (second circumscribing area) detected by the detection unit 116, and determines whether or not the moving object corresponding to the first circumscribing area and the moving object corresponding to the second circumscribing area are the same object based on the degree of overlap between the first circumscribing area and the second circumscribing area after the alignment, and a vector calculation unit 118 as an update unit that updates the second movement information corresponding to the moving object determined to be the same object by the determination unit 117, stored in the storage unit 12, based on the positions of the first circumscribing area and the second circumscribing area. As a result, even if the moving objects around the vehicle 101 include a moving object that moves in a direction perpendicular to the light projection angle of the lidar 5, the moving objects around the vehicle 101 can be tracked accurately without losing sight of them.

[0050] (3) The estimation unit 112 estimates the absolute moving speed of the host vehicle 101 based on the position information and speed information of the representative measurement point extracted from the point cloud data acquired by the data acquisition unit 111, and acquires the estimation result as speed information. The representative measurement point is selected from the remaining measurement points excluding the measurement points corresponding to the three-dimensional object from the multiple measurement points. In this way, the absolute moving speed of the host vehicle 101 is estimated based on the measurement point corresponding to the road surface. As a result, the absolute moving speed of the host vehicle 101 can be estimated with high accuracy. In addition, since the absolute moving speed of the host vehicle (moving body) 101 is estimated and acquired without relying on sensor values ​​such as a vehicle speed sensor, the present invention can also be applied to a self-propelled robot that does not have a vehicle speed sensor, etc.

[0051] The above embodiment can be modified into various forms. Modifications will be described below. In the above embodiment, the LIDAR 5 as a detector is mounted on a vehicle, and the LIDAR 5 irradiates electromagnetic waves into a three-dimensional space around the vehicle and detects the external environment around the vehicle based on the reflected waves. However, the detector may be something other than a LIDAR, such as a radar. In addition, the moving body on which the detector is mounted may be something other than a vehicle, such as a self-propelled robot.

[0052] In the above embodiment, the estimation unit 112 as a speed acquisition unit selects a representative measurement point P from the remaining measurement points after removing the measurement points corresponding to the three-dimensional object from the plurality of measurement points. i The speed acquisition unit 111 selects a representative measurement point and estimates the absolute moving speed of the vehicle 101 based on the position information and speed information of the representative measurement point extracted from the point cloud data acquired by the data acquisition unit 111, and acquires the estimation result as the second speed information. However, the speed acquisition unit may acquire the measurement result of the absolute moving speed of the vehicle 101 acquired by a measuring instrument included in the internal sensor group 3 as the second speed information. In this case, the object detection device 50 includes at least a vehicle speed sensor of the internal sensor group 3 as a measuring instrument. In addition, the speed acquisition unit may calculate and acquire the absolute moving speed of the vehicle 101 based on the current position of the vehicle 101 measured by the positioning unit 2. In this case, the object detection device 50 includes the positioning unit 2.

[0053] In the above embodiment, the detection unit 116 executes a predetermined clustering process on the three-dimensional point cloud data (movement point cloud data and change point cloud data) to detect a moving object from the three-dimensional point cloud data. However, the detection unit may generate speed-added data (XYV data) by adding the absolute moving speed of each measurement point calculated by the speed calculation unit 113 to two-dimensional point cloud data obtained by projecting each measurement point corresponding to the three-dimensional point cloud data onto the XY plane, and execute a predetermined clustering process on the speed-added data. This allows the clustering process to be performed in consideration of the position and moving speed of the moving object. As a result, it is possible to prevent a plurality of moving objects that are close to each other, such as two moving objects passing each other, from being detected as one (as one moving object), and the detection accuracy of the moving object can be further improved. In addition, when precision in cluster size in three-dimensional space (XYZ space) is required, the detection unit may generate velocity-added data (XYZV data) by adding the absolute movement speed of each measurement point calculated by the speed calculation unit 113 to the three-dimensional point cloud data, and perform a predetermined clustering process on the velocity-added data.

[0054] In the above embodiment, the estimation unit 112 as an information acquisition unit acquires the azimuth angle difference and the movement vector of the vehicle 101 as the first movement information indicating the movement state of the vehicle 101 from a past time point to the present time, but the information acquisition unit may acquire other information as the first movement information. In the above embodiment, the storage unit 12 stores the movement vector of the moving object as the second movement information indicating the movement state of the moving object from a past time point to the present time. However, the second movement information may be other information.

[0055] In the above embodiment, the estimation unit 112 as the information acquisition unit performs alignment between the still point cloud data of the previous frame (FIG. 5A) and the still point cloud data of the current frame (FIG. 5B) to estimate the azimuth angle difference and the movement vector of the vehicle 101 at a predetermined time (between frames). However, the information acquisition unit may convert the three-dimensional still point cloud data into two-dimensional still point cloud data represented in the XY coordinate system and then perform the above alignment. The information acquisition unit may perform the above alignment using not only the still point cloud data of a past time point (time t1) but also still point cloud data of multiple past time points, that is, using not only the still point cloud data of the previous frame but also still point cloud data of multiple past frames. In this way, by using the still point cloud data of multiple past frames, the above alignment can be performed well even when a stationary object in front of the vehicle 101 is temporarily blocked by another vehicle or the like, and robustness can be improved.

[0056] In the above embodiment, the driving control unit 119 controls the driving of the vehicle 101 so as to avoid a collision or contact with an object detected by the detection unit 116. However, the driving control unit 119 may function as a notification unit to predict the possibility of a collision or contact with a moving object based on the size, position, and moving speed of the moving object detected by the detection unit 116. When the possibility of a collision or contact with a moving object is equal to or greater than a predetermined level, the driving control unit 119 may notify the occupants of the vehicle 101 of attention-calling information (video information or audio information) regarding the collision or contact with the moving object detected by the detection unit 116 via a display and a speaker (not shown) provided in the vehicle control device 100.

[0057] Furthermore, in the above embodiment, the object detection device 50 is applied to an autonomous vehicle, but the object detection device 50 can also be applied to vehicles other than autonomous vehicles. For example, the object detection device 50 can also be applied to a manually driven vehicle equipped with an ADAS (Advanced Driver Assistance Systems).

[0058] The above description is merely an example, and the present invention is not limited to the above-mentioned embodiment and modifications, as long as the features of the present invention are not impaired. It is also possible to arbitrarily combine one or more of the above-mentioned embodiment and modifications, and it is also possible to combine modifications together. [Explanation of symbols]

[0059] 1 communication unit, 2 positioning unit, 3 internal sensor group, 4 camera, 5 lidar, 10 controller, 11 calculation unit, 12 memory unit, 111 data acquisition unit, 112 estimation unit, 113 speed calculation unit, 114 classification unit, 115 extraction unit, 116 detection unit, 117 judgment unit, 118 vector calculation unit, 119 driving control unit, 50 object detection device, 100 vehicle control device, AC actuator

Claims

1. a detector mounted on the moving body, irradiating electromagnetic waves around the moving body and detecting an external environment around the moving body based on reflected waves; a data acquisition unit that acquires point cloud data for each predetermined time period that indicates a detection result of the detector, the point cloud data including position information of a measurement point on a surface of an object from which the reflected wave is obtained and first speed information that indicates a relative moving speed of the measurement point; a speed acquisition unit that acquires second speed information indicating an absolute moving speed of the moving object; a calculation unit that calculates an absolute moving speed of each of a plurality of measurement points corresponding to the point cloud data based on the first speed information and the second speed information; a classification unit that classifies the point cloud data into moving point cloud data corresponding to the measurement points where the absolute value of the absolute moving speed calculated by the calculation unit is equal to or greater than a predetermined speed, and stationary point cloud data other than the moving point cloud data; an extraction unit that calculates a movement amount of the still point cloud data at the predetermined time and extracts, from the still point cloud data, change point cloud data corresponding to the measurement point where the movement amount is equal to or greater than a predetermined threshold; a detection unit that detects the object around the moving body based on the movement point cloud data and the change point cloud data.

2. 2. The object detection device according to claim 1, The object detection device is characterized in that the detection unit performs a predetermined clustering process on the movement point cloud data and the change point cloud data to detect a circumscribing area of ​​the object from each of the movement point cloud data and the change point cloud data, and detects the position and size of the object based on the position and size of the circumscribing area.

3. 3. The object detection device according to claim 2, an information acquisition unit that acquires first movement information indicating a movement state of the moving object from a past time point to the present time point based on the still point cloud data at a current time point and the still point cloud data at a time point earlier than the current time point; a storage unit configured to store second movement information indicating a movement state of the object from the past time point to the present time point; a determination unit that performs registration to superimpose a first circumscribing area detected by the detection unit from the movement point cloud data and the change point cloud data at the past time point on a second circumscribing area detected by the detection unit from the movement point cloud data and the change point cloud data at the current time point, based on the first movement information acquired by the information acquisition unit and the second movement information stored in the storage unit, and determines whether the object corresponding to the first circumscribing area and the object corresponding to the second circumscribing area are the same object based on a degree of overlap between the first circumscribing area and the second circumscribing area after the registration; an update unit that updates the second movement information stored in the memory unit and corresponding to the object determined to be the same object by the determination unit, based on the positions of the first circumscribing area and the second circumscribing area.

4. 2. The object detection device according to claim 1, the speed acquisition unit estimates an absolute moving speed of the moving object based on the position information of the representative measurement point and the first speed information extracted from the point cloud data acquired by the data acquisition unit, and acquires the estimation result as the second speed information; An object detection device, characterized in that the representative measurement point is selected from among the remaining measurement points excluding the measurement point corresponding to a three-dimensional object from the plurality of measurement points.

5. 2. The object detection device according to claim 1, A measuring device for measuring an absolute moving speed and a moving direction of the moving object is further provided, The object detection device, wherein the speed acquisition unit acquires the measurement result of the measuring instrument as the second speed information.

6. The object detection device according to any one of claims 1 to 5, The object detection device, characterized in that the detector is a lidar.

Images