Object detection device

Abstract

To accurately detect a moving object while reducing the processing load. [Solution] The object detection device 50 includes a lidar 5, a data acquisition unit 111 that acquires point cloud data including position information and relative movement speeds of multiple measurement points on the surface of the object from the lidar 5, an estimation unit 112 that estimates the absolute movement speed of the vehicle, a classification unit 114 that classifies the multiple measurement points into multiple moving points and multiple stationary points based on the absolute movement speeds of each of the multiple measurement points calculated based on the relative movement speeds of the multiple measurement points and the absolute movement speed of the vehicle, a filter processing unit 116 that performs kernel operations on distance data indicating the distances of the multiple moving points from the vehicle and speed data indicating the absolute movement speeds of the multiple moving points to calculate the differences in distance data and speed data between the multiple moving points, and a detection unit 117 that detects the position and size of each of multiple moving objects around the vehicle based on the results of the kernel operation.

Description

[Technical Field]

[0001] The present invention relates to an object detection device that detects objects around a vehicle. [Background technology]

[0002] As this type of device, a device that detects moving objects by using three-dimensional point cloud data acquired by a lidar is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7126633 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when detecting objects using point cloud data as in the device described in Patent Document 1, a large amount of point cloud data is input by the lidar, which may increase the processing load of the device. [Means for solving the problem]

[0005] An object detection device according to one aspect of the present invention includes a detector mounted on a moving body, which irradiates electromagnetic waves into a three-dimensional space around the moving body and detects the external environment around the moving body based on the reflected waves; a data acquisition unit which acquires point cloud data from the detector, the point cloud data including three-dimensional position information of a plurality of measurement points on the surface of the object from which the reflected waves are obtained and first velocity information indicating the relative movement speed of the plurality of measurement points; a velocity acquisition unit which acquires second velocity information indicating the absolute movement speed of the moving body; a calculation unit which calculates the absolute movement speed of each of the plurality of measurement points based on the first velocity information and the second velocity information; and a detector which detects the plurality of measurement points when the absolute value of the absolute movement speed calculated by the calculation unit is equal to or greater than a predetermined speed. The system includes a classification unit that classifies the moving points into a plurality of moving points and a plurality of stationary points moving at speeds less than a predetermined speed; a generation unit that generates distance data indicating the distances of the moving points from the moving body and speed data calculated by a calculation unit that indicates the absolute moving speeds of the moving points; a filter processing unit that performs kernel operations on each of the distance data and speed data generated by the generation unit, with each of the moving points as its center position, to calculate the differences in the distance data and speed data between the plurality of moving points; and an object detection unit that detects the position and size of each of the plurality of moving objects around the moving body based on the results of the kernel operations. [Effects of the Invention]

[0006] According to the present invention, it is possible to accurately detect a moving object while reducing the processing load on the device. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing the configuration of a main part of a vehicle control device including an object detection device according to an embodiment of the present invention; [Figure 2A] FIG. 1 is a diagram showing an example of a three-dimensional space seen from a lidar's viewpoint. [Figure 2B] FIG. 2B is a diagram showing an example of point cloud data for the three-dimensional space of FIG. 2A. [Figure 2C] FIG. 4 is a diagram showing an example of distance data. [Figure 2D] FIG. 4 is a diagram showing an example of absolute velocity data. [Figure 3A] FIG. 4 is a diagram schematically showing distance data. [Figure 3B] FIG. 10 shows an example of a kernel applied to range data. [Figure 3C] FIG. 10 shows another example of a kernel applied to distance data. [Figure 4A] 10A and 10B are diagrams showing examples of calculation results of difference calculation processing on distance data. [Figure 4B] FIG. 10 is a diagram showing an example of a calculation result of a difference calculation process on absolute velocity data. [Figure 5A] FIG. 10 is a diagram showing an example of a calculation result of the Hadamard product. [Figure 5B] FIG. 10 is a diagram showing an example of identifiers assigned to measurement points in a kernel. [Figure 6A] FIG. 10 is a diagram for explaining a process of combining moving objects. [Figure 6B] FIG. 10 is a diagram for explaining a process of combining moving objects. [Figure 7] 4 is a flowchart showing an example of processing executed by a CPU of the controller of FIG. 1; DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An object detection device according to an embodiment of the present invention can be applied to a vehicle having an automatic driving function, i.e., an automatic driving vehicle. Note that a vehicle to which an object detection device according to the present embodiment is applied may be referred to as a host vehicle to distinguish it from other vehicles. The host vehicle may be an engine vehicle having an internal combustion engine (engine) as a driving source, an electric vehicle having a driving motor as a driving source, or a hybrid vehicle having an engine and a driving motor as driving sources. The host vehicle can be driven not only in an automatic driving mode in which no driving operation by the driver is required, but also in a manual driving mode in which the driver operates the vehicle.

[0009] When a self-driving vehicle is driving in self-driving mode (hereinafter referred to as self-driving or autonomous driving), it recognizes the external environment around the vehicle based on detection data from on-board detectors such as cameras and LiDAR (Light Detection and Ranging).Based on the recognition results, the self-driving vehicle generates a driving trajectory (target trajectory) for a predetermined time from the current time, and controls driving actuators so that the vehicle drives along the target trajectory.

[0010] 1 is a block diagram showing the configuration of a main part of a vehicle control device 100 including an object detection device. The vehicle control device 100 has a controller 10, a communication unit 1, a positioning unit 2, an internal sensor group 3, a camera 4, a LIDAR 5, and a driving actuator AC. The vehicle control device 100 also has an object detection device 50 that forms part of the vehicle control device 100. The object detection device 50 detects objects around the vehicle based on detection data from the LIDAR 5.

[0011] The communication unit 1 communicates with various servers (not shown) via networks including wireless communication networks such as the Internet and mobile phone networks, and acquires map information, driving history information, traffic information, and the like from the servers periodically or at any timing. Networks include not only public wireless communication networks but also closed communication networks established for each specific management area, such as wireless LAN, Wi-Fi (registered trademark), and Bluetooth (registered trademark). The acquired map information is output to the memory unit 12, where it is updated. The positioning unit (GNSS unit) 2 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 2 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

[0012] The internal sensor group 3 is a collective term for a plurality of sensors (internal sensors) that detect the driving state of the host vehicle. For example, the internal sensor group 3 includes a vehicle speed sensor that detects the vehicle speed of the host vehicle, an acceleration sensor that detects the longitudinal acceleration and the lateral acceleration (lateral acceleration) of the host vehicle, a rotation speed sensor that detects the rotation speed of the driving source, a yaw rate sensor that detects the rotation angular velocity around the vertical axis of the center of gravity of the host vehicle, etc. The internal sensor group 3 also includes sensors that detect the driving operations of the driver in manual driving mode, such as operation of the accelerator pedal, operation of the brake pedal, operation of the steering wheel, etc.

[0013] The camera 4 has an imaging element such as a CCD or CMOS and captures images of the surroundings (front, rear, and sides) of the vehicle. The LIDAR 5 irradiates electromagnetic waves (reflected waves) into the three-dimensional space around the vehicle and detects the external environment around the vehicle based on the reflected waves. More specifically, the electromagnetic waves (laser light, etc.) irradiated by the LIDAR 5 are reflected at a point (measurement point) on the surface of an object, and the distance from the laser source to that point, the intensity of the reflected electromagnetic waves, the relative speed of the object located at that measurement point, etc. are measured. The LIDAR 5 is attached to a predetermined position (front) of the vehicle and scans the electromagnetic waves of the LIDAR 5 horizontally and vertically around the surroundings (front) of the vehicle to detect the position, shape, relative moving speed, etc. of objects ahead of the vehicle (moving objects such as other vehicles and stationary objects such as road surfaces and structures). The objects detected by the LIDAR 5, including people, are called objects. Therefore, moving objects include not only moving vehicles such as automobiles and bicycles, but also moving people (pedestrians, etc.). Note that, hereinafter, the above three-dimensional space is represented by an X axis along the traveling direction of the host vehicle, a Y axis along the width direction of the host vehicle, and a Z axis along the height direction of the host vehicle. Therefore, the above three-dimensional space may be referred to as an XYZ space.

[0014] Actuators AC are driving actuators for controlling the driving of the host vehicle. When the driving source is an engine, actuators AC include a throttle actuator that adjusts the opening of the engine's throttle valve (throttle opening). When the driving source is a driving motor, actuators AC include the driving motor. Actuators AC also include a brake actuator that operates the host vehicle's braking device and a steering actuator that drives the steering device.

[0015] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 includes a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM and a RAM, and other peripheral circuits (not shown) such as an I / O interface. Note that although multiple ECUs with different functions, such as an engine control ECU, a traction motor control ECU, and a braking device ECU, can be provided separately, for convenience, the controller 10 is shown in FIG. 1 as a collection of these ECUs.

[0016] The memory unit 12 stores highly accurate, detailed map information (referred to as high-accuracy map information). The high-accuracy map information includes road position information, road shape (curvature, etc.) information, road gradient information, intersection and branch point position information, number of lanes (driving lanes), lane width and position information for each lane (information on lane center positions and lane boundary lines), position information of landmarks (traffic lights, signs, buildings, etc.) on the map, and road surface profile information such as road surface irregularities. The memory unit 12 also stores various control programs, information such as thresholds used in the programs, and setting information for on-board detectors such as the LIDAR 5.

[0017] The calculation unit 11 has, as functional components, a data acquisition unit 111, an estimation unit 112, a calculation unit 113, a classification unit 114, a conversion unit 115, a filter processing unit 116, an object detection unit (hereinafter simply referred to as a detection unit) 117, and a driving control unit 118. As shown in FIG. 1 , the data acquisition unit 111, the estimation unit 112, the calculation unit 113, the classification unit 114, the conversion unit 115, the filter processing unit 116, and the detection unit 117 are included in an object detection device 50. Details of the data acquisition unit 111, the estimation unit 112, the calculation unit 113, the classification unit 114, the conversion unit 115, the filter processing unit 116, and the detection unit 117 included in the object detection device 50 will be described later.

[0018] In the autonomous driving mode, the driving control unit 118 generates a target trajectory based on the external conditions around the vehicle, including the size, position, and moving speed of an object detected by the object detection device 50. Specifically, the driving control unit 118 generates a target trajectory based on the size, position, and moving speed of the object detected by the object detection device 50, so as to avoid collision or contact with the object or to follow the object. The driving control unit 118 controls the actuators AC so that the host vehicle travels along the target trajectory. Specifically, the driving control unit 118 controls the actuators AC along the target trajectory to adjust the accelerator opening and drive the braking device and the steering device. In the manual driving mode, the driving control unit 118 controls the actuators AC in accordance with a driving command (such as a steering operation) from the driver acquired by the internal sensor group 3.

[0019] The object detection device 50 will now be described in detail. As described above, the object detection device 50 includes a data acquisition unit 111, an estimation unit 112, a calculation unit 113, a classification unit 114, a conversion unit 115, a filter processing unit 116, and a detection unit 117. The object detection device 50 further includes a LIDAR 5.

[0020] The data acquisition unit 111 acquires four-dimensional data (hereinafter referred to as point cloud data) including position information indicating three-dimensional position coordinates of measurement points on the surface of an object at which reflected waves from the LIDAR 5 are obtained, and velocity information indicating the relative movement velocity of the measurement points, as detection data of the LIDAR 5. The point cloud data is acquired by the LIDAR 5 on a frame-by-frame basis, more specifically, at predetermined time intervals (time intervals determined by the frame rate of the LIDAR 5).

[0021] The estimation unit 112 estimates the absolute moving speed of the host vehicle (a speed vector of X, Y, and Z coordinates) based on the point cloud data acquired by the data acquisition unit 111. Here, the estimation of the absolute moving speed of the host vehicle by the estimation unit 112 will be described.

[0022] First, the estimation unit 112 extracts point cloud data from the point cloud data acquired by the data acquisition unit 111, excluding information on measurement points corresponding to three-dimensional objects, that is, point cloud data corresponding to the road surface around the vehicle (hereinafter referred to as road surface point cloud data). The estimation unit 112 extracts the road surface point cloud data, that is, the measurement points P corresponding to the road surface, by the following equation (i): i (i=1,2,…,n) 4-dimensional data (x i ,y i ,z i ,v i ) contains the position coordinates (x i ,y i ,z i ) based on the relative movement speed v i The unit vector e indicates the direction of i Calculate.

[0023]

number

[0024] Next, the estimation unit 112 estimates the vehicle's travel speed (absolute travel speed) V self Specifically, the estimation unit 112 estimates the measurement point P i The relative movement speed v iThe transformation equation for converting the measured point P into the absolute moving speed is set as the objective function L, and the optimization problem is solved to optimize the objective function L so that it approaches zero. i are measurement points on the road surface, the absolute movement speed of each measurement point should be zero. Therefore, by optimizing the objective function L to approach zero, the correct V self can be estimated. V self is expressed as velocity components in the X, Y, and Z axes directions as shown in the following equation (ii). The objective function L is expressed by the following equation (iii). By solving the above optimization problem, V is obtained so that the right-hand side of equation (iii) becomes zero. self is searched. Note that V self The initial value of may be set to zero, or V estimated in the previous frame. self may be set.

[0025]

number

number

[0026] In equation (iii), A is the unit vector e of n measurement points corresponding to the road surface. i and is expressed by the formula (iv). In the formula (iii), V is the number of n measurement points P corresponding to the road surface. i is a 1×n matrix representing the velocity components (relative movement velocity) of V, which is expressed by the formula (v). The estimation unit 112 uses V obtained by solving the optimization problem. self is obtained as an estimate of the absolute moving speed of the host vehicle in the current frame.

[0027]

number

number

[0028] The calculation unit 113 calculates the absolute moving speed V of the vehicle estimated by the estimation unit 112. self Based on this, the absolute movement speed of all measurement points, more specifically, all measurement points including measurement points corresponding to three-dimensional objects, is calculated. Here, the calculated absolute movement speed has a negative value when approaching the host vehicle, and a positive value when moving away.

[0029] The classification unit 114 classifies the point cloud data acquired by the data acquisition unit 111 into moving point cloud data corresponding to measurement points where the absolute value of the absolute movement speed calculated by the calculation unit 113 is equal to or greater than a predetermined speed Th_V, and stationary point cloud data corresponding to measurement points where the absolute value is less than the predetermined speed Th_V.

[0030] The conversion unit 115 converts the moving point cloud data classified by the classification unit 114 into distance data and absolute velocity data (hereinafter, sometimes simply referred to as velocity data) expressed in a two-dimensional coordinate system of horizontal angle θ and vertical angle φ. Hereinafter, the measurement points corresponding to the moving point cloud data may be referred to as moving points.

[0031] Specifically, the conversion unit 115 converts the position coordinates (x i ,y i ,z i ) and generates distance data based on the distance data. The distance data is data that associates the position coordinates of each moving point expressed in the two-dimensional coordinate system with distance information that indicates the distance of each moving point from the vehicle (more specifically, the rider 5). Alternatively, the horizontal angle θ and vertical angle φ, which are set values ​​at the time of measurement, may be used to associate the distance data, which is the measured value at that position, in the two-dimensional coordinate system.

[0032] Furthermore, the conversion unit 115 converts the position coordinates (x i ,y i ,z i) and the absolute movement speed of each moving point calculated by the calculation unit 113, absolute speed data is generated. The absolute speed data is data that associates the position coordinates of each moving point expressed in the above-mentioned two-dimensional coordinate system with movement speed information that indicates the absolute movement speed of each moving point. Alternatively, the horizontal angle θ and vertical angle φ, which are set values ​​at the time of measurement, may be used to associate the data in the above-mentioned two-dimensional coordinate system using absolute speed data converted from the relative speed data, which is the measurement value at that position, using the above-mentioned method.

[0033] 2A to 2D are diagrams for explaining data conversion from moving point cloud data to distance data and absolute velocity data by conversion unit 115. Here, data conversion by conversion unit 115 will be explained using as an example data detected by a lidar installed in a concourse AS in a commercial facility as shown in Fig. 2A, rather than data detected by a lidar mounted on a vehicle. Note that in Figs. 2A and 2B, the depth direction (extension direction of the concourse AS) corresponds to the X-axis direction, the left-right direction corresponds to the Y-axis direction, and the vertical direction corresponds to the Z-axis direction.

[0034] FIG. 2A shows a concourse AS in a commercial facility as seen from the rider's viewpoint. FIG. 2A is a diagram showing a plurality of pedestrians HM1 to HM8 passing through the concourse AS. FIG. 2A shows pedestrians HM2 and HM3 moving (walking) in the same direction (X-axis direction), and pedestrian HM1 moving (walking) in the opposite direction. The absolute values ​​of the absolute movement speeds of pedestrians HM1 to HM8 are equal to or greater than a predetermined speed Th_V.

[0035] Figure 2B shows an example of lidar detection data (point cloud data) for the three-dimensional space of Figure 2A. The lightly shaded area in Figure 2B represents a measurement point cloud corresponding to a stationary object (the wall WL of the concourse AS in Figure 2A), i.e., stationary point cloud data. The darkly shaded area represents a measurement point cloud corresponding to a moving object (a pedestrian moving through the concourse AS in Figure 2A), i.e., moving point cloud data. Note that measurement point clouds PC1 to PC8 correspond to pedestrians HM1 to HM8 in Figure 2A, respectively.

[0036] 2C and 2D show examples of distance data and absolute velocity data obtained by converting the moving point cloud data of FIG. 2B. The measurement point clouds PC11 to PC18 in FIG. 2C and the measurement point clouds PC21 to PC28 in FIG. 2D are respectively representations of the measurement point clouds PC1 to PC8 in FIG. 2B in a two-dimensional coordinate system of vertical angle φ and horizontal angle θ. In FIGS. 2C and 2D, the vertical axis represents the vertical angle φ, and the horizontal axis represents the horizontal angle θ. Furthermore, in the distance data of FIG. 2C, moving points closer to the lidar are shown in a darker color. In the absolute velocity data of FIG. 2D, moving points with higher absolute moving velocities are shown in a darker color.

[0037] 2C and 2D are generated by the conversion unit 115, the filter processing unit 116 applies filter processing to each of the distance data and absolute velocity data. More specifically, the filter processing unit 116 performs a kernel operation on the distance data, with each of the moving points as its center position, to calculate the difference in distance data between each of the moving points. The filter processing unit 116 also performs a kernel operation on the absolute velocity data, with each of the moving points as its center position, to calculate the difference in absolute velocity data between each of the moving points.

[0038] Here, kernel calculations will be described. FIGS. 3A to 3C are diagrams for explaining the kernel applied to distance data. FIG. 3A shows a schematic diagram of distance data. Each grid in the distance data D in FIG. 3A corresponds to a position (illumination point) where the light emitted by the lidar 5 is irradiated, and the vertical and horizontal spacing of each grid corresponds to the angular resolution of the lidar in the vertical and horizontal directions. Alternatively, the horizontal angle θ and vertical angle φ may be calculated from the position coordinates xi, yi, and zi of each point according to the range of the grid set with a predetermined angular resolution. The shaded grids represent illumination points (measurement points) where reflected light from the surface of a moving object is obtained, i.e., moving points.

[0039] The filter processing unit 116 applies a kernel with each moving point of the distance data D as its center position to each moving point, and repeatedly performs difference calculation processing. The difference calculation processing will be described later with reference to FIGS. 4A to 4C. The filter processing unit 116 performs such kernel calculations in the order shown by the arrows in FIG. 3A, from the upper left moving point MP1 to the lower right moving point MP2. Therefore, in the first kernel calculation performed on the distance data D, the kernel is applied so that the center position is aligned with the upper left moving point MP1, as shown in FIG. 3B. In the last calculation process performed on the distance data D, the kernel is applied so that the center position is aligned with the lower right moving point MP2, as shown in FIG. 3C. The area KN in FIGS. 3B and 3C schematically represents the kernel, and the thick-line frame within the area KN represents the center position of the kernel.

[0040] 3B and 3C show the kernel KN having a size of 5 × 5, the kernel size may be other than 5 × 5. The kernel size may also be changed according to the distance indicated by the distance information of the moving point to which the kernel is applied (the moving point of the center position of the kernel).

[0041] For example, the filter processing unit 116 may determine the kernel size for each moving point based on the minimum angular resolution of the LIDAR 5, the minimum size (minimum size in the horizontal and vertical directions) of a moving object previously designated as a detection target, or any other arbitrarily designated size, and the distance indicated by the distance information. In this case, the filter processing unit 116 first calculates the grid width of the kernel KN to be applied to the moving point to be processed based on the minimum angular resolution of the LIDAR 5 and the distance indicated by the distance information. For example, when the minimum angular resolution is A [rad] and the distance of the moving point to be processed from the host vehicle (LIDAR 5) is B [m], the grid width is calculated as A × B. Next, based on the calculated grid width and the minimum size, the filter processing unit 116 calculates the number of grids (hereinafter referred to as the required kernel width) such that the distance between both ends of the kernel KN (the distance between both ends in the horizontal and vertical directions) is the minimum size. Finally, the filter processing unit 116 determines the kernel size based on the calculated required kernel width. For example, when the required kernel width is 9, the kernel size is determined to be 9 × 9.

[0042] Similarly, the filter processing unit 116 applies a kernel with each moving point as its center position to each moving point of the absolute velocity data, and repeatedly executes the difference calculation process.

[0043] 4A and 4B are diagrams for explaining the difference calculation process in kernel calculation. In the difference calculation process for distance data, the distance information of each moving point in the kernel KN is compared with the distance information of the moving point at the center position of the kernel KN, and the absolute value of the difference between the value and the moving point at the center position is calculated. Then, the calculation result for a moving point whose absolute value is equal to or greater than a predetermined threshold Th1 is output as "0," and the calculation result for a moving point whose absolute value is less than the predetermined threshold Th1 is output as "1." FIG. 4A shows an example of the calculation result of the difference calculation process executed for the kernel KN whose center position is the moving point MP1 at the top left of the distance data.

[0044] In the difference calculation process for absolute velocity data, the movement velocity information of each movement point within the kernel KN is compared with the movement velocity information of the movement point at the center position of the kernel KN, and the absolute value of the difference from the value of the movement point at the center position is calculated.Then, the calculation result for a movement point whose absolute value is equal to or greater than a predetermined threshold Th2 is output as "0," and the calculation result for a movement point whose absolute value is less than the predetermined threshold Th2 is output as "1." Figure 4B shows an example of the calculation result of the difference calculation process executed for the kernel KN whose center position is the movement point MP1 in the upper left of the absolute velocity data.

[0045] The detection unit 117 detects the positions and sizes of moving objects around the vehicle based on the results of the kernel calculation by the filter processing unit 116, that is, the calculation results of the difference calculation process executed on the distance data and absolute velocity data.

[0046] 5A and 5B are diagrams illustrating the detection process of a moving object by the detection unit 117. The detection unit 117 calculates, for each moving point, the Hadamard product of the calculation result of the difference calculation process obtained from the distance data and the calculation result of the difference calculation process executed on the absolute velocity data. FIG. 5A shows the calculation result of the Hadamard product corresponding to the upper left moving point MP1, i.e., the calculation result of the Hadamard product of the calculation result of FIG. 4A and the calculation result of FIG. 4B. As shown in FIG. 5A, the detection unit 117 calculates, for each moving point within the kernel KN, the logical product (AND operation) of the calculation result of the difference calculation process obtained from the distance data and the calculation result of the difference calculation process obtained from the absolute velocity data.

[0047] The detection unit 117 counts the number of movement points within the kernel for which the calculation result of the Hadamard product is "1". ExaminationWhen the counted number is equal to or greater than a predetermined number Th_N, the output unit 117 determines that the moving point clouds for which the Hadamard product calculation result is "1" are part of the measurement point clouds obtained from the surface of the same moving object, and assigns the same identifier (hereinafter referred to as an identification ID) to the moving point clouds (hereinafter referred to as the moving point clouds to be assigned). FIG. 5B shows an example of an identifier assigned based on the calculation result of the Hadamard product in FIG. 5A. In the example shown in FIG. 5B, "n (integer)" is assigned as the identification ID. Note that the predetermined number Th_N may be changed depending on the kernel size. For example, the larger the required kernel width, the larger the value set for the predetermined number Th_N.

[0048] The detection unit 117 performs such an identifier assignment process based on the result of the kernel operation (more specifically, the result of the Hadamard product operation) each time a kernel operation is performed with each moving point as the center position. At this time, the detection unit 117 increments the value of the identification ID by one each time the identifier assignment process is performed. Furthermore, the detection unit 117 stores the result of the identifier assignment process in the storage unit 12 each time the identifier assignment process is performed. More specifically, the detection unit 117 stores information (hereinafter referred to as identifier data) in which position information (position coordinates expressed in a two-dimensional coordinate system of a horizontal angle θ and a vertical angle φ) of the moving point to be assigned and the identification ID are associated with each other in the storage unit 12. Note that the position information of the moving point in the identifier data may be position coordinates expressed in a three-dimensional coordinate system (XYZ coordinate system).

[0049] When the identifier assignment process is repeatedly executed as described above, the group of move points to be assigned selected from within the kernel may include move points to which an ID has already been assigned by a previously executed identifier assignment process (hereinafter referred to as assigned move points). In such cases, the detection unit 117 assigns an ID with the same value as the ID of the assigned move points to move points to which no ID has been assigned (hereinafter referred to as unassigned move points) among the group of move points to be assigned. For example, when an assigned move point to which an ID "4" has been assigned exists among the group of move points to be assigned, the detection unit 117 assigns the ID "4" to the unassigned move points.

[0050] Furthermore, if there are multiple assigned moved points with different identification IDs in the target moved point cloud, the detection unit 117 assigns the smallest identification ID (hereinafter referred to as "Min") among the assigned moved points to the unassigned moved point. Furthermore, the detection unit 117 reassigns the identification ID of assigned moved points assigned with an identification ID other than "Min" to "Min." For example, if there are assigned moved points assigned with the identification ID "3" and assigned moved points assigned with the identification ID "4" in the target moved point cloud selected from within the kernel, the detection unit 117 assigns the identification ID "3" to the unassigned moved point. Furthermore, the detection unit 117 reassigns the identification ID of assigned moved points assigned with the identification ID "4" to "3." Note that if there are one or more assigned moved points assigned with the identification ID "4" outside the kernel to be processed, these assigned moved points are also considered to be part of the measurement point cloud obtained from the surface of the same moving object. Therefore, the detection unit 117 may also reassign the identification IDs of these assigned movement points to "3".

[0051] Based on the identifier data, the detection unit 117 classifies the moving point cloud data of Fig. 2B into moving point clouds (clusters) assigned with the same identification ID. Furthermore, the detection unit 117 detects a circumscribing area (bounding box) of each cluster on the moving point cloud data, and detects the position and size of each moving object around the vehicle in a three-dimensional space (XYZ space) based on the position and size of each detected circumscribing area on the moving point cloud data. The detection unit 117 outputs information indicating the detection results of the moving objects to the storage unit 12, a display device (not shown), or the like.

[0052] Depending on the type of moving object, the absolute moving speed of the main body of the moving object may differ from the absolute moving speed of parts attached to the main body. For example, since a pedestrian moves while moving their limbs, the absolute moving speeds calculated by the calculation unit 113 for the torso and limbs of the moving pedestrian may differ. In such a case, if object detection is performed taking into account the absolute moving speed of each moving point as described above, the torso and limbs of the pedestrian may be detected as separate moving objects. Therefore, to address this problem, the detection unit 117 performs a moving object combining process as described below.

[0053] 6A and 6B are diagrams for explaining the moving object combining process. FIG. 6A shows a schematic diagram of a pedestrian HM61 moving in the X direction and a pedestrian HM62 moving opposite the pedestrian HM61, as viewed from the Y direction. In FIG. 6A, bounding boxes detected by the detection unit 117 are schematically shown as rectangular frames BD1, BD2, PT11 to PT14, and PT21 to PT24. Bounding boxes BD1 and BD2 correspond to the torsos of pedestrians HM61 and HM62. Bounding boxes PT11, PT12, PT21, and PT22 correspond to the arms of pedestrians HM61 and HM62. Bounding boxes PT13, PT14, PT23, and PT24 correspond to the legs of pedestrians HM61 and HM62.

[0054] The detection unit 117 determines whether or not a bounding box (hereinafter referred to as a bounding box to be combined, or simply a bounding box to be combined) exists among the bounding boxes detected from the moving point cloud data, the bounding box being less than a predetermined distance from other bounding boxes whose sizes are equal to or greater than a predetermined size and whose own size is less than the predetermined size. If it is determined that a bounding box to be combined exists, the detection unit 117 combines the bounding box to the other bounding box (hereinafter referred to as a bounding box to be combined, or simply a bounding box to be combined). In the example of FIG. 6A, bounding boxes PT11 to PT14 are selected as the bounding boxes, and bounding box BD1 is selected as the bounding box to be combined corresponding to the bounding boxes PT11 to PT14. As a result, bounding boxes PT11 to PT14 are combined with bounding box BD1. Similarly, bounding boxes PT21 to PT24 are combined with bounding box BD2. This prevents the torsos and limbs of pedestrians HM61 and HM62 from being detected as separate moving objects. Note that bounding boxes BD1 and BD2, which are equal to or larger than a predetermined size, are boxes to be joined, and therefore will not be joined even if the distance between them is less than a predetermined length. Also, bounding boxes PT11 to PT14 and PT21 to PT24, which are smaller than a predetermined size, are boxes to be joined, and therefore will not be joined even if the distance between them is less than a predetermined length.

[0055] Note that while FIG. 6A illustrates an example of a pedestrian's torso and limbs, the absolute movement speeds calculated by the calculation unit 113 for the vehicle body and its associated parts, such as wheels, may differ even for vehicles traveling around the host vehicle. FIG. 6B illustrates a schematic diagram of a vehicle CA moving in the X direction as viewed from the Y direction. In FIG. 6B, the rectangular frames BD3, PT31, and PT32 in FIG. 6B schematically illustrate bounding boxes detected from the movement point cloud data. The bounding box BD3 corresponds to the vehicle body. The bounding boxes PT31 and PT32 correspond to the front and rear wheels of the vehicle CA. The vehicle body and its associated parts, such as wheels, have the same movement speed in the vehicle's traveling direction, but their relative positions with respect to the rider 5 are different, and therefore, different relative movement speeds may be calculated by the calculation unit 113. Even in such cases, the bounding boxes PT31 and PT32 can be merged with the bounding box BD3 by performing the merging process described above. As a result, it is possible to prevent the main body and parts of the vehicle CA from being detected as separate moving objects.

[0056] 7 is a flowchart showing an example of processing executed by the calculation unit 11 of the controller 10 in FIG. 1 in accordance with a predetermined program. The processing shown in this flowchart is repeated at a predetermined cycle while the object detection device 50 is running. More specifically, the processing is repeated at a cycle according to the frame rate of the LIDAR 5.

[0057] First, in step S1, the external environment surrounding the vehicle is detected. Specifically, an irradiation command is sent to the LIDAR 5, and point cloud data (detection data) including position information and speed information of measurement points at which reflected waves of electromagnetic waves irradiated from the LIDAR 5 in response to the irradiation command are obtained is acquired. In step S2, the point cloud data acquired in step S1 is classified into moving point cloud data and stationary point cloud data. More specifically, the point cloud data is classified into moving point cloud data corresponding to measurement points whose absolute value of the absolute movement speed is equal to or greater than a predetermined speed Th_V, and stationary point cloud data corresponding to other measurement points.

[0058] Next, the processing of steps S3 to S9 is executed for the moving point cloud data classified from the point cloud data. Note that the controller 10 also executes predetermined processing for the stationary point cloud data, but the explanation thereof will be omitted.

[0059] In step S3, the moving point cloud data is converted into distance data and absolute velocity data expressed in a two-dimensional coordinate system of horizontal angle θ and vertical angle φ. Next, steps S41 to S42 are executed on the distance data, and in parallel, steps S51 to S52 are executed on the absolute velocity data. Note that steps S41 to S42 and steps S51 to S52 may be executed in this order.

[0060] In step S41, a kernel with the moving point at its center position is applied to a single moving point included in the distance data. In step S42, a difference calculation process is performed on the kernel applied in step S41. Specifically, the distance information of each moving point in the kernel is compared with the distance information of the moving point at the center position of the kernel, and the absolute value of the difference between the value at the center position and the moving point is calculated. Then, the calculation result for moving points whose absolute value is equal to or greater than a predetermined threshold Th1 is output as "0", and the calculation result for moving points whose absolute value is less than the predetermined threshold Th1 is output as "1".

[0061] In step S51, a kernel with the moving point at its center position is applied to a single moving point included in the absolute velocity data. In step S52, a difference calculation process is performed on the kernel applied in step S51. Specifically, the moving speed information of each moving point within the kernel is compared with the moving speed information of the moving point at the center position of the kernel, and the absolute value of the difference between the values ​​of the moving point at the center position is calculated. Then, the calculation result for moving points whose absolute value is equal to or greater than a predetermined threshold Th2 is output as "0", and the calculation result for moving points whose absolute value is less than the predetermined threshold Th2 is output as "1".

[0062] In step S6, the Hadamard product of the calculation result of the difference calculation process in step S42 (FIG. 4A) and the calculation result of the difference calculation process in step S52 (FIG. 4B) is calculated. In step S7, the calculation result of the Hadamard product in step S6 (FIG. 5A) is referenced to count the number of movement points for which the calculation result is "1", and it is determined whether this number is equal to or greater than a predetermined number Th_N. If the result in step S7 is negative, the process proceeds to step S9. If the result in step S7 is positive, in step S8, an identification ID is assigned to the movement points for which the calculation result of the Hadamard product is "1". Specifically, information associating the position information of the movement points for which the calculation result of the Hadamard product is "1" with the identification ID, i.e., identifier data, is stored in storage unit 12.

[0063] In step S9, it is determined whether or not the distance data and absolute velocity data include any unprocessed moving points for which the processing in steps S41 to S42 and S51 to S52 has not been performed. If the result in step S9 is affirmative, the next moving point to be processed is selected in the order indicated by the arrows in Fig. 3A, and the kernel is applied to the selected moving point to perform the processing in steps S41 to S42, S51 to S52, and S6 to S8.

[0064] If the result in step S9 is negative, moving objects are detected based on the identifier data stored in the storage unit 12. Specifically, the moving point cloud data is classified into moving point clouds (clusters) assigned the same identification ID based on the identifier data. Then, a bounding box of each cluster is detected in the moving point cloud data, and the position and size of each moving object around the vehicle in three-dimensional space are detected based on the position and size of each bounding box in the moving point cloud data. Note that when the bounding boxes are combined by the above-mentioned combining process, the position and size of the moving objects around the vehicle in three-dimensional space are detected based on the position and size of the combined bounding box in the moving point cloud data.

[0065] According to the embodiment described above, the following advantageous effects are achieved. (1) The object detection device 50 is mounted on a vehicle and includes a lidar 5 that irradiates electromagnetic waves into a three-dimensional space around the vehicle and detects the external environment around the vehicle based on the reflected waves; a data acquisition unit 111 that acquires point cloud data from the detector, the point cloud data including three-dimensional position information of a plurality of measurement points on the surface of the object from which the reflected waves are obtained and first speed information indicating the relative movement speed of the plurality of measurement points; an estimation unit 112 as a speed acquisition unit that acquires second speed information indicating the absolute movement speed of the vehicle; a calculation unit 113 that calculates the absolute movement speed of each of the plurality of measurement points based on the first speed information and the second speed information; and a calculation unit 113 that calculates the absolute movement speed of each of the plurality of measurement points based on the first speed information and the second speed information. and a plurality of stationary points that are moving at a speed less than a predetermined speed; a conversion unit 115 as a generation unit that generates distance data (FIG. 2C) indicating the distances of the plurality of moving points from the vehicle itself and speed data (FIG. 2D) that indicates the absolute moving speeds of the plurality of moving points calculated by the calculation unit 113; a filter processing unit 116 that performs a kernel operation on each of the distance data and speed data generated by the conversion unit 115, with each of the plurality of moving points as its center position, to calculate the difference in distance data and speed data between the plurality of moving points; and a detection unit 117 that detects the position and size of each of the plurality of moving objects around the vehicle itself based on the result of the kernel operation. Detection unit 117 is a kernel operation Based on the results of The detection unit 117 calculates the number of second moving points in the kernel whose difference in distance data from the first moving point set as the center position is less than a predetermined threshold Th1 and whose difference in speed data from the first moving point is less than a predetermined threshold Th2. Th_N ), the first moving point and the second moving point in the kernel are assigned an ID with a different value for each kernel operation. The detection unit 117 further detects the position and size of a single moving object based on the positions and sizes of moving points that are assigned the same ID value among the multiple moving points.

[0066] In this way, by detecting moving objects using a kernel parallel calculation method used in image processing, etc., moving objects can be detected without using clustering methods with high computational loads such as DBSCAN (Density-based spatial clustering of applications with noise). As a result, the processing load on the device can be reduced. In addition, by detecting moving objects using the results of kernel calculations performed on both distance data and absolute velocity data, moving objects that are close to each other, such as pedestrians passing each other, can be detected as separate moving objects. As a result, the accuracy of detecting moving objects can be improved.

[0067] (2) When assigning identification IDs to the first moving points and the second moving points within the kernel, if the first moving points and the second moving points include assigned moving points to which identification IDs have already been assigned, the detection unit 117 assigns an identification ID with the same value as the identification ID assigned to the assigned moving points. Furthermore, when assigning identification IDs to the first moving points and the second moving points within the kernel, if the first moving points and the second moving points include multiple assigned moving points to which different identification IDs have been assigned, the detection unit 117 assigns the smallest identification ID among the identification IDs assigned to the multiple assigned moving points. This further improves the accuracy of detecting moving objects using the kernel parallel computing method.

[0068] (3) For each kernel calculation, the filter processing unit 116 determines the size of the kernel based on the distance from the vehicle (lidar 5) to the moving point set as the center position, which is indicated by the distance data. This allows for accurate detection of moving objects regardless of the distance from the vehicle to the moving object.

[0069] (4) The detection unit 117 classifies multiple moving points into moving point clouds assigned the same identification ID, detects the circumscribing area of ​​each moving point cloud, and detects the position and size of a moving object corresponding to each moving point cloud based on the position and size of the circumscribing area of ​​each moving point cloud. When the circumscribing area of ​​each moving point cloud includes a first circumscribing area whose size is less than a predetermined size and a second circumscribing area whose size is equal to or greater than the predetermined size and whose distance from the first circumscribing area is less than a predetermined distance, the detection unit 117 combines the first circumscribing area with the second circumscribing area. This allows a moving object consisting of a body and parts, such as a person's torso and limbs or a vehicle's body and wheels, to be properly detected as a single moving object.

[0070] The above embodiment can be modified in various ways. Modifications will be described below. In the above embodiment, a lidar 5 as a detector is mounted on a vehicle, irradiates electromagnetic waves into a three-dimensional space around the vehicle, and detects the external environment around the vehicle based on the reflected waves. However, the detector may be anything that can obtain distance and speed. For example, a detector other than a lidar, such as a radar, may be used. Furthermore, the moving body on which the detector is mounted may be a self-propelled robot or the like, other than a vehicle.

[0071] In the above embodiment, the estimation unit 112 as a velocity acquisition unit selects a measurement point P as a representative measurement point from the remaining measurement points after excluding the measurement points corresponding to the three-dimensional object from the plurality of measurement points. i and estimates the absolute moving speed of the host vehicle based on the position information and speed information of the representative measurement points extracted from the point cloud data acquired by the data acquisition unit 111, and acquires the estimated result as the second speed information. However, the speed acquisition unit may acquire, as the second speed information, a measurement result of the absolute moving speed of the host vehicle acquired by a measuring instrument included in the internal sensor group 3. In this case, the object detection device 50 includes at least a vehicle speed sensor from the internal sensor group 3 as a measuring instrument. Furthermore, the speed acquisition unit may calculate and acquire the absolute moving speed of the host vehicle based on the current position of the host vehicle measured by the positioning unit 2. In this case, the object detection device 50 includes the positioning unit 2.

[0072] In the above embodiment, the filter processing unit 116 performs a kernel operation on distance data expressed in a two-dimensional coordinate system of a horizontal angle θ and a vertical angle φ to calculate the difference in distance data between each moving point. However, the filter processing unit may perform a kernel operation on the position coordinates of each moving point of the moving point cloud data (position coordinates expressed in a three-dimensional coordinate system (XYZ coordinate system)). The filter processing unit may then calculate the differences in the X coordinate, Y coordinate, and Z coordinate between each moving point. In this case, in the difference calculation process for the X coordinate, the filter processing unit 116 compares the X coordinate of each moving point within the kernel with the X coordinate of the moving point at the center position of the kernel and calculates the absolute value of the difference from the value of the moving point at the center position. The filter processing unit 116 also performs a similar difference calculation process for the Y coordinate and the Z coordinate. The detection unit 117 calculates the Hadamard product of the calculation results of the difference calculation process for each of the X coordinate, Y coordinate, Z coordinate, and absolute velocity data. Note that instead of executing the difference calculation process for all of the X coordinate, Y coordinate, and Z coordinate, the difference calculation process may be executed for one of the X coordinate, Y coordinate, and Z coordinate, and the Hadamard product of the calculation result and the calculation result of the difference calculation process for the absolute velocity data may be calculated for each moving point.

[0073] Furthermore, in the above embodiment, the driving control unit 118 controls the driving of the vehicle so as to avoid a collision or contact with an object detected by the detection unit 117. However, the driving control unit 118 may function as a notification unit and predict the possibility of a collision or contact with a moving object based on the size, position, and moving speed of the moving object detected by the detection unit 117. When the possibility of a collision or contact with a moving object is equal to or greater than a predetermined level, the driving control unit 118 may notify the occupants of the vehicle of warning information (video information or audio information) regarding the collision or contact with the moving object detected by the detection unit 117 via a display or speaker (not shown) provided in the vehicle control device 100.

[0074] Furthermore, in the above embodiment, the object detection device 50 is applied to an autonomous vehicle, but the object detection device 50 can also be applied to vehicles other than autonomous vehicles. For example, the object detection device 50 can also be applied to a manually driven vehicle equipped with an ADAS (Advanced Driver-Assistance Systems).

[0075] The above description is merely an example, and the present invention is not limited to the above-described embodiment and modifications, as long as the features of the present invention are not impaired. One or more of the above-described embodiment and modifications can be arbitrarily combined, and modifications can also be combined with each other. [Explanation of symbols]

[0076] 5 Lidar, 10 Controller, 11 Calculation unit, 12 Memory unit, 111 Data acquisition unit, 112 Estimation unit, 113 Calculation unit, 114 Classification unit, 115 Conversion unit, 116 Filter processing unit, 117 Detection unit, 118 Travel control unit, 50 Object detection device, 100 Vehicle control device, AC actuator

Claims

1. a detector mounted on the moving body, which irradiates electromagnetic waves into a three-dimensional space around the moving body and detects an external environment around the moving body based on reflected waves; a data acquisition unit that acquires, from the detector, point cloud data including three-dimensional position information of a plurality of measurement points on a surface of an object from which the reflected waves are obtained and first velocity information that indicates relative movement velocities of the plurality of measurement points; a speed acquisition unit that acquires second speed information indicating an absolute moving speed of the moving object; a calculation unit that calculates an absolute moving speed of each of the plurality of measurement points based on the first speed information and the second speed information; a classification unit that classifies the plurality of measurement points into a plurality of moving points where the absolute value of the absolute movement speed calculated by the calculation unit is equal to or greater than a predetermined speed, and a plurality of stationary points where the absolute value is less than the predetermined speed; a generating unit that generates distance data indicating distances of the plurality of moving points from the moving body and speed data that indicates the absolute moving speeds of the plurality of moving points calculated by the calculating unit; a filter processing unit that performs a kernel operation on each of the distance data and the speed data generated by the generating unit, with each of the plurality of moving points as a center position, to calculate differences in the distance data and the speed data between the plurality of moving points; and an object detection unit that detects the position and size of each of a plurality of moving objects around the moving body based on the result of the kernel operation.

2. 2. The object detection device according to claim 1, the object detection unit calculates, based on a result of the kernel calculation, the number of second moving points in the kernel, each of which has a difference in distance data from the first moving point set as the center position that is less than a first threshold value and a difference in speed data from the first moving point that is less than a second threshold value; when the number of the second moving points in the kernel is equal to or greater than a predetermined number, assigning identifiers, the identifiers having different values ​​for each kernel operation, to the first moving points and the second moving points in the kernel; Furthermore, the object detection device is characterized in that it detects the position and size of a single moving object based on the positions and sizes of a group of moving points among the plurality of moving points that have been assigned the identifier of the same value.

3. 3. The object detection device according to claim 2, The object detection device is characterized in that, when assigning the identifier to the first moving point and the second moving point within the kernel, if the first moving point and the second moving point include an assigned moving point to which the identifier has already been assigned, the object detection unit assigns the identifier with the same value as the identifier assigned to the assigned moving point.

4. 4. The object detection device according to claim 3, The object detection device is characterized in that, when assigning the identifier to the first moving point and the second moving point within the kernel, if the first moving point and the second moving point include multiple assigned moving points to which different identifiers have been assigned, the object detection unit assigns the identifier with the smallest value among the identifiers assigned to the multiple assigned moving points.

5. 2. The object detection device according to claim 1, The object detection device is characterized in that the filter processing unit determines the size of the kernel for each kernel calculation based on the distance from the moving body to the moving point set as the center position, as indicated by the distance data.

6. 3. The object detection device according to claim 2, The object detection unit classifies the multiple moving points into groups of moving points that have the same identifier, detects the circumscribing area of ​​each group of moving points, and detects the position and size of the moving object corresponding to each group of moving points based on the position and size of the circumscribing area of ​​each group of moving points.

7. 7. The object detection device according to claim 6, The object detection device is characterized in that, when the circumscribing area of ​​each moving point group includes a first circumscribing area whose size is less than a predetermined size and a second circumscribing area whose size is equal to or greater than the predetermined size and whose distance from the first circumscribing area is less than a predetermined length, the object detection unit combines the first circumscribing area with the second circumscribing area.

8. The object detection device according to any one of claims 1 to 7, An object detection device, characterized in that the detector is a lidar.

Images