Object detection device

Abstract

An object detection device (50) is provided with: a lidar (5); a data acquisition unit (111) that acquires point cloud data for each predetermined time indicating the detection result of the lidar (5), the point cloud data including position information of a measurement point on the surface of the object and first speed information indicating the relative movement speed of the measurement point; an estimation unit (112) that acquires second speed information indicating the absolute movement speed of the moving body; a speed calculation unit (113) that calculates the absolute movement speed of each of the plurality of measurement points on the basis of the first speed information and the second speed information; a classification unit (114) that classifies the point cloud data into moving point cloud data corresponding to the measurement points at which the absolute value of the absolute movement speed is equal to or greater than a prescribed speed, and stationary point cloud data other than the moving point cloud data; an extraction unit (115) that extracts, from the stationary point cloud data, changed point cloud data corresponding to a measurement point for which the amount of movement during a predetermined time is equal to or greater than a predetermined threshold value; and a detection unit (116) that detects objects around the moving body on the basis of the moving point cloud data and the changing point cloud data.

Description

Technical Field

[0001] The present invention relates to an object detection device for detecting objects around a vehicle. Background Art

[0002] As such a device, there is known a device that uses machine learning to detect moving objects from three-dimensional point cloud data obtained by a laser radar (for example, see Patent Document 1).

[0003] However, the method using machine learning, as in the device described in Patent Document 1, may not be able to sufficiently ensure the detection accuracy of a moving object because the reliability of the learning model affects the detection accuracy.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent document 1: Japanese Patent Application Publication No. 2023-027736 (JP 2023-027736A). Summary of the invention

[0007] An object detection device according to one embodiment of the present invention comprises: a detector mounted on a moving body, irradiating electromagnetic waves around the moving body, and detecting an external condition around the moving body based on reflected waves; a data acquisition unit, which acquires point cloud data at each predetermined time indicating a detection result of the detector, the point cloud data including position information of a measurement point on the surface of the object where the reflected waves are obtained and first speed information indicating a relative moving speed of the measurement point; a speed acquisition unit, which acquires second speed information indicating an absolute moving speed of the moving body; a calculation unit, which calculates the absolute moving speed of each of a plurality of measurement points corresponding to the point cloud data based on the first speed information and the second speed information; a classification unit, which classifies the point cloud data into moving point cloud data corresponding to a measurement point whose absolute value of the absolute moving speed calculated by the calculation unit is greater than a predetermined speed and stationary point cloud data other than the moving point cloud data; an extraction unit, which calculates a movement amount of the stationary point cloud data at a predetermined time, and extracts change point cloud data corresponding to a measurement point whose movement amount is greater than a predetermined threshold value from the stationary point cloud data; and a detection unit, which detects an object around the moving body based on the moving point cloud data and the change point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The objects, features and advantages of the present invention will be further clarified through the following description of the embodiments in conjunction with the accompanying drawings.

[0009] Figure 1 1 is a block diagram showing a main part configuration of a vehicle control device including the object detection device according to the embodiment of the present invention.

[0010] Figure 2This is a diagram used to illustrate the relationship between the relative moving speed of a moving object measured by a laser radar and the moving direction of the moving object.

[0011] Figure 3 It is shown by Figure 1 A flowchart of an example of a process executed by the CPU of the controller.

[0012] Figure 4 This is a diagram schematically showing an example of point cloud data obtained by a laser radar.

[0013] Figure 5A This is a diagram schematically showing an example of stationary point cloud data of the previous frame.

[0014] Figure 5B This is a diagram schematically showing an example of stationary point cloud data of the current frame.

[0015] Fig. 6A Schematically shows a state in which the static point cloud data of the previous frame and the static point cloud data of the current frame are aligned.

[0016] Figure 6B 3 is a diagram showing the static point cloud data of the previous frame and the static point cloud data of the current frame after alignment.

[0017] Figure 7 This is a diagram schematically showing another example of point cloud data obtained by a laser radar.

[0018] Figure 8 3 is a diagram showing a state where change points are extracted from stationary point cloud data.

[0019] Fig. 9 A diagram for explaining the motion vector of a moving object.

[0020] Fig.10 This is a diagram showing an example of information indicating a detection result of a moving object. DETAILED DESCRIPTION

[0021] Hereinafter, the embodiments of the present invention will be described with reference to the accompanying drawings. The object detection device of the embodiments of the present invention can be applied to vehicles with an automatic driving function, i.e., automatic driving vehicles. It should be noted that sometimes, the vehicle to which the object detection device of the present embodiment is applied is referred to as the present vehicle, as distinguished from other vehicles. The present vehicle may be any one of an engine vehicle having an internal combustion engine (engine) as a driving source, an electric vehicle having a driving motor as a driving source, and a hybrid vehicle having an engine and a driving motor as driving sources. The present vehicle can not only travel in an automatic driving mode that does not require the driver's driving operation, but can also travel in a manual driving mode in which the driver performs driving operations.

[0022] When an autonomous vehicle is driving in autonomous driving mode (hereinafter referred to as automatic driving or autonomous driving), it recognizes the external conditions around the vehicle based on the detection data of on-board detectors such as cameras and LiDAR (Light Detection and Ranging). Based on the recognition results, the autonomous vehicle generates a driving trajectory (target trajectory) from the current time point to a specified time, and controls the driving actuators so that the vehicle drives along the target trajectory.

[0023] Figure 1 1 is a block diagram showing the main structure of a vehicle control device 100 including an object detection device. The vehicle control device 100 includes a controller 10, a communication unit 1, a positioning unit 2, an internal sensor group 3, a camera 4, a laser radar 5, and an actuator AC for driving. In addition, the vehicle control device 100 includes an object detection device 50 constituting a part of the vehicle control device 100. The object detection device 50 detects objects around the vehicle based on detection data of the laser radar 5.

[0024] The communication unit 1 communicates with various servers not shown in the figure through a network including a wireless communication network represented by the Internet, a mobile phone network, etc., and obtains map information, driving history information, traffic information, etc. from the server regularly or at any time. The network includes not only a public wireless communication network, but also a closed communication network set up in each specified management area, such as a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc. The acquired map information is output to the storage unit 12, and the map information is updated. The positioning unit (GNSS unit) 2 has a positioning sensor that receives a positioning signal sent from a positioning satellite. The positioning satellite is an artificial satellite such as a GPS satellite and a quasi-zenith satellite. The positioning unit 2 uses the positioning information received by the positioning sensor to determine the current position (latitude, longitude, altitude) of the vehicle.

[0025] The internal sensor group 3 is a general term for a plurality of sensors (internal sensors) for detecting the driving state of the vehicle. For example, the internal sensor group 3 includes a vehicle speed sensor for detecting the vehicle speed, an acceleration sensor for detecting the acceleration in the front-rear direction and the left-right direction (lateral acceleration) of the vehicle, a rotation speed sensor for detecting the rotation speed of the driving source, and a yaw rate sensor for detecting the rotational angular velocity of the center of gravity of the vehicle rotating around the vertical axis. Sensors for detecting the driving operation of the driver in the manual driving mode, such as the operation of the accelerator pedal, the operation of the brake pedal, and the operation of the steering wheel, are also included in the internal sensor group 3.

[0026] The camera 4 has a CCD (charge coupled device), a CMOS (complementary metal oxide semiconductor) and other imaging elements, and captures the surroundings (front, rear and side) of the vehicle. The laser radar 5 irradiates electromagnetic waves (reflected waves) to the three-dimensional space around the vehicle, and detects the external conditions around the vehicle based on the reflected waves. In more detail, the electromagnetic waves (laser, etc.) irradiated by the laser radar 5 are reflected and returned at a certain point (measurement point) on the surface of the object, thereby measuring the distance from the laser source to the point, the intensity of the reflected and returned electromagnetic waves, the relative speed of the object at the measurement point, etc. The electromagnetic waves of the laser radar 5 installed at a predetermined position (front) of the vehicle scan the surroundings (front) of the vehicle in the horizontal and vertical directions, thereby detecting the position, shape, relative movement speed, etc. of the object in front of the vehicle (moving objects such as other vehicles, road surfaces, structures, etc.). It should be noted that, hereinafter, the above three-dimensional space is represented by the X-axis along the travel direction of the vehicle, the Y-axis along the vehicle width direction of the vehicle, and the Z-axis along the height direction of the vehicle. Therefore, the above three-dimensional space is sometimes called XYZ space.

[0027] The actuator AC is a driving actuator for controlling the driving of the vehicle. When the driving source is an engine, the actuator AC includes a throttle actuator for adjusting the opening of the throttle valve of the engine (throttle opening). When the driving source is a driving motor, the driving motor is included in the actuator AC. The brake actuator that operates the braking device of the vehicle and the steering actuator that drives the steering device are also included in the actuator AC.

[0028] The controller 10 includes an electronic control unit (ECU). More specifically, the controller 10 includes a computer having a computing unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM (read-only memory) and a RAM (random access memory), and other peripheral circuits not shown in the figure, such as an I / O (input / output) interface. It should be noted that multiple ECUs with different functions, such as an engine control ECU, a travel motor control ECU, and a brake device ECU, can be separately set, but for convenience, Figure 1 The controller 10 is shown in FIG. 1 as a collection of these ECUs.

[0029] The storage unit 12 stores high-precision detailed map information (referred to as high-precision map information). The high-precision map information includes: road location information, road shape (curvature, etc.) information, road slope information, intersection and fork location information, lane number (driving lane) information, lane width and each lane location information (lane center location, lane location boundary line information), location information of landmarks (signals, signs, buildings, etc.) marked on the map, and road surface contour information such as road surface bumps. In addition, the storage unit 12 also stores various control programs, thresholds used in the programs, and setting information for vehicle-mounted detectors such as the laser radar 5.

[0030] The computing unit 11 includes a data acquisition unit 111, an estimation unit 112, a speed calculation unit (hereinafter referred to as a calculation unit) 113, a classification unit 114, an extraction unit 115, a detection unit 116, a determination unit 117, a vector calculation unit 118, and a driving control unit 119 as functional structures. Figure 1 As shown in FIG. 1 , the data acquisition unit 111, the estimation unit 112, the speed calculation unit 113, the classification unit 114, the extraction unit 115, the detection unit 116, the determination unit 117, and the vector calculation unit 118 are included in the object detection device 50. The data acquisition unit 111, the estimation unit 112, the speed calculation unit 113, the classification unit 114, the extraction unit 115, the detection unit 116, the determination unit 117, and the vector calculation unit 118 included in the object detection device 50 will be described in detail later.

[0031] In the automatic driving mode, the driving control unit 119 generates a target trajectory based on the external conditions around the vehicle including the size, position, relative moving speed, etc. of the object detected by the object detection device 50. Specifically, the driving control unit 119 generates a target trajectory in a manner to avoid collision, contact, or following the object based on the size, position, relative moving speed, etc. of the object detected by the object detection device 50. The driving control unit 119 controls the actuator AC so that the vehicle travels along the target trajectory. Specifically, the actuator AC is controlled along the target trajectory to adjust the accelerator opening, drive the brake device, or the steering device. It should be noted that in the manual driving mode, the driving control unit 119 controls the actuator AC based on the driving instructions (steering operation, etc.) from the driver obtained by the internal sensor group 3.

[0032] The object detection device 50 is described in detail. As described above, the object detection device 50 includes the data acquisition unit 111, the estimation unit 112, the speed calculation unit 113, the classification unit 114, the extraction unit 115, the detection unit 116, the determination unit 117, and the vector calculation unit 118. The object detection device 50 further includes the laser radar 5.

[0033] The data acquisition unit 111 acquires four-dimensional data (hereinafter referred to as point cloud data) as detection data of the laser radar 5, and the four-dimensional data includes position information showing the three-dimensional position coordinates of the measurement point on the surface of the object that receives the reflected wave of the laser radar 5 and speed information showing the relative moving speed of the measurement point. The point cloud data is acquired by the laser radar 5 in units of frames, specifically, at each predetermined time (time interval determined by the frame rate of the laser radar 5).

[0034] As a method for detecting a moving object from the detection data (point cloud data) of the laser radar 5, there is a method of performing clustering processing on the point cloud data, classifying the point cloud data into measurement points corresponding to the moving object and other measurement points, thereby detecting the moving object. However, the point cloud data also includes information on the measurement points corresponding to the stationary object. Therefore, when the point cloud data obtained from the laser radar 5 is directly used for clustering processing, not only the measurement point group corresponding to the moving object, but also the measurement point group corresponding to the stationary object is also used as a classification object, so the calculation load in the clustering processing may increase. Therefore, in order to reduce the calculation load, it is possible to consider converting the relative moving speed shown by the speed information of each measuring point into an absolute speed (hereinafter referred to as the absolute moving speed), classifying the point cloud data into stationary point cloud data corresponding to the stationary object and moving point cloud data corresponding to the moving object according to the absolute moving speed, and performing clustering processing on the classified moving point cloud data.

[0035] However, due to the moving direction of the moving object, its relative moving speed cannot sometimes be measured by the laser radar 5. Figure 2 5 is a diagram for explaining the relationship between the relative moving speed of a moving object measured by the laser radar 5 and the moving direction of the moving object. Figure 2 In FIG. 1 , the hollow arrow lines indicate the state where the moving objects M1 and M2 move in the direction of the arrows. The solid arrow lines schematically indicate the state where the electromagnetic waves emitted by the laser radar 5 are reflected on the surfaces of the moving objects M1 and M2 and return. The dotted line CL1 is a virtual line connecting points at equal distances from the laser radar 5 mounted on the vehicle 101.

[0036] The laser radar 5 cannot detect the velocity in the vertical direction relative to the projection angle. Figure 2When moving on the dotted line CL1, its moving direction is perpendicular to the projection angle of the laser radar 5, so the relative moving speed of the moving objects M1 and M2 measured by the laser radar 5 is zero and unchanged. As a result, the moving objects M1 and M2 are detected as stationary objects rather than moving objects, and the measurement points corresponding to the moving objects M1 and M2 are classified as stationary point cloud data. Therefore, in the method of performing clustering processing only on the moving point cloud data, the moving objects M1 and M2 that move in the perpendicular direction relative to the projection angle of the laser radar 5 may be excluded from the objects of clustering processing. In this case, the moving objects M1 and M2 will be missed, and the moving objects M1 and M2 cannot be tracked with high precision. Therefore, in this embodiment, clustering processing of the point cloud data is performed as follows so that the calculation load can be reduced while detecting the moving objects with high precision.

[0037] Figure 3 It is shown by Figure 1 The flowchart is a flowchart of an example of a process executed by the CPU of the controller 10 according to a predetermined program. The process shown in the flowchart is repeated at a predetermined cycle during the startup of the object detection device 50. More specifically, it is repeated every time the detection data of the laser radar 5 is acquired by the data acquisition unit 111, that is, every predetermined time.

[0038] When the detection data (point cloud data) of the laser radar 5 is obtained, the process of classifying the point cloud data into moving point cloud data and stationary point cloud data is first performed (step S1). Next, the detection process of moving objects on the moving point cloud data (step S2) and the detection process of moving objects on the stationary point cloud data (steps S31 to S33) are performed in parallel. It should be noted that the detection process (step S2) and the detection process (steps S31 to S33) may not be performed in parallel, but after executing one side, the other side is executed. Next, for the detected moving object, the process of identifying the same object between frames (between the current frame and the previous frame) is performed (step S4), and finally, the process of calculating the motion vector of the moving object identified as the same object is performed (step S5).

[0039] In the detection process of moving objects on the moving point cloud data (step S2), clustering processing is performed on the moving point cloud data. In the detection process of moving objects on the stationary point cloud data (steps S31 to S33), first, in step S31, a prescribed scan matching process is performed to overlap the stationary point cloud data of the previous frame with the stationary point cloud data of the current frame, thereby estimating the azimuth angle difference and movement vector of the vehicle 101. The movement vector represents the moving direction of the representative point (center of gravity, etc.) of the vehicle 101 between frames and the moving speed in the moving direction. The azimuth angle difference is the angular difference between the azimuth of the vehicle 101 in the current frame and the azimuth (travel direction) in the previous frame. In the following, the axis along the travel direction of the moving body is set as the X-axis, and the axes in the lateral and height directions relative to the travel direction are set as the Y-axis and the Z-axis, respectively. It should be noted that the movement vector can be two-dimensional (X, Y) or three-dimensional (X, Y, Z). In addition, the azimuth angle difference can be a 1-axis angle (Z-axis rotation angle), a 2-axis angle (X-axis rotation angle, Z-axis rotation angle), or a 3-axis angle (X-axis rotation angle, Y-axis rotation angle, Z-axis rotation angle). In the scan matching process, ICP (Iterative Closest Point), NDT (Normal Distributions Transform), or other methods can be used. Next, in step S32, based on the result of the overlap of the static point cloud data of the previous frame and the static point cloud data of the current frame obtained by the scan matching process in step S31, the change points are extracted. Specifically, the distance difference between each measurement point of the previous frame corresponding to each other by overlap and each measurement point of the current frame is calculated, and the measurement points with the distance difference above the specified threshold are extracted as change points. Finally, in step S33, clustering processing is performed on the extracted measurement point group (change point group). It should be noted that the heading angle difference and movement vector of the host vehicle 101 estimated in step S31 are accumulated, and a own position estimation process (not shown) is performed to estimate the own position (traveling position of the host vehicle 101) based on the accumulated heading angle difference and movement vector.

[0040] Detailed description Figure 3 processing in each step.

[0041] <Classification of point cloud (S1)>

[0042] The estimating unit 112 estimates the absolute moving speed (speed vector of X, Y, and Z coordinates) of the host vehicle 101 based on the point cloud data acquired by the data acquiring unit 111. Here, the estimation of the absolute moving speed of the host vehicle 101 by the estimating unit 112 will be described.

[0043] The estimation unit 112 extracts point cloud data other than information of the measurement point corresponding to the three-dimensional object, that is, point cloud data corresponding to the road surface around the vehicle 101 (hereinafter referred to as road surface point cloud data) from the point cloud data acquired by the data acquisition unit 111. The estimation unit 112 calculates the unit vector ei indicating the direction of the relative moving speed vi based on the position coordinates (xi, yi, zi) included in the four-dimensional data (xi, yi, zi, vi) of the measurement point Pi (i=1, 2...n) corresponding to the road surface, that is, the road surface point cloud data. Specifically, the estimation unit 112 calculates the unit vector ei by the following formula (i).

[0044]

[0045] Next, the estimation unit 112 estimates the moving speed (absolute moving speed) Vself of the vehicle 101. Specifically, the estimation unit 112 sets the conversion formula for converting the relative moving speed vi of the measurement point Pi corresponding to the road surface into the absolute moving speed as the objective function L, and solves the optimization problem of optimizing the objective function L to approach zero. The measurement point Pi is a measurement point on the road surface, so the absolute speed of each of those measurement points should be zero. Therefore, by optimizing the objective function L to approach zero, the correct Vself can be estimated. Vself is shown in the following formula (ii), expressed as the velocity component in the XYZ axis. The objective function L is expressed by the following formula (iii). By solving the above-mentioned optimization problem, Vself can be obtained so that the right side of formula (iii) is zero. It should be noted that Vself can be set to zero as the initial value, or it can be set to the Vself estimated in the previous frame.

[0046] V self =(v x , v y , v z ) (ii)

[0047] L(V,f(A,V self ))=V+A·V self T (iii)

[0048] In formula (iii), A is a matrix of unit vectors ei of n measurement points corresponding to the road surface, which is represented by formula (iv). In addition, in formula (iii), V is a 1×n matrix representing the velocity components (relative moving speed) of n measurement points Pi corresponding to the road surface, which is represented by formula (v). The estimation unit 112 obtains Vself obtained by solving the above-mentioned optimization problem as an estimated value of the absolute moving speed of the host vehicle 101 in the current frame.

[0049]

[0050] V = [v1, v2, ..., v n ] T (v)

[0051] The speed calculation unit 113 calculates the absolute moving speeds of all the measurement points, more specifically, the absolute moving speeds of all the measurement points including the measurement points corresponding to the three-dimensional objects, based on the absolute moving speed Vself of the host vehicle 101 estimated by the estimation unit 112. Here, the calculated absolute moving speed is a negative value when approaching the host vehicle, and is a positive value when leaving the host vehicle.

[0052] The classification unit 114 classifies the point cloud data acquired by the data acquisition unit 111 into moving point cloud data corresponding to measurement points whose absolute values ​​of absolute moving speeds calculated by the speed calculation unit 113 are greater than or equal to a predetermined speed Th_V and stationary point cloud data corresponding to measurement points whose absolute moving speeds are less than the predetermined speed Th_V.

[0053] <Estimation of Movement Vector (S31)>

[0054] Figure 4 , Figure 5A , Figure 5B , Fig. 6A as well as Figure 6B 1 is a diagram for explaining the process of estimating the movement vector and the heading angle of the vehicle 101 between frames (step S31). Figure 4 In FIG. 1 , the arrow on the vehicle 101 indicates the moving direction (travel direction) of the vehicle 101, that is, the X-axis direction. Figure 4 ) and height direction ( Figure 4 , and , respectively, represent the Y-axis direction and the Z-axis direction. It should be noted that the definitions of the X-axis direction, the Y-axis direction, and the Z-axis direction are the same in other figures, so the markings of the X-axis, the Y-axis, and the Z-axis are omitted in those figures. Figure 4An example of a schematic diagram of the point cloud data obtained by the laser radar 5 when observed from above (Z-axis direction) at a past time point (time t1) is shown in FIG. Areas N1 to N6 schematically represent the measurement point group corresponding to the stationary object, and more specifically, the position and size of the measurement point group. Stationary objects include the road surface of the road on which the vehicle 101 is traveling, walls set on the side of the road, structures such as separation strips, and other vehicles parked on the shoulder of the road. Areas M31 and M32 schematically show the measurement point group corresponding to the moving object, and more specifically, the position and size of the measurement point group. It should be noted that the objects detected by the laser radar 5, including people, are referred to as objects below. Therefore, in addition to moving vehicles such as cars and bicycles, moving objects also include moving people (pedestrians, etc.). The arrows marked on areas M31 and M32 indicate the moving direction of the moving object.

[0055] exist Figure 5A It is schematically shown in Figure 4 Point cloud data is static point cloud data classified from the point cloud data acquired by the laser radar 5 at a past time point (time t1). Figure 5B Schematically shows the stationary point cloud data classified from the point cloud data acquired by the laser radar 5 at the current time point (time t2) after a predetermined time has passed from a predetermined time point in the past (time t1).

[0056] The estimation unit 112 first uses the static point cloud data of the previous frame ( Figure 5A ) and the static point cloud data of the current frame ( Figure 5B ) performs alignment and estimates the azimuth angle difference and movement vector of the vehicle 101 within a specified time (frame interval). Fig. 6A and Figure 6B The static point cloud data of the previous frame ( Figure 5A ) and the static point cloud data of the current frame ( Figure 5B ) is aligned. Fig. 6A , the static point cloud data of the previous frame is shown by the dotted line, and the static point cloud data of the current frame is shown by the solid line. The estimation unit 112 searches (estimates) the azimuth angle difference and the movement vector of the host vehicle 101 such that the measurement point group N1 to N6 of the previous frame shown by the dotted line overlaps (is consistent with) the measurement point group N1 to N6 of the current frame shown by the solid line. Figure 6B The static point cloud data of the previous frame and the static point cloud data of the current frame after alignment are shown in FIG. Figure 6BIn the figure, the angle MA represents the azimuth angle difference of the vehicle 101 between frames. The hollow arrow MV represents the movement vector of the vehicle 101 between frames. The white circle in the figure schematically represents the center of gravity of the vehicle 101. The estimation unit 112 solves the optimization problem of minimizing the deviation (error) between the positions of the measurement point group N1~N6 of the previous frame after alignment and the positions of the measurement point group N1~N6 of the current frame, and outputs the final search result (estimation result) of the azimuth angle difference and the movement vector of the vehicle 101. It should be noted that the above-mentioned alignment can also be performed after the three-dimensional point cloud data is converted into two-dimensional point cloud data represented by the XY coordinate system to reduce the calculation load.

[0057] <Extraction of change points (S32)>

[0058] Figure 7 and Figure 8 is a diagram for explaining the extraction process of the change point (step S32). Figure 7 2 shows another example of a schematic diagram when the point cloud data acquired by the laser radar 5 at a past time point (time t1) is viewed from above (Z-axis direction). Figure 7 The point cloud data shown is Figure 4 The point cloud data shown is the same, but Figure 7 The point cloud data shown includes a measurement point group (area M33) corresponding to a moving object moving in a direction perpendicular to the projection angle of the laser radar 5. Figure 7 In FIG. 1 , the shaded area indicates the measurement point group classified as the stationary point cloud data. As mentioned above, the laser radar 5 cannot detect the velocity in the vertical direction relative to the projection angle. Figure 7 As shown in FIG. 1 , the object corresponding to the region M33 is mistakenly detected as a stationary object instead of a moving object. Therefore, the extraction unit 115 calculates the change amount (movement amount) of the position of each measurement point of the stationary point cloud data between frames (predetermined time), and extracts the measurement points whose change amount is greater than a predetermined threshold as change points, so that the object corresponding to the region M33 is detected as a moving object. Figure 8 The state of extracting change points from static point cloud data is shown in FIG. Figure 8 In the figure, the dotted line shows the static point cloud data of the previous frame ( Figure 7 ), and the solid line shows the static point cloud data of the current frame. In addition, Figure 8 1 shows a state where the point cloud data of the previous frame and the point cloud data of the current frame are aligned using the movement vector and the azimuth angle difference of the host vehicle estimated by the estimation unit 112. Figure 8As shown in FIG. 1 , the position of the measurement point group M33 corresponding to the moving object that is mistakenly detected as a stationary object changes greatly between frames compared with the other measurement point groups N1 to N6. Therefore, each measurement point of the measurement point group M33 is extracted as a change point by the extraction unit 115. Hereinafter, the point cloud data corresponding to the change point (change point group) extracted by the extraction unit 115 is referred to as change point cloud data.

[0059] <Clustering Process (S2, S33)>

[0060] In step S2, the detection unit 116 performs clustering processing on the moving point cloud data (the moving point cloud data classified from the point cloud data of the current frame in step S1). In addition, in step S33, the detection unit 116 performs clustering processing on the change point cloud data extracted from the static point cloud data (the static point cloud data classified from the point cloud data of the current frame in step S1). Thus, the bounding boxes (circumscribed areas) corresponding to the measurement point groups M31 and M32, respectively, are detected from the moving point cloud data, and the bounding box corresponding to the measurement point group M33 is detected from the change point cloud data. The detection unit 116 detects the position and size of the detected bounding box as the position and size of the moving object. In this way, the moving object contained in the three-dimensional space around the vehicle 101 is detected. The detection unit 116 outputs information (image information, etc.) showing the detection result of the moving object to a display device, etc., not shown in the figure. It should be noted that in the clustering process performed by the detection unit 116, any method such as DBSCAN (Density-based spatial clustering of applications with noise) and k-MEANS method (k-means clustering algorithm) may be used.

[0061] <Identification of the same object (S4)>

[0062] The determination unit 117 determines whether the moving object detected from the previous frame and the moving object detected from the current frame are the same object.

[0063] Specifically, first, the determination unit 117 performs an offset rotation process of offsetting (parallel translation) and rotating each bounding box detected by the clustering process performed by the detection unit 116 in the previous frame, based on the motion vector and azimuth angle difference between the frames of the vehicle 101 estimated in step S31. In the offset rotation process, the determination unit 117 first offsets each bounding box detected in the previous frame according to the above motion vector and rotates the above azimuth angle difference. Next, the determination unit 117 further offsets each offset and rotated bounding box according to the motion vector of the corresponding moving object. Specifically, the bounding boxes of the measurement point groups M31, M32, and M33 are further offset by a motion amount obtained by multiplying the vector quantity of the motion vector of the moving object corresponding to the measurement point groups M31, M32, and M33 by the frame time (prescribed time). The motion vector of the moving object will be described later.

[0064] As described above, the determination unit 117 overlaps each bounding box of the previous frame that has undergone the offset rotation processing based on the movement vector and the azimuth angle difference of the vehicle 101 and the offset processing based on the movement vector of the moving object with the current frame. As a result of the overlap, when a bounding box that is repeated with the bounding box of the previous frame that has undergone the offset rotation processing and the offset processing exists in the current frame, the determination unit 117 determines that the moving objects corresponding to the repeated bounding boxes are the same object. It should be noted that the determination of whether the bounding boxes are repeated can be made based on whether the repetition rate is above a specified threshold value, or based on whether the distance between the centers of gravity of the bounding boxes is less than a specified length.

[0065] <Calculation of movement vector (S5)>

[0066] The vector calculation unit 118 calculates the motion vector of the moving object based on the determination result of the determination unit 117. Fig. 9 The calculation of the motion vector of a moving object is described below. Fig. 9 A diagram for explaining the motion vector of a moving object. Fig. 9 In the figure, the dotted line shows the point cloud data of the previous frame, and the solid line shows the point cloud data of the current frame. Fig. 9The state in which the point cloud data of the previous frame and the point cloud data of the current frame are overlapped after the offset rotation processing based on the movement vector and the azimuth angle difference of the vehicle 101 is performed is shown. The white circles G1, G2, and G3 represent the centroids of the measurement point groups M31, M32, and M33, and more specifically, represent the centroids of the bounding boxes of the measurement point groups M31, M32, and M33 detected by the clustering processing of the detection unit 116. The vector calculation unit 118 calculates the movement vector of the moving object corresponding to the measurement point groups M31, M32, and M33 based on the positional relationship between the centroids G1, G2, and G3 between frames (between the previous frame and the current frame). The vector calculation unit 118 stores the calculated movement vector of the moving object together with the information (identifier) ​​that can identify the moving object in the storage unit 12. The movement vector of the moving object stored in the storage unit 12 is used for the next offset processing based on the movement vector of the moving object, that is, for the offset processing in step S4 performed on the next frame. As a result, in the next frame, the same object can be identified with high accuracy, and the position, moving direction, and moving speed of the moving object can be well tracked.

[0067] Fig.10 1 is a diagram showing an example of information indicating a detection result of a moving object output by the detection unit 116. It should be noted that Fig.10 The image IM captured by the camera 4 is superimposed with image information of bounding boxes BB1 to BB5 of moving objects (pedestrians P1 to P5 moving in different directions) detected by the clustering process of the detection unit 116, as information showing the detection result of the moving object. The image IM captured by the camera 4 includes stationary objects such as buildings in addition to the pedestrians P1 to P5, but Fig.10 In order to simplify the drawings, those drawings are omitted. It should be noted that, instead of the captured image IM of the camera 4, image information of the bounding box of the moving object superimposed on the image showing the detection data of the laser radar 5 can be output as information showing the detection result of the moving object.

[0068] Fig.10 The detection result of the moving object from time t11 to time t13 is shown in FIG. The dotted line CL2 in the figure is a virtual line connecting points at equal distances from the laser radar 5. The dotted line CL2 is for illustration and is not actually included in the detection result of the moving object. In this embodiment, as described above, clustering is also performed on the change point cloud data extracted from the static point cloud data, so Fig.10 Moving objects that move perpendicularly to the projection angle of the laser radar 5, such as the pedestrian P1, are also included, and moving objects around the vehicle 101 can be tracked without omission.

[0069] The above-described embodiment has the following effects.

[0070] (1) The object detection device 50 comprises: a laser radar 5 mounted on the vehicle 101, irradiating electromagnetic waves around the vehicle 101, and detecting the external conditions around the moving body based on the reflected waves; a data acquisition unit 111, which acquires point cloud data at each predetermined time indicating the detection result of the laser radar 5, wherein the point cloud data includes position information of a measurement point on the surface of the object from which the reflected waves are obtained and speed information indicating the relative moving speed of the measurement point (referred to as first speed information); an estimation unit 112, which acquires speed information indicating the absolute moving speed of the vehicle 101 (referred to as second speed information); and a speed calculation unit 113, which calculates the speed of the vehicle 101 based on the first speed information. The method comprises the following steps: a first step is to calculate the absolute moving speed of each of the plurality of measuring points corresponding to the point cloud data by using the speed information and the second speed information; a classification unit 114, which classifies the point cloud data into moving point cloud data corresponding to the measuring points whose absolute moving speed calculated by the speed calculation unit 113 is greater than the specified speed and stationary point cloud data other than the moving point cloud data; an extraction unit 115, which calculates the movement amount of the stationary point cloud data in a specified time, and extracts the change point cloud data corresponding to the measuring points whose movement amount is greater than the specified threshold value from the stationary point cloud data; and a detection unit 116, which detects the moving objects around the moving body based on the moving point cloud data and the change point cloud data. The detection unit 116 performs the specified clustering processing on the moving point cloud data and the change point cloud data, detects the circumscribed area of ​​the object from the moving point cloud data and the change point cloud data, respectively, and detects the position and size of the object based on the position and size of the circumscribed area. Thus, the processing load can be reduced while detecting the moving object with high precision. In addition, the moving object that moves in the vertical direction relative to the projection angle of the laser radar 5 can also be detected with high precision.

[0071] (2) The object detection device 50 includes: an estimation unit 112, which, as an information acquisition unit, acquires first movement information showing the movement state of the vehicle 101 from the past time point to the current time point based on the stationary point cloud data at the current time point and the stationary point cloud data at the past time point relative to the current time point; a storage unit 12, which stores second movement information showing the movement state of the moving object from the past time point to the current time point; and a determination unit 117, which executes, based on the first movement information acquired by the estimation unit 112 and the second movement information stored in the storage unit 12, to convert the moving point cloud data from the past time point acquired by the detection unit 116 into the moving point cloud data from the past time point. The apparatus 110 is a method for determining whether the mobile object corresponding to the first external area and the mobile object corresponding to the second external area are the same object according to the overlap degree of the first external area and the second external area after the alignment; and the vector calculation unit 118, which acts as an updating unit, updates the second movement information corresponding to the mobile object determined to be the same object by the determination unit 117 and stored in the storage unit 12 according to the positions of the first external area and the second external area. Thus, even when the mobile objects around the vehicle 101 include mobile objects that move in the vertical direction relative to the projection angle of the laser radar 5, the mobile objects around the vehicle 101 can be tracked with high accuracy without omission.

[0072] (3) The estimation unit 112 estimates the absolute moving speed of the vehicle 101 based on the position information and speed information of the representative measurement point extracted from the point cloud data obtained by the data acquisition unit 111, and obtains the estimation result as speed information. The representative measurement point is selected from the measurement points remaining after removing the measurement points corresponding to the three-dimensional object from the plurality of measurement points. Thus, the absolute moving speed of the vehicle 101 is estimated with the measurement points corresponding to the road surface as a reference. As a result, the absolute moving speed of the vehicle 101 can be estimated with high accuracy. In addition, since the absolute moving speed of the vehicle (moving body) 101 is estimated and obtained without relying on the sensor value of the vehicle speed sensor, etc., the present invention can also be applied to self-propelled robots, etc. that do not have a vehicle speed sensor, etc.

[0073] The above-mentioned embodiment can be modified into various forms. The following is an explanation of the modified example. In the above-mentioned embodiment, the laser radar 5 as a detector is mounted on a vehicle, irradiates electromagnetic waves into the three-dimensional space around the vehicle, and detects the external conditions around the vehicle based on the reflected waves. However, the detector can also be a device other than a laser radar such as a radar. In addition, the mobile body equipped with the detector can also be a mobile body other than a vehicle such as a self-propelled robot.

[0074] In addition, in the above-mentioned embodiment, the estimation unit 112 as the speed acquisition unit selects the measurement point Pi as the representative measurement point from the measurement points remaining after removing the measurement point corresponding to the three-dimensional object from the plurality of measurement points, and estimates the absolute moving speed of the host vehicle 101 based on the position information and speed information of the representative measurement point extracted from the point cloud data acquired by the data acquisition unit 111, and acquires the estimation result as the second speed information. However, the speed acquisition unit may also acquire the measurement result of the absolute moving speed of the host vehicle 101 acquired by the measuring device included in the internal sensor group 3 as the second speed information. In this case, the object detection device 50 includes at least the vehicle speed sensor in the internal sensor group 3 as the measuring device. In addition, the speed acquisition unit may also calculate and acquire the absolute moving speed of the host vehicle 101 based on the current position of the host vehicle 101 measured by the positioning unit 2. In this case, the object detection device 50 includes the positioning unit 2.

[0075] In addition, in the above-mentioned embodiment, the detection unit 116 performs a predetermined clustering process on the three-dimensional point cloud data (moving point cloud data and changing point cloud data) to detect the moving object from the three-dimensional point cloud data. However, the detection unit can also project each measurement point corresponding to the three-dimensional point cloud data onto the XY plane to convert it into two-dimensional point cloud data, and generate speed-added data (XYV data) in which the absolute moving speed of each measurement point calculated by the speed calculation unit 113 is added to the two-dimensional point cloud data, and perform a predetermined clustering process on the speed-added data. Thus, a clustering process that takes into account the position and moving speed of the moving object is performed. As a result, it is possible to suppress a plurality of moving objects that are close to each other, such as two moving objects that pass by each other, from being detected as one (as one moving object), and the detection accuracy of the moving object can be further improved. It should be noted that, in the case where the accuracy of the cluster size in the three-dimensional space (XYZ space) is required, the detection unit can also generate speed-added data (XYZV data) in which the absolute moving speed of each measurement point calculated by the speed calculation unit 113 is added to the three-dimensional point cloud data, and perform a predetermined clustering process on the speed-added data.

[0076] In addition, in the above-mentioned embodiment, the estimation unit 112 as the information acquisition unit acquires the azimuth angle difference and the movement vector of the host vehicle 101 as the first movement information showing the movement state of the host vehicle 101 from the past time point to the current time point, but the information acquisition unit may also acquire other information as the first movement information. In addition, in the above-mentioned embodiment, the storage unit 12 stores the movement vector of the mobile object as the second movement information showing the movement state of the mobile object from the past time point to the current time point. However, the second movement information may also be other information.

[0077] In the above-mentioned embodiment, the estimating unit 112 as the information acquiring unit obtains the static point cloud data ( Figure 5A ) and the static point cloud data of the current frame ( Figure 5B ) to perform alignment, thereby estimating the azimuth angle difference and movement vector of the vehicle 101 at a specified time (between frames). However, the information acquisition unit can also perform the above alignment after converting the three-dimensional static point cloud data into two-dimensional static point cloud data represented by an XY coordinate system. In addition, the information acquisition unit can also use not only the static point cloud data of a past time point (time t1) but also the static point cloud data of multiple time points in the past, that is, not only the static point cloud data of the previous frame but also the static point cloud data of multiple frames in the past to perform the above alignment. In this way, by using the static point cloud data of multiple frames in the past, even when the stationary object in front of the vehicle 101 is temporarily blocked by other vehicles, etc., the above alignment can be performed well, which can improve the robustness.

[0078] In addition, in the above-mentioned embodiment, the driving control unit 119 controls the driving of the host vehicle 101 so as to avoid collision or contact with the object detected by the detection unit 116. However, the driving control unit 119 may also serve as a notification unit to predict the possibility of collision or contact with the moving object based on the size, position, and moving speed of the moving object detected by the detection unit 116. Furthermore, when the possibility of collision or contact with the moving object is above a predetermined level, the information (image information, sound information) to be noted about the collision or contact with the moving object detected by the detection unit 116 may be notified to the occupants of the host vehicle 101 via a display or speaker (not shown) provided by the vehicle control device 100.

[0079] In addition, in the above embodiment, the object detection device 50 is applied to an autonomous vehicle, but the object detection device 50 can also be applied to vehicles other than autonomous vehicles. For example, the object detection device 50 can also be applied to a manually driven vehicle equipped with an ADAS (Advanced driver-assistance system).

[0080] The above description is merely an example, and the above embodiment and modified examples do not limit the present invention unless the features of the present invention are impaired. One or more of the above embodiment and modified examples can be arbitrarily combined, and the modified examples can be combined with each other.

[0081] By adopting the present invention, it is possible to detect moving objects with high accuracy while reducing the processing load.

[0082] The present invention has been described above in conjunction with preferred embodiments, but it should be understood by those skilled in the art that various modifications and changes can be made without departing from the scope of the claims.

Claims

1. An object detection device, characterized in that: have: A detector (5) is mounted on a moving body, irradiates electromagnetic waves around the moving body, and detects external conditions around the moving body based on reflected waves; a data acquisition unit (111) for acquiring point cloud data at each predetermined time indicating the detection result of the detector (5), the point cloud data including position information of a measurement point on the surface of the object from which the reflected wave is obtained and first speed information indicating a relative moving speed of the measurement point; a speed acquisition unit that acquires second speed information indicating an absolute moving speed of the moving object; a calculation unit (113) for calculating the absolute moving speed of each of a plurality of measurement points corresponding to the point cloud data based on the first speed information and the second speed information; a classification unit (114) for classifying the point cloud data into moving point cloud data corresponding to the measurement point whose absolute value of the absolute moving speed calculated by the calculation unit (113) is greater than a predetermined speed and stationary point cloud data other than the moving point cloud data; an extraction unit (115) for calculating the movement amount of the stationary point cloud data in the specified time, and extracting the change point cloud data corresponding to the measurement point where the movement amount is greater than a specified threshold value from the stationary point cloud data; and A detection unit (116) detects the object around the moving body based on the movement point cloud data and the change point cloud data.

2. The object detection device according to claim 1, characterized in that: The detection unit (116) performs a prescribed clustering process on the moving point cloud data and the changing point cloud data, detects the circumscribed area of ​​the object from the moving point cloud data and the changing point cloud data respectively, and detects the position and size of the object based on the position and size of the circumscribed area.

3. The object detection device according to claim 2, characterized in that: The position information is three-dimensional position information, The detection unit (116) converts the three-dimensional position information of each measuring point corresponding to the moving point cloud data and the changing point cloud data into two-dimensional position information, generates speed-added data by adding the absolute moving speed corresponding to each measuring point calculated by the calculation unit (113) to the two-dimensional position information of each measuring point obtained by the conversion, and performs the prescribed clustering processing on the speed-added data.

4. The object detection device according to claim 2, characterized in that: Also available: an information acquisition unit that acquires first movement information showing a movement state of the moving body from the past time point to the current time point based on the stationary point cloud data at the current time point and the stationary point cloud data at the past time point relative to the current time point; a storage unit (12) storing second movement information showing a movement state of the object from the past time point to the current time point; a determination unit (117) for performing an alignment so that a first circumscribed area detected by the detection unit (116) from the movement point cloud data and the change point cloud data at the past time point overlaps with a second circumscribed area detected by the detection unit (116) from the movement point cloud data and the change point cloud data at the current time point, based on the first movement information acquired by the information acquisition unit and the second movement information stored in the storage unit (12), and determining whether the object corresponding to the first circumscribed area and the object corresponding to the second circumscribed area are the same object based on the degree of overlap between the first circumscribed area and the second circumscribed area after the alignment; and An updating unit (118) updates the second movement information corresponding to the object determined by the determining unit (117) to be the same object and stored in the storage unit (12) based on the positions of the first circumscribed area and the second circumscribed area.

5. The object detection device according to claim 1, characterized in that: The speed acquisition unit estimates the absolute moving speed of the moving body based on the position information of the representative measurement point extracted from the point cloud data acquired by the data acquisition unit (111) and the first speed information, and acquires the estimation result as the second speed information. The representative measurement point is selected from the measurement points remaining after removing the measurement point corresponding to the three-dimensional object from the plurality of measurement points.

6. The object detection device according to claim 1, characterized in that: A measuring device is also provided, the measuring device measures the absolute moving speed and moving direction of the moving body, The speed acquisition unit acquires a measurement result of the measurement device as the second speed information.

7. The object detection device according to any one of claims 1 to 5, characterized in that: The detector (5) is a laser radar.

Images